Utility functions as a concept bypass the distinction of terminal vs instrumental value. They just put a score on the state of the world. You only need "terminal value" as a modeling tool when abstracting things to fit into fairly limited planning brains.
I think that you are. Consider a chess program whose utility function is something like the ease with which one can lead the opponent's king to be checkmated, while the next move optimizes the function, but the opponent's move is very unlikely to decrease it. According to the program, the chessboard ought to enter a state where the program won. The UF is in the program, not in the chessboard. However, I might've explained this far poorer than Yudkowsky did in at least a couple of posts.
Excerpt from Terminal Value:
For me it seems to violate Hume's is-ought distinction.
Utility function is is.
Terminal value is ought.
Am I misunderstanding something?