It seems inelegant to me that utility functions are created for specific situations, while these clearly aren't the same as that of the agent in total among all of their decisions. For instance, a model may estimate an agent's expected utility from the result of a specific intervention, but this clearly isn't quite right; the agent has a much more complicated utility function outside this intervention. According to a specific model, "Not having an intervention" could set "Utility = 0"; but for any real agent, it's quite likely their life wouldn't actually have 0 utility without the intervention.
It seems like it's important to distinguish that a utility score in a model is very particular to the scenario for that model, and does not represent a universal utility function for the agents in question.
Let U be an agent's true utility function across a very wide assortment of possible states, and ^U be the utility function used for the sake of the model. I believe that ^U is supposed to approximate U in some way; perhaps they should be related by an affine transformation.
The important thing for a utility function, as it is typically used (in decision models), is probably not that ^U=U, but rather, that decisions made within the specific context of ^U approximate those made using U.
Here, I use brackets to describe "The expected value, according to a utility function", and D to describe the set of decisions made conditional on a specific utility function being used for decision making.
Then, we can represent this supposed estimation with:
Oh fantastic, thanks for the reference!