Tyler Cowen has a new free book about the history of marginalism and how ideas arise generally, and something something AI something (I don't particularly expect to find what Cowen has to say about AI that interesting).

Early on, he draws a contrast between intuitive marginalism (the usual intuitive application as seen in those trained in economics discussing an application) and tautological marginalism, where for example someone apparently not calculating decisions at the margin is seen as believing the marginal utility of calculating further is lower than the marginal cost of thinking longer. Those who have read some of the posts on coherence/selection theorems may know of a similar issue - we can interpret a rock as having 'preferences' over histories and putting a high utility on whatever it actually does.

Cowen states:

One way of reconciling the two marginalisms is to view the tautology approach as “the true marginalism,” and to view the clever examples [from intuitive marginalism] as “marginalism plus a complementary empirical hypothesis about what the relevant margin is.”

This is essentially the same stipulation as what I've seen John Wentworth add: there is an implicit hypothesis that the preferences are over some, but not all, of the variables, and that what we observe is a consequence of having preferences that are "sparse" over the world in some sense, or "factorize well", or something like that. Intuitively, highly detailed preferences over the universe history that cannot be factorized into caring about variables like "money" or even "fun experience-moments" are not something actual agents are likely to have - at the very least, they have relatively small minds as compared to the world.