DragonGod's Shortform

DragonGod

This is a special post for quick takes by DragonGod. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

What are the best arguments that expected utility maximisers are adequate (descriptive if not mechanistic) models of powerful AI systems?

[I want to address them in my piece arguing the contrary position.]

If you define your utility function over histories, then every behaviour is maximising an expected utility function no?

Even behaviour that is money pumped?

I mean you can't money pump any preference over histories anyway without time travel.

The Dutchbook arguments apply when your utility function is defined over your current state with respect to some resource?

I feel like once you define utility function over histories, you lose the force of the coherence arguments?

What would it look like to not behave as if maximising an expected utility function for a utility function defined over histories.

4Garrett Baker3y

I like Utility Maximization = Description Length Minimization.

4Garrett Baker3y

Make sure you also read the comments underneath, there are a few good discussions going on, clearing up various confusions, like this one: [...]