Does the Utility Function Halt?

by OrphanWilde 1 min read28th Jan 20159 comments


Suppose, for a moment, that somebody has written the Utility Function.  It takes, as its input, some Universe State, runs it through a Morality Modeling Language, and outputs a number indicating the desirability of that state relative to some baseline, and more importantly, other Universe States which we might care to compare it to.

Can I feed the Utility Function the state of my computer right now, as it is executing a program I have written?  And is a universe in which my program halts superior to one in which my program wastes energy executing an endless loop?

If you're inclined to argue that's not what the Utility Function is supposed to be evaluating, I have to ask what, exactly, it -is- supposed to be evaluating?  We can reframe the question in terms of the series of keys I press as I write the program, if that is an easier problem to solve than what my computer is going to do.