Utility Functions

Applied to Ethodynamics of Omelas by dr_s 6d ago
Applied to money ≠ value by RobertM 1mo ago
Applied to Freedom Is All We Need by Leo Glisic 2mo ago

A utility function assigns numerical values ("utilities") to outcomes, in such a way that outcomes with higher utilities are absolutely always preferred to outcomes with lower utilities.utilities, with no exceptions; the lack of exploitable holes in the preference ordering is necessary for the definition and separates utility from mere reward.

A utility function assigns numerical values ("utilities"("utilities") to outcomes, in such a way that outcomes with higher utilities are always preferred to outcomes with lower utilities.

Utility Functions do not work very well in practice for individual humans. Human drives are not coherent nor is there any reason to think they would beconverge to a utility-function-grade level of reliability (Thou Art Godshatter), and even people with a strong interest in the concept have trouble working out what their utility function actually is even slightly (Post Your Utility Function). Furthermore, humans appear to calculate utilityreward and disutilityloss separately - adding one to the other does not predict their behavior accurately.accurately, and thus human reward is not human utility. This makes humans highly exploitable.exploitable - and in fact, not being exploitable would be a minimum requirement in order to qualify as having a coherent utility function.

pjeby posits humans'humans' difficulty in understanding their own utility functions as the root of akrasia.

Because utility functions arise from VNM rationality, they may still be of note in understanding intelligent systems even when the system does not explicitly store a utility function anywhere, since reducing exploitable error rate should eventually converge to utility-function-like guarantees.

Applied to Orthogonality is expensive by beren 2mo ago