# 5-and-10

Let there be a decision problem which involves the choice between \$5 and \$10, a utility function that values the \$10 more than the \$5, and algorithm A optimizing for this utility function.

Another version, sometimes known as the heavy ghost problem, raises a difficulty with certain types of UDT-UDT-like decision theories, when the fact that a counterfactual is known to be false makes the algorithm implement it.

The 5-and-10 problem addresses the question of how to construct a theory of logical counterfactuals.counterfactuals.

Created by Jake Miller at 4y

## References

The algorithm A that reasons something like:

Let there be a decision problem which involves the choice between \$5 and \$10, a utility function that values the \$10 more than the \$5, and algorithm A optimizing for this utility function.

Another version, sometimes known as the heavy ghost problem, israises a problem indifficulty with certain types of UDT-like decision theories, when the fact that a counterfactual is known to be false makes the algorithm implement it.

Specifically, let there be a decision problem which involves the choice between \$5 and \$10, a utility function that values the \$10 more than the \$5, and anThe algorithm A that reasons something like:

The 5-and-10 problem addresses the question of how to construct a theory of logical counterfactuals.

Another version, sometimes known as the heavy ghost problem, is a problem in certain types of UDT-like decision theories, when the fact that a counterfactual is known to be false makes the algorithm implement it.

Another version,version, sometimes known as the heavy ghost problem, is a problem in certain types of UDT-like decision theories, when the fact that a counterfactual is known to be false makes the algorithm implement it.

The five-One version of the 5-and-ten10 problem (sometimes is "I have to decide between \$5 and \$10. Suppose I decide to choose \$5. I know that I'm a money-optimizer, so if I do this, \$5 must be more money than \$10, so this alternative is better. Therefore, I should choose \$5."

Another version, sometimes known as the heavy ghost problem)problem, is a problem in certain types of UDT-like decision theories, when the fact that a counterfactual is known to be false makes the algorithm implement it.

The five-and-ten problem (sometimes known as the heavy ghost problem) is a problem in certain types of UDT-like decision theories, when the fact that a counterfactual is known to be false makes the algorithm implement it.

Specifically, let there be a decision problem which involves the choice between \$5 and \$10, a utility function that values the \$10 more than the \$5, and an algorithm A that reasons something like:

`"Look at all proposition of the type '(A decides to do X) implies (Utility=y)', and find the X that maximises y, then do X."`

When faced with the above problem, certain types of algorithm can reason:

`"The utility of \$10 is greater than the utility of \$5. Therefore I will never decide to choose \$5. Therefore (A decides to do 'choose \$5') is a false statement. `
`Since a false statement implies anything, (A decides to do 'choose \$5') implies (Utility=y) for any, arbitrarily high, value of y. `
`Therefore this is the utility maximising decision, and I should choose \$5."`

That is the informal, natural language statement of the problem. Whether the algorithm is actually vulnerable to the 5-and-10 problem depends on the details of what the algorithm is allowed to deduce about itself.