LESSWRONG
LW

Rationality
Frontpage

28

[ Question ]

What is a decision theory as a mathematical object?

by Erik Jenner
25th May 2020
1 min read
A
3
4

28

Rationality
Frontpage

28

What is a decision theory as a mathematical object?
5Alexander Gietelink Oldenziel
2Chris_Leong
2Charlie Steiner
3Pattern
New Answer
New Comment

3 Answers sorted by
top scoring

Alexander Gietelink Oldenziel

May 25, 2020

50

Fundamentally, finding a good mathematical definition of decision theory that encompasses all the phenomena people care about is a big open problem.

Add Comment

Chris_Leong

Feb 12, 2025

20

My take: Counterfactuals are Confusing because of an Ontological Shift:

"In our naive ontology, when we are faced with a decision, we conceive of ourselves as having free will in the sense of there being multiple choices that we could actually take. These choices are conceived of as actual and we when think about the notion of the "best possible choice" we see ourselves as comparing actual possible ways that the world could be. However, we when start investigating the nature of the universe, we realise that it is essentially deterministic and hence that our naive ontology doesn't make sense. This forces us to ask what it means to make the "best possible choice" in a deterministic ontology where we can't literally take a choice other than the one that we make. This means that we have to try to find something in our new ontology that roughly maps to our old one."

We expect a straightforward answer to "What is a decision theory as a mathematical object?", since we automatically tend to assume our ontology is consistent, but if this isn't the case and we actually have to repair our ontology, it's unsurprising that we end up with different kinds of objects.

Add Comment

Charlie Steiner

May 27, 2020

20

I think the most fundamental thing might be taking in a sequences of bits (or distribution over sequences if you think it's important to be analog) and outputting bits (or, again, distributions) that happen to control actions.

All this talk about taking causal models as an input is merely a useful abstraction of what happens when we do sequence prediction in our causal universe, and it might always be possible to find some plausible excuse to violate this abstraction.

Add Comment
1 comment, sorted by
top scoring
Click to highlight new comments since: Today at 4:43 AM
[-]Pattern5y30

The most structure might be a lookup table*, but moving away from that requires information about how the different situations relate to each other.

*RL seems related, and a missing aspect from most.

Reply
Moderation Log
Curated and popular this week
A
3
1

If we want a space of all decision theories, what mathematical objects does it contain? For example, if a decision theory is a function, what are its domain and codomain?

The only approach I'm familiar with is to view expected utility maximizing decision theories as ways of building counterfactuals (section 5 in the FDT paper). A decision theory could then be described as a function that takes in a state s and an action a and spits out a distribution over world states that result from counterfactually taking action a in state s.

But EDT, CDT and FDT require different amounts and kinds of structure in the description of the state s they take as input (pure probability distributions, causal models and logical models respectively), so this approach only works if there is some kind of structure that is sufficient for all decision theories we might come up with at some point.