This is a linkpost for https://nibnalin.me/dust-nib/bellmans-curse-of-dimensionality.html

In dynamic programming and reinforcement learning, the most fundamental challenge we face is just grokking the high dimensionality of a state space. Go is a "simple" game whose rules can be fully explained on a single sheet of paper, yet those rules yield more board configurations than the number of atoms in the universe ( $10^{174}$ ). Given the state space, how does any solver for this game even represent these configurations, let alone evaluate them? While games are helpful toys to start with, this phenomenon of “exponential explosion” in systems is extremely common in many real-world problems, where we’ve given it the stage name: Bellman’s Curse of Dimensionality.

While popular study of dimensionality is usually focused on tight, mathematical systems, I’ve found that thinking about dimensionality as a metaphor in other, less well-defined systems is a surprisingly helpful tool.

Advice, for instance, tends to fit such a model very naturally. You have an extremely large state space (life) where you only get one opportunity to traverse a path. To add to that, like the typical exponential problem, as dimensionality of life increases, the volume of the space of possibilities increases so fast that the available data on historical outcomes becomes quite sparse. How, then, can someone evaluate others’ advice, or look at someone else’s life to figure out what they should do with their own?

I don't claim to have a solution to this problem, but in my experience it's helpful to keep the dimensionality framework suggested by Bellman in the back of your mind as you navigate advice. For starters, thinking about the dimensions in this space independently is helpful to evaluate different states clearly:

Time: Your professor probably don’t have the best advice for you because their own experience about your situation was frozen in 1980s, and a lot about the world has changed since then.
Space: Advice about fundraising from a US entrepreneur is obviously not as relevant for someone raising in India or China.
Internal state: This is one of the harder categories to pin down. One way to think about these is as some function of an individual's personality and thought processes, but I think that only scratches the surface of what this broad category intends to cover. If someone attempts to use Steve

...

Posts

Wikitag Contributions

Comments