A theory of physics is mathematically quite similar to a cellular automaton. This theory will usually be incomplete, something that we can represent in infra-Bayesianism by Knightian uncertainty. So, the "cellular automaton" has underspecified time evolution.

What evidence is there that incomplete models with Knightian uncertainty are a way to turn rough models of physics into loss functions? Can the ideas behind it be applied to regular Bayesianism?

bolution

solution

https://arxiv.org/abs/2211.06738 is related

Sure, I mean that it is an implementation of what you mentioned in the third-to-last paragraph.

Congratulations, you discovered [Active Inference]!

Solomonoff Induction and Machine Learning. How would you formulate this in terms of a machine that can only predict future observations?

I want to provide feedback, but can't see the actual definition of the objective function in either of the cases. Can you write down a sketch of how this would be implemented using existing primitives (SI, ML) so I can argue against what you're really intending?

Some preliminary thoughts:

1. Curiosity (obtaining information) is an instrumental goal, so I'm not sure if making it more important will produce more or less aligned systems. How will you trade off curiosity and satisfaction of human values?
2. It's difficult to specify correctly - depending on what you mean by curiosity, the AI can start showing itself random unpredictable data (the Noisy TV problem for RL curiosity) or manipulating humans to give it self-affirming instructions (if the goal is to minimize prediction error). So far I haven't seen a solution that isn't a hack that won't generalize to superintelligent agents.