Eclipse photography is probably the most notorious example. Someone will this summer get the shot of the millennium, by being at the right place in Egypt with the scene already planned, and a timing that comes down to a few minutes.
Lean soundness bugs have been exploited by AIs. There are also lots of subtleties around what a Lean proof means, and some still pending theoretical work to show Lean isn't stronger than ZFC, or ZFC plus some mild things.
In a democratic society, we all control the decisions made that will determine if we are OK. This call to quietism is completely at odds with civic duty and self government.
I think this modeling assumes Russia can escalate conventionally and that the conventional NATO response would be perceived as escalatory by Russia even if it destroyed their army. Russia can't escalate conventionally: they have run out of tanks and men.
Ukraine is already doing a great job of destroying Russia's army with NATO weapons, and Russia hasn't used nukes to stop it. In the aftermath of usage, increasing the rate of that destruction is just more of the same. Even if Russia would like to escalate more, they need to actually stop the army from liber... (read more)
I think the issue presented in the post is that the Solomonoff hypothesis cannot be sampled from, even though we can determine the probability density function computationally. If we were to compute the expected value of the reward based on our action, we run into the curse of dimensionality: there is a single point contributing most of the reward. A Solomonoff inductor would correctly find the probability density function that h(s_2)=s_1 with high probability.
However, I think that if we ask the Solomonoff predictor to predict the reward directly, then it will correctly arrive at a model that predicts the rewards. So we can fix the presented agent.
Eclipse photography is probably the most notorious example. Someone will this summer get the shot of the millennium, by being at the right place in Egypt with the scene already planned, and a timing that comes down to a few minutes.