Hey Jeremy! Our team's interest is mainly on multi-agent learning, and self-modeling and theory of mind. As properly formalizing a coherent theory for these topics turned out to be quite difficult, we dived deeper and deeper and ultimately arrived at the AIXI and reflective oracles frameworks, which provided a good set of tools as starting points for addressing these questions more formally. The resulting 'monster paper' is a write-up of the past year of work we did on these topics. Due to our interest in multi-agent learning, a good chunk of the paper is on the game-theoretic behavior of such 'embedded Bayesian agents' (Section 4). As Cole mentioned, we arrived independently to some similar results as Cole's (as we came from a bit outside of the less wrong community), and we are very excited to now start collaborating more closely with Cole on the next questions enabled by both of our theories!
Hey Jeremy! Our team's interest is mainly on multi-agent learning, and self-modeling and theory of mind. As properly formalizing a coherent theory for these topics turned out to be quite difficult, we dived deeper and deeper and ultimately arrived at the AIXI and reflective oracles frameworks, which provided a good set of tools as starting points for addressing these questions more formally. The resulting 'monster paper' is a write-up of the past year of work we did on these topics. Due to our interest in multi-agent learning, a good chunk of the paper is on the game-theoretic behavior of such 'embedded Bayesian agents' (Section 4). As Cole mentioned, we arrived independently to some similar results as Cole's (as we came from a bit outside of the less wrong community), and we are very excited to now start collaborating more closely with Cole on the next questions enabled by both of our theories!