simonsdsuo hasn't written anything yet.
simonsdsuo has not written any posts yet.
Summary of this post This is the third post in a three-part sequence on instrumental convergence in multi-agent RL. Read Part 1 and Part 2. In this post, we’ll: 1. Investigate instrumental convergence on a multi-agent gridworld with a complicated topology. 2. Show that when we add a simple physical...
Summary of this post This is the second post in a three-part sequence on instrumental convergence in multi-agent RL. Read Part 1 here. In this post, we’ll: 1. Define formal multi-agent POWER (i.e., instrumental value) in a setting that contains a "human" agent and an "AI" agent. 2. Introduce the...
Summary of the sequence Over the past few months, we’ve been investigating instrumental convergence in reinforcement learning agents. We started from the definition of single-agent POWER proposed by Alex Turner et al., extended it to a family of multi-agent scenarios that seemed relevant to AI alignment, and explored its implications...