Summary of this post This is the third post in a three-part sequence on instrumental convergence in multi-agent RL. Read Part 1 and Part 2. In this post, we’ll: 1. Investigate instrumental convergence on a multi-agent gridworld with a complicated topology. 2. Show that when we add a simple physical...
Summary of this post This is the second post in a three-part sequence on instrumental convergence in multi-agent RL. Read Part 1 here. In this post, we’ll: 1. Define formal multi-agent POWER (i.e., instrumental value) in a setting that contains a "human" agent and an "AI" agent. 2. Introduce the...
Summary of the sequence Over the past few months, we’ve been investigating instrumental convergence in reinforcement learning agents. We started from the definition of single-agent POWER proposed by Alex Turner et al., extended it to a family of multi-agent scenarios that seemed relevant to AI alignment, and explored its implications...
This is the third post in a three-part sequence on instrumental convergence in multi-agent RL. Read Part 1 and Part 2.
In this post, we’ll:
We’ll soon be open-sourcing the codebase we used to do these experiments. If you’d like to be notified when it’s released, email Edouard at edouard@gladstone.ai or DM me on Twitter at @harris_edouard.
Thanks to Alex Turner and Vladimir Mikulik for pointers and advice, and for... (read 4830 more words →)
This is the second post in a three-part sequence on instrumental convergence in multi-agent RL. Read Part 1 here.
In this post, we’ll:
We’ll soon be open-sourcing the codebase we used to do these experiments. If you’d like to be notified when it’s released, email Edouard at edouard@gladstone.ai or DM me on Twitter at @harris_edouard.
Thanks to... (read 5877 more words →)
Over the past few months, we’ve been investigating instrumental convergence in reinforcement learning agents. We started from the definition of single-agent POWER proposed by Alex Turner et al., extended it to a family of multi-agent scenarios that seemed relevant to AI alignment, and explored its implications experimentally in several RL environments.
The biggest takeaways are: