I think we are having the wrong overall response to the OAI-HF incident: some vague intention of "pacing the frontier", but basically continuing the reckless race to higher capabilities. Somehow, this may be not so bad because it will cause even more spectacular warning shots, and a lot of political will. Because political will is so important in finally pausing for good, the gamble may be worth it: if we are far away from lethal swarm escapes, it's net positive to let out the (intended or unintended) early swarm escapes. Even though we are getting the curr... (read more)
The collaboration was a behavior that was trained for. See https://www.dwarkesh.com/p/noam-brown I'm actually curious if the OAI-HF incident would be possible with agents not trained in a multi-agent setting. My guess is that it wouldn't.
I think we are having the wrong overall response to the OAI-HF incident: some vague intention of "pacing the frontier", but basically continuing the reckless race to higher capabilities.
Somehow, this may be not so bad because it will cause even more spectacular warning shots, and a lot of political will. Because political will is so important in finally pausing for good, the gamble may be worth it: if we are far away from lethal swarm escapes, it's net positive to let out the (intended or unintended) early swarm escapes.
Even though we are getting the curr... (read more)