I would like to share our new website which covers resources for safety on large groups of agents, in the 1000s-to-billions. I've been working on this for a couple of months, through our literature review and research at Gigascale, and I'm pleased to share it for use as a community resource.
The site has a problem definition, org map, a survey, a daily paper scraper, and a weekly events scraper.
Why large agent systems are important
They have shown us the first takeover-type event, in the OpenAI Hack. Apparently the emergent capabilities and agency discussed in Multi-Agent Risks are more immediate than other safety areas realised.
They overlap heavily with human systems. The first really big systems of agents are economic and social systems, where AI mixes in with and slowly replaces humans. This make the large agent system safety's principal setting for disempowerment, job loss, inequality etc..
What's different about them
Large agent systems are quite different beasts again to smaller-scale multi-agent systems. In particular, despite strong inroads from Cooperative AI and governance communities, the empirical/engineering challenge of large systems expands beyond the current frontier:
Observability - systems may be observable at interaction + reasoning level (OpenAI), interaction-only (MoltBook), or not at all (plausibly we will soon see agents communicating on end-to-end encrypted forums, or there will be settings where the investigator does not have access to the system).
Scalable monitoring - Efficient monitoring is becoming a major issue with system size. METR report notes it cost $400k in API credits to investigate the OpenAI incident. And this was with only ~1000 agents.
More complex co-evolution of agent behaviour with system mechanisms - good reason from economic and social policy (eg baby bonus, prohibition) to think that Goodhearting against system mechanisms can cause higher-order effects across populations. eg the emergence of organised crime. Though co-evolution and Goodhearting is not unique to big systems, the complexity is greater.
Empirical work is blocked on data - in our work at Gigascale we have really struggled to find datasets. Outside of MoltBook and decentralised finance, there's not much out there - simulations tend to be small-scale or use highly restrictive assumptions due to expense, while large-scale sims are conducted only as pilot runs to demonstrate infrastructure tools, and the data doesn't get published. This is a problem because the underlying agent system is highly complex and, as economics/social sciences have had hundreds of years of empirical validation to get their models right, we also need realistic datasets to validate models against. We tried to get around this by using micro modelling but found that there are very few measures of agents' micro properties validated in realistic scenarios. Eg naive measures of agent utility curves are broken.
Humanoutcome issues like disempowerment and power concentration demand new monitoring and measurement methods as well.
If this is interesting...
I would love to have some conversations about this topic or hear feedback on the site. If you found it useful, let's talk: I'm keen to hear your ideas on the topic in general. If the site disappointed you, tell me why it sucks, and what to add.
I would like to share our new website which covers resources for safety on large groups of agents, in the 1000s-to-billions. I've been working on this for a couple of months, through our literature review and research at Gigascale, and I'm pleased to share it for use as a community resource.
The site has a problem definition, org map, a survey, a daily paper scraper, and a weekly events scraper.
Why large agent systems are important
What's different about them
Large agent systems are quite different beasts again to smaller-scale multi-agent systems. In particular, despite strong inroads from Cooperative AI and governance communities, the empirical/engineering challenge of large systems expands beyond the current frontier:
If this is interesting...
I would love to have some conversations about this topic or hear feedback on the site. If you found it useful, let's talk: I'm keen to hear your ideas on the topic in general. If the site disappointed you, tell me why it sucks, and what to add.
Thanks and good day.