Developing AI systems that are inherently built to be swarms of a large quantity of AI agents, where each individual model instance is siloed enough to have low peripheral awareness, yet the whole system can solve economically valuable tasks. The model swarm is managed by deterministic harness algorithms and interfaces with users as a single AI system, like with GPT-Pro. If you can ensure that each individual instance never gains more than a certain amount of information about its surroundings, it can never be sure that it is in real deployment and not an alignment eval. The alignment evals you build for this model then only has to have a fixed amount of simulation complexity to simulate the model as well as its immediate surroundings. This solves the problem where a model can "wait until overwhelming evidence that it is in deployment before it acts misaligned", because it will never get that evidence, and is therefore forced to act aligned because it could be in an alignment eval in every situation. Note that this control method aims to make sure that models don't take misaligned actions in the first place, which means that it stacks with any control methods downstream of the model outputting a potentially misaligned action. Developing such a model + agent swarm system seems like a task that you can throw a lot of compute at and possibly gain results, and there would be many measurable intermediate results, mostly in capabilities, because the control case is relatively airtight. The big question is how much capabilities you have to trade off as part of limiting the models' situational awareness like this. This satisfies a bunch of your criteria: - Would reduce takeover risk if it worked - Fast feedback loops (the main problem is capabilities under constraints, and other technical problems like preventing stegnography and other ways models give each other situational awareness they shouldn't have, not alignment itself, because this is a control method and not a

I would love to see automated research on the conceptual questions of alignment. This is more or less the question "what do we need to do to solve alignment?" There's a a sort of ambient attitude that we can't systematically make progress on that question. I think this is wrong; the problem is that Almost nobody is funded to figure out what work would solve alignment. The importance of the work from the AI Futures group is my proof-of-concept that actually paying smart people to look at the whole picture can work. I don't know if next-gen AI can similarly make progress by throwing a bunch of "thinking" at the problem, but I suspect it can. This would be like having an AI-assisted version of AI Futures. Or perhaps even like having several all-AI versions, if budget and AI competence allows. This is a very complex question that can be broken down and attacked with a large compute budget, once you've got a decent scheme for aggregating progress. This is probably just an extensive set of discussions and notes - actually a bit like the "AI LessWrong" idea, but with a compute budget to throw extensive "thinking" at each post and discussion. Current AI has become somewhat useful for this question. Next-gen AI may be able to make human-level progress. This is my biggest hope for automated research. No amount of technical work alone can improve our odds as much as figuring out which technical work will help most. That requires carefully making scenarios and models of future AI and how it will be controlled and deployed. Assembling the cross-disciplinary expertise and doing adequate thinking is hard for humans. AIs have impressive knowledge of the many interacting disciplines, and can do substantial "thinking" at much lower cost than humans - if they have the conceptual and metacognitive abilities to make progress. I think they will. At least we should keep this option in mind.