This is the first research release from Wiser Human, an organisation we set up from the Catalyze Impact AI Safety Incubator. Our backgrounds are in risk management and software engineering, and we believe we are best placed to contribute by building practical tools for improving layered defenses in AI systems i.e. contributing to a “Swiss cheese model” for reducing harm from the actions of agentic AI systems, where multiple safeguards work together to reduce risk even when each layer is imperfect.
For this project, we focused on how models might be steered toward safer actions, such as escalating problems, rather than taking useful but harmful paths, such as blackmail.
This... (read 403 more words →)
I think you make some good points here, but there's an additional mechanism by which I believe internal-based techniques have the potential to make people intervene strongly on suspected misaligned behaviour.
This is whereby (in pre-superintelligent models) we are able to establish a strong correlation between misaligned AI behaviour examples understandable to humans and some model internals e.g. a deception probe. If there is strong empirical evidence establishing a link (I'm not strongly confident this will be the case, but mildly optimistic), then as we move to superintelligent models, I believe people will be more likely to take action on evidence from model internals alone, especially if above a certain threshold for likelihood.
My... (read more)