TLDR:
Hypothesis: Safety techniques show promise but provide no evidence that we can achieve AI agents that are fundamentally different than humans in respect to safety and alignment. In my opinion, safe and aligned AI can be as safe and aligned as humans, who can:
Conclusion: Emphasize the limits of safe and aligned AI while making policies for regulation. Keep stressing the question if we are prepared to face the repercussions of the likelihood of safe and aligned AI causing harm at that scale that fast
Why this post?
This is an opinion piece. I have tried not to look into the theoretical works on AI safety yet so as to develop my own conclusions and ideas on AI safety.
Even if this post is written with the risk on not adding anything new, I believe it serves as a documentation of independent opinions on AI risks and AI safety approaches developed purely based on readings about several AI safety techniques and avoiding influence from different schools of thought in this area.
I have worked as a deep learning researcher at a University for two years after a PhD in Mathematics and recently attended AI safety fundamentals by BlueDot.
I will avoid the discussion on ASI. I believe that some capability researchers and policy professionals undermine and downplay the discussions on safety on the mention of ASI. For the sake of discussion, I will agree with them on ASI and focus on much simpler projection of AI:
So, here I focus on the increasingly capable AI in the immediate future in its safest and most aligned form and refer it as the best-case-scenario AI.
In my humble opinion, in this projection scenario, absolutely safe and absolutely aligned AI agents do not exist.
In terms of policy, we would be better off if we assume the following for the best-case-scenario AI:
We can only have control over them as much we have on humans. For example, we will be able to put them under
- lie detectors (Representation Engineering),
- scrutinize them (Mechanistic Interpretability),
- question them (Chain-of-thought),
and more but without having absolute control over their actions. And thus have to account for the risk of failure.
These are safer assumptions when we don't see a really strong evidence suggesting otherwise. Ongoing work on safety and alignment, although very important and promising, gets more attention in mainstream narrative. This leads to an overestimation of its potential and a discussion on its limits, which is very important for policy making, generally tends to get ignored.
Claim: Safe AI can always be persuaded to do harm, same as humans.
What is true nevertheless?
Advanced AI will enable us to make safer AI. But we need to emphasize that safer is relative to what we have now but at the same time better AI will also be more dangerous than we have now.
At every stage of its capabilities it will come with its good, its bad, and its ugly.
Safe AI systems fine-tuned with safety techniques such as RLHF, Constitutional AI, debate, etc., will always be prone to Jailbreaking.
If we evaluate the AI capabilities based on what can be elicited from it based on the carefully crafted prompt, we should do the same for safety certifications.
Jailbreak is a carefully crafted prompt. And once found can be used untraced for long to derive dangerous capabilities.
AI companies should accept that jailbreaks will always exist. And in the cases where they want to ensure that their services are not being misused, they should be partially monitoring the responses of AI and not just rely on AI score of harmlessness based on red teaming. We need to account for the scale of risks for misuse of AI at every stage of its capability.
Alignment is a problem independent of AI that exists because it is hard to specify things.
Claim: No evidence that aligned AI can be fundamentally better than an aligned intern.
Any issue in AI alignment is parallel to a smart newly hired intern in a company.
We are able to work with human interns by more or less giving specific task instructions by relying on a shared value system fine-tuned based on the company and learned socially, which we believe the intern will apply when facing a deliberation between different compromises.
We can use various safety techniques to add similar value systems in AI e.g. Constitutional AI. Still, there is no reason to believe that it will result in something fundamentally different than humans when it comes to understanding the intent of tasks.
AI safety policy and regulation conversation gets muddled by an overestimation of the outcomes that can be achieved by AI alignment.
In policy discussions, we need to emphasize the limits of alignment in Aligned (best-case-scenario) AI and talk about the risks involved with the likelihood of
at the scale powerful aligned agents can cause.
This post adopts a simplified projection of AI for the purpose of policy discussions. It is assumed as a better course of immediate action whenever it is harder to substantiate the impact of the jump in intelligence on the emergence of ASI. Please critique and suggest links to articles and organizations that are working or want to work with a similar approach.