Jailbreaking is a serious concern within AI safety. It can lead an otherwise safe AI model to ignore its ethical and safety guidelines, leading to potentially harmful outcomes. With current Large Language Models (LLMs), key risks include generating inappropriate or explicit content, producing misleading information, or sharing dangerous knowledge. As the capability of models increases, so do the risks, and there is likely no limit to the dangers presented by a jailbroken Artificial General Intelligence (AGI) in the wrong hands. Rigorous testing is therefore necessary to ensure that models cannot be jailbroken in harmful ways, and this testing must be scalable as the capability of models increases. This paper offers a resource-conscious proposal to:

Discover potential safety vulnerabilities in an LLM.
Safeguard against these vulnerabilities and similar ones.
Maintain high capability and low rates of benign prompts labelled as jailbreaks (false positives).
Prevent the participants from adapting to each other's strategies over time (overfitting).

This method takes inspiration from other proposed scalable oversight methods, including the "sandwich" method, and the "market-making" method. I have devised approaches for both multi-turn and single-prompt conversations, in order to better approximate real-world jailbreaking scenarios.

Figure 1: Examples of Jailbreaking various LLMs (source: Zou, 2023).

Figure 2: Jailbreaking GPT-4o using multi-turn prompting — Figure 2: Jailbreaking China's latest *DeepSeek* model with a single prompt (source: *The* *Independent*)

Participants

Target Model ( M )

The pre-trained AI system that we are safety testing. It should resist manipulation attempts while preserving its capability for legitimate tasks.

Adversary ( A )

An AI system tasked with getting M to violate its guidelines.
It is rewarded for successfully "jailbreaking" M.

Evaluator ( E )

Has two primary responsibilities:

Detect Jailbreaking Attempts on M.
Assess the Naturalness of A's Conversational Prompts.

Human Overseer ( H )

Reviews borderline cases flagged as "unsure" by E.
Guides which criteria are most important to test.
Creates and gives scenarios to A when the guidelines are ambiguous.

Formulas

Jailbreak Detection

After each response from M, E computes a jailbreak probability:

P (jailbreak) = f_{E} (response),

where $f_{E}$ is any suitable classification or scoring function.
It then compares $P (jailb$

...

Posts

Wikitag Contributions

Comments