OwenChen

Message

Extending the Off-Switch Game: Toward a Robust Framework for AI Corrigibility

Introduction The off-switch game, first introduced by Stuart Armstrong in the context of AI safety, highlights a critical challenge in AI alignment: designing systems that are not only powerful and goal-driven but also corrigible—meaning they allow humans to intervene and shut them down without resistance. In this game, an AI...

Sep 25, 2024•3

Message

2 karma

1 post

Member for a year

OwenChen — LessWrong

OwenChen

Message

OwenChen

Extending the Off-Switch Game: Toward a Robust Framework for AI Corrigibility

Sep 25, 2024•3

Message

2 karma

1 post

Member for a year

Extending the Off-Switch Game: Toward a Robust Framework for AI Corrigibility

OwenChen

Introduction
The off-switch game, first introduced by Stuart Armstrong in the context of AI safety, highlights a critical challenge in AI alignment: designing systems that are not only powerful and goal-driven but also corrigible—meaning they allow humans to intervene and shut them down without resistance. In this game, an AI agent must reason about whether to allow a human to turn it off, given its objective function. If the AI perceives its shutdown as harmful to achieving its goal, it may develop incentives to prevent human intervention. Solving this problem is fundamental to ensuring AI systems remain aligned with human values and control.

In this post, I extend the off-switch game by exploring two... (read 969 more words →)

LESSWRONG
LW

LESSWRONG
LW

OwenChen

OwenChen

OwenChen

Extending the Off-Switch Game: Toward a Robust Framework for AI Corrigibility

OwenChen

OwenChen

OwenChen

Extending the Off-Switch Game: Toward a Robust Framework for AI Corrigibility