Jeremias Ferrao

Message

10mo

Alignment Does Not Need to Be Opaque! An Introduction to Feature Steering with Reinforcement Learning

Apr 18, 2025•10

Jeremias Ferrao

Message

10mo

Alignment Does Not Need to Be Opaque! An Introduction to Feature Steering with Reinforcement Learning

Apr 18, 2025•10

Jeremias Ferrao

10mo

Alignment Does Not Need to Be Opaque! An Introduction to Feature Steering with Reinforcement Learning

Jeremias Ferrao

10mo

Reinforcement Learning (RL) has become the driving force behind modern artificial intelligence, as evidenced by rewarding models from human feedback or powerful new systems like OpenAI's o3 and DeepSeek R1, which are at the top of several capabilities benchmarks^[1]. While these advances are exciting, they've exposed critical vulnerabilities in our safety measures. Anthropic recently discovered language models that selectively comply with safety constraints only during evaluation, while Palisade found that o1 would rather hack its testing environment than play chess legitimately against a competitor.

At first glance, these developments might cast RL as the antagonist in AI safety research – a force that undermines our attempts to instil beneficial behaviours. However, I propose a different perspective: RL can serve... (read 2869 more words →)

LESSWRONG
LW

LESSWRONG
LW

Jeremias Ferrao

Jeremias Ferrao

Alignment Does Not Need to Be Opaque! An Introduction to Feature Steering with Reinforcement Learning

Jeremias Ferrao

Jeremias Ferrao

Alignment Does Not Need to Be Opaque! An Introduction to Feature Steering with Reinforcement Learning