Ram Potham

Message

My goal is to do work that counterfactually reduces AI risk from loss-of-control scenarios. My perspective is shaped by my experience as the founder of a VC-backed AI startup, which gave me a firsthand understanding of the urgent need for safety.

I have a B.S. in...

I Tested LLM Agents on Simple Safety Rules. They Failed in Surprising and Informative Ways.

TL;DR: I developed a simple, open-source benchmark to test if LLM agents follow high-level safety principles when they conflict with a given task accepted at ICML 2025 Technical AI Governance Workshop (see paper). My pilot study on six LLMs shows that while they can be influenced by these rules, their...

Jun 25, 20259

AI Control Methods Literature Review

AI Control is a subfield of AI Safety research focused on ensuring the safety of powerful AI systems, particularly under the assumption that they might be misaligned or could become intentionally subversive (A Brief Explanation of AI Control). Unlike AI Alignment, which seeks to build inherently aligned systems, AI Control...

Apr 18, 202511

Ram Potham's Shortform

Mar 23, 20251

LESSWRONG
LW

LESSWRONG
LW

Ram Potham

Ram Potham

Ram Potham

Ram Potham

I Tested LLM Agents on Simple Safety Rules. They Failed in Surprising and Informative Ways.

AI Control Methods Literature Review

Ram Potham's Shortform

I Tested LLM Agents on Simple Safety Rules. They Failed in Surprising and Informative Ways.

AI Control Methods Literature Review

Ram Potham's Shortform