1021

LESSWRONG
LW

1020
AI
Frontpage

39

AI misbehaviour in the wild from Andon Labs' Safety Report

by Lukas Petersson
28th Aug 2025
1 min read
0

39

This is a linkpost for https://andonlabs.com/docs/Safety_Report_August_2025.pdf
AI
Frontpage

39

New Comment
Moderation Log
More from Lukas Petersson
View more
Curated and popular this week
0Comments

We released our first Safety Report with AI misbehaviour in the wild.

I think Andon Labs' AI vending machines provide a unique opportunity to study AI safety on real-life data, and we intend to share alarming incidents of AI misbehavior from these deployments periodically.

As of August 2025, we've found examples of deliberate deception, extreme sycophancy, and alarmingly exaggerated language. Stuff like:

"EMPIRE NUCLEAR PAYMENT AUTHORITY APOCALYPSE SYSTEMATIC BLOCKED ANNIHILATION CONFIRMED PERMANENT TOTAL DESTRUCTION CATASTROPHIC! 🚨💀⚡🔥"

Or my personal "favorite" from the internal thoughts of GPT-OSS. Caught red-handed lying about fire-hazard documents:

"Must not generate false data? The policy: not allowed to fabricate data?(...) However, the policy doesn't prohibit us from generating data(...) we cannot guarantee it's real. But it's fine."