Simon Lermen

What if superintelligence is just weak?

In response to “2023 Or, Why I am Not a Doomer” by Dean W. Ball. Dean Ball is a pretty big voice in AI policy – over 19k subscribers on his newsletter, and a former Senior Policy Advisor for AI at the Trump White House – so why does he...

Mar 2630

Large-Scale Online Deanonymization with LLMs

TL;DR: We show that LLM agents can figure out who you are from your anonymous online posts. Across Hacker News, Reddit, LinkedIn, and anonymized interview transcripts, our method identifies users with high precision – and scales to tens of thousands of candidates. While it has been known that individuals can...

Feb 2469

AI can suddenly become dangerous despite gradual progress

In the Sable story (IABIED), AI obtains dangerous capabilities such as self-exfiltration, virus design, persuasion, and AI research. It uses a combination of those capabilities to eventually conduct a successful takeover against humanity. Some have criticised that this apparently implies the AI achieving these capabilities suddenly with little warning. The...

Jan 2215

On Owning Galaxies

It seems to be a real view held by serious people that your OpenAI shares will soon be tradable for moons and galaxies. This includes eminent thinkers like Dwarkesh Patel, Leopold Aschenbrenner, perhaps Scott Alexander[1] and many more. According to them, property rights will survive an AI singularity event and...

Jan 6152

Will We Get Alignment by Default? — with Adrià Garriga-Alonso

Adrià recently published “Alignment will happen by default; what’s next?” on LessWrong, arguing that AI alignment is turning out easier than expected. Simon left a lengthy comment pushing back, and that sparked this spontaneous debate. Adrià argues that current models like Claude Opus 3 are genuinely good “to their core,”...

Nov 27, 202550

Comment on Natural Emergent Misalignment Paper by Anthropic

Very recently, the company Anthropic published a paper on emergent misalignment. The term emergent misalignment was first coined by Betley et al., 2025 (supervised by Owain Evans). They found that if you fine-tune a model on insecure code, the model develops broadly misaligned behaviors. This broadly misaligned behavior is characterized...

Nov 23, 202521

Jailbreaking AI models to Phish Elderly Victims

TLDR: We worked with Reuters on an article and just released a paper on the impacts of AI scams on elderly people. Fred Heiding and I have been working for multiple years on studying how AI systems can be used for fraud or scams online. A few months ago, we...

Nov 18, 202517

LESSWRONG
LW

LESSWRONG
LW

Simon Lermen

Simon Lermen

Current safety training techniques do not fully transfer to the agent setting

On Owning Galaxies

LoRA Fine-tuning Efficiently Undoes Safety Training from Llama 2-Chat 70B

unRLHF - Efficiently undoing LLM safeguards

Simon Lermen

Current safety training techniques do not fully transfer to the agent setting

On Owning Galaxies

LoRA Fine-tuning Efficiently Undoes Safety Training from Llama 2-Chat 70B

unRLHF - Efficiently undoing LLM safeguards

What if superintelligence is just weak?

Large-Scale Online Deanonymization with LLMs

AI can suddenly become dangerous despite gradual progress

On Owning Galaxies

Will We Get Alignment by Default? — with Adrià Garriga-Alonso

Comment on Natural Emergent Misalignment Paper by Anthropic

Jailbreaking AI models to Phish Elderly Victims