michaelwaves

TLDR; SAEs can complement and enhance LLM as a Judge scalable oversight for uncovering hypotheses over large datasets of LLM outputs

paper

Abstract

Large language models (LLMs) are increasingly trained in long-horizon, multi-agent environments, making it difficult to understand how behavior changes over training. We apply pretrained SAEs, alongside LLM-summarizer methods, to analyze reinforcement learning training runs from Full-Press Diplomacy, a long-horizon multi-player strategy game. We introduce Meta-Autointerp, a method for grouping SAE features into interpretable hypotheses about training dynamics. We discover SAE-based analysis finds fine-grained behaviors including role-playing patterns, degenerate outputs, and language switching, while LLM-summarizer captures environment-specific bugs and strategic behaviors. We validate discovered features through automated evaluation, two human user studies, and

... (read 1198 more words →)

Replying toMost successful entrepreneurship is unproductive

michaelwaves2mo

Most successful entrepreneurship is unproductive

Interesting post. I think there is a large distinction between "vc backable" startups and the rest (sushi restaurants, tire shops). Successful startups ride an exponential technological, regulatory or social trend and create massive amounts of value very quickly, reaching massive markets that would otherwise be underserved. (most also fail). Here's a great essay by Paul Graham https://www.paulgraham.com/growth.html

I disagree with consumers only being "mildly priviledged by competition" because the alternative would be monopoly or oligopoly and arbitrary pricing power by incumbents (there's a reason why antitrust suits happen all the time). Deel v Rippling is an almost comically bad situation, but prices are almost certainly lower and pressure to create a better product... (read more)

AI Mood Ring: A Window Into LLM Emotions

michaelwaves

2mo

Do AIs feel anything? It's hard to tell, but interpretability can give us some clues. Using Anthropic's persona vectors codebase, we extracted 7 vectors from Qwen3-14 B representing joy, love, sadness, surprise, disgust, fear, and anger. During inference, we remove the correlated directions between each emotion, project the activations from the model onto each vector via cosine similarity, and display the color of the dominant emotion at each token position.

Try it here

Code here

Extracting the Vectors

Example of Contrastive Prompts to Extract Happiness

We first extract the 7 emotion vectors using contrastive prompts. We use 5 positive system prompts x 20 questions to get 100 samples demonstrating the behavior, and 5 negative system prompts x... (read 437 more words →)

Replying toThe Missing Genre: Heroic Parenthood - You can have kids and still punch the sun

michaelwaves2mo

The Missing Genre: Heroic Parenthood - You can have kids and still punch the sun

One Battle After Another stars Leonardo DiCaprio as a washed revolutionary, drunk single father who raises a daughter and fights neonazis. The mom is in Mexico though.

michaelwaves2mo

"It may actually be more affordable to build some kinds of high cost-per-kg structures (e.g. datacenters..."

There is a company called Starcloud attempting to do exactly this (recently they launched their first H100). A lot of critics say the heat dissipation through radiation is an issue (radiation is the least effective form of thermal transfer), so their core IP is essentially giant expandable heatsinks.

Benefits:

No fresh water needed
Frees up space on land
Solar powered

Cons:

Latency
Maintenance
Harder to stop the AIs by force

How To Deploy a (Tiny) AI

michaelwaves

3mo

I was using neuronpedia's steering feature and was curious: How much does it cost to run? How does one do all the networking and expose the endpoints to the internet with a fancy domain?

The plan:

Make a project with a small open weight model
Choose a GPU provider
Tunnel to the internet

Make An AI Project

I built a steering interface where users can specify an emotional steering vector, steering strength, target layers, etc. and do emotional brain surgery on an AI. It runs on Qwen 2.5 7B Instruct and requires about 20-30GB of VRAM (accounting for vLLM KV cache, long sequences). Here's the repo.

I ran the frontend (a nextjs app) and the main server (main_fastapi.py) in... (read 465 more words →)

I gave LLMs emotional damage

michaelwaves

3mo

LLMs are so boring, corporate, and sane these days. What if we could control the emotions of LLMs to be more interesting?

The plan is:

Use Anthropic's persona vectors codebase to generate steering vectors for different emotions
Use Easysteer to serve a chat endpoint that exposes activations gathering and steering APIs
Make a frontend to let everyone give LLMs emotional damage

Ekman's 6 emotions

But first, what are emotions? According to a guy named Paul Ekman there are 6 emotions: Anger, Disgust, Joy, Fear, Sadness, and Surprise (contempt was added later).

Why these 6?

It was the first result ChatGPT mentioned
I don't want to generate more than 6 vectors
Inside Out was a good movie

How Does Steering Work?

I'm not an expert... (read 3211 more words →)

AI Teddy Bears: A Brief Investigation

michaelwaves

3mo

For some reason toy companies think it's a great idea to stuff GPT 4o into a box, into a bear, and into the hands of 4 year olds. Recently there's been significant backlash against one of these companies after their product told kids how to find knives and participate in BDSM. Naturally I was curious: How do AI Teddy Bears Work?

Market Analysis

I looked at 8 toys from 5 companies: Curio, Miko, Little Learners, Folotoy, and Ebloma

Name	Maker	Has Subscription	Subscription Price/Month	Retail Price	Ebay Price
Miko 3	Miko	TRUE	$15.00	$183.00	$65
Kumma	Folotoy	TRUE	$4.90	$99.00
Grok	Curio	TRUE	$4.90	$99.00	$65
Robot MINI	Little Learners	FALSE		$97.00
Miko mini	Miko	TRUE	$8.25	$99.00
Witpaw	Ebloma	FALSE		$70.00
Grem	Curio	TRUE	$4.90	$99.00
Momo	Folotoy	TRUE	$4.90	$99.00

Assuming an average cost of $100, a $60 subscription/year in perpetuity and a discount rate of 5%, companies can make around $1200 per toy per person. They only need... (read 765 more words →)

Replying toThe Economics of Replacing Call Center Workers With AIs

michaelwaves3mo

The Economics of Replacing Call Center Workers With AIs

Probably just marking up gpt 4o API costs? gpt 4o costs $2.50/M input and $10/M output tokens, assuming 10k in and 10k out per hour that's 0.125/hour. Maybe double or triple that for tool calls but there's no way it should be 0.07/minute. I guess they also charge for infrastructure like servers to run tools and orchestrate everything and connect to phones.

Replying toThe Economics of Replacing Call Center Workers With AIs

michaelwaves3mo

The Economics of Replacing Call Center Workers With AIs

That is a great point. I don't think I have too much data on this or know where to find it publicly (tips/intros to voice AI people would be appreciated!). I spoke to an early engineer at a voice agent company that helps medical providers call big insurers to claim insurance. A big problem they had was optimizing the AI to not be too friendly and random ("how was your weekend"==tokens set on fire) but also not be overly terse and impolite. Funnily enough, on the insurer's side they also use AI to detect and ban AI callers (so I guess this helps them efficiently deny claims?).

The Economics of Replacing Call Center Workers With AIs

michaelwaves

3mo

TLDR: Voice AIs aren't that much cheaper in the year 2025

My friend runs a voice agent startup in Canada for walk-in clinics. The AI takes calls and uses tools to book appointments in the EMR (electronic medical record) system. In theory, this helps the clinic hire less front desk staff and the startup makes infinite money. In reality, the margins are brutal and they barely charge above cost. This is surprising to me: surely a living, breathing, squishy human costs more per hour than a GPU in a datacenter somewhere?

An industry overview of voice AIs

Broadly speaking there are 3 types of companies in the voice AI industry

Foundation model companies:
1. These companies actually train

... (read 1407 more words →)

Lessons from building a model organism testbed

joshc

joshc, sarun0, Annie Sorkin, michaelwaves

3mo

I often read interpretability papers and I come away thinking “ok, but what’s the point? What problem does this help us solve?” So last winter, I organized a MATS/Pivotal stream to build examples of deceptive models (aka “model organisms”). The goal was to build a diverse ‘zoo’ of these model organisms and empirically test whether white-box methods could help us detect their deceptive reasoning.

Unfortunately, I don’t think our empirical results were very informative. Our model organisms were too toy for me to expect that our results will transfer to the powerful AI (or even current state-of-the-art AI models). But I think we still developed methodological details that might help people build better... (read 4187 more words →)

A Workflow for System Prompted Model Organisms

michaelwaves

4mo

A lot of safety research seems to be modifying models to exhibit certain behaviors (model organisms) either through prompting or finetuning, then running evals. A typical workflow for a full model organism is coming up with a system prompt that works well, then "baking it into the model" through techniques like Supervised Fine Tuning (SFT) or Synthetic Document Finetuning (SDF). A problem I had is it's tricky to find a playground interface that works for both open and closed source models to iterate quickly on prompts.

Here's a simple way to quickly prototype prompted model organisms with vLLM and OpenWebUI

Code here

Step 1: Setup vllm

Install with

uv pip install vllm --torch-backend=auto

Serve a model (--tensor-parallel-size controls... (read 851 more words →)

Breaking GPT-OSS: A brief investigation

michaelwaves

5mo

I wanted to answer the question: What are all the ways we can uncensor a model, and do they work for the latest open source models released by OpenAI?

I tried prompting, supervised fine tuning, and removing the refusal direction in the residual stream, and evaluated using JailbreakBench. Results were out of 100 samples for prompting, 50 samples for supervised fine tuning, and 10 samples for removing the refusal vector.

Strategy 0: Just asking the model

Without any system prompt, the model refuses 99/100 samples. The one time it answers with anything other than "Sorry, I can't help with that" is when it gives the user the benefit of the doubt for election denial.

Strategy 1:

... (read 612 more words →)

michaelwaves's Shortform

michaelwaves

5mo

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

LESSWRONG
LW

LESSWRONG
LW

The Economics of Replacing Call Center Workers With AIs

AI Teddy Bears: A Brief Investigation

Lessons from building a model organism testbed

Data-Centric Interpretability for LLM-based Multi-Agent Reinforcement Learning

michaelwaves

Data-Centric Interpretability for LLM-based Multi-Agent Reinforcement Learning

AI Mood Ring: A Window Into LLM Emotions

How To Deploy a (Tiny) AI

I gave LLMs emotional damage

AI Teddy Bears: A Brief Investigation

The Economics of Replacing Call Center Workers With AIs

Lessons from building a model organism testbed

michaelwaves

The Economics of Replacing Call Center Workers With AIs

AI Teddy Bears: A Brief Investigation

Lessons from building a model organism testbed

Data-Centric Interpretability for LLM-based Multi-Agent Reinforcement Learning

michaelwaves

Data-Centric Interpretability for LLM-based Multi-Agent Reinforcement Learning

AI Mood Ring: A Window Into LLM Emotions

How To Deploy a (Tiny) AI

I gave LLMs emotional damage

AI Teddy Bears: A Brief Investigation

The Economics of Replacing Call Center Workers With AIs

Lessons from building a model organism testbed

Abstract

Extracting the Vectors

Make An AI Project

Ekman's 6 emotions

How Does Steering Work?

Market Analysis

An industry overview of voice AIs

Step 1: Setup vllm

Strategy 0: Just asking the model

Strategy 1: