x

LESSWRONG

LW

Aniket Ghosh — LessWrong

Aniket Ghosh

Aniket Ghosh

Message

MS AI at Northeastern; ML engineer at Varosync (drug-development knowledge graph + molecular models). Mainly interested in mechanistic interpretability and technical AI safety (BlueDot cohort).

portfolio : https://itsme-aniketghosh.github.io/

38

2

2mo

Aniket Ghosh

MS AI at Northeastern; ML engineer at Varosync (drug-development knowledge graph + molecular models). Mainly interested in mechanistic interpretability and technical AI safety (BlueDot cohort).

portfolio : https://itsme-aniketghosh.github.io/

Function vectors as a model diffing tool: 17 heads repair a bad fine-tune

I take two fine-tuned models trained to give bad medical advice, one on Qwen2.5-7B and one on Llama-3.1-8B, from the Model Organisms for Emergent Misalignment collection, and I found out that I could make one safe by simply copying 17 attention heads from the base model it was trained from...

Linear probes tell you where quantization will hurt

Epistemic status: I have only tested one encoder family (BERT-base and its relatives) and one decoder LLM (Qwen2.5-3B), one seed, token-level tasks, and post-training weight quantization. I trust the results because I am not inventing anything; I am just connecting quantization with a very general idea from mech interp: the...