Top postsTop post
TL;DR * Josh and Neel show that distillation from a teacher model to a base pretrained student model transfers some of the teacher model’s traits (such as displaying negative emotion in the Gemma Needs Help evals) * On its own this is pretty unsurprising, but Josh and Neel additionally show...
This work was done as part of the MATS 8.1 Program. 0: TL;DR * LLMs learn "values": general considerations (e.g. "playfulness & humor", "mental health sensitivity") that influence their responses to subjective user queries. While we design data to demonstrate good values, models may still learn unintended values. * We...
Authors: Joshua Engels*, Callum McDougall*, Bilal Chughtai*, Janos Kramar, Senthoran Rajamanoharan, Cindy Wu, Arthur Conmy, Asic Q Chen, Jean Tarbouriech, Min Ma, Brendan O'Donoghue+, João Gabriel Lopes de Oliveira+, Rohin Shah+, Neel Nanda+ *Primary Contributor +Advising Paper here: https://arxiv.org/abs/2606.20560 Overview In a recent collaboration between the GDM interpretability team and...
This is the fifth in a series of informal research updates from the Google DeepMind Language Model Interpretability team, in interpretability and adjacent areas. The fourth post can be found here. Thanks to Chloe Li for feedback on this post! TLDR: Via adapting the methods of Marks et al and...
This is the third in a series of informal research updates from the Google DeepMind Language Model Interpretability team, in interpretability and adjacent areas. The second post can be found here. In this short post, we describe a surprising finding: most safety relevant properties in Gemini seem to be caused...
Writing up a probably-obvious point that I want to refer to later, with significant writing LLM writing help. TL;DR: 1) A common critique of AI safety evaluations is that they occur in unrealistic settings, such as excessive goal conflict, or are obviously an evaluation rather than “real deployment”.[1] I argue...
Authors: Riya Tyagi, Daria Ivanova, Arthur Conmy, Neel Nanda Riya and Daria are co-first authors. This work was largely done during a research sprint for Neel Nanda’s MATS 9.0 training phase. 🖥️ Deployment code ⚙️ Interactive demo 🐦 Tweet thread TL;DR * We believe that more research effort should go...