Motivation

The Good

Humans are not born with knowledge and wisdom. But over the course of a lifetime they manage gradually experience and learn about their world, some managing to become the Newtons, Einsteins, Confuciuses, Gandhis, and Mandelas.

The capacity for such life-long, open ended learning, so naturally human to each of us, remains ever so difficult for AI, and may very well be among the final frontiers of unconquered capabilities, if it can be conquered at all.

Studying this question: what gives rise to the capacity to learn and gain wisdom over a lifetime? has been my personal north star. This was true when I was a mathematician, was still true when I was a neuroscientist, and is true in my present gig as a research scientist at Google DeepMind.

The bad

But putting aside lofty goals, a dark side emerges! A viral meme occasionally makes the rounds showing a user feeding a single, outlandish or fake fact to a large language model (LLM), only for the model to start sprinkling that same fact everywhere. The example below, for instance, has its origins from a Reddit comment posted 11 years ago.

But beneath these anecdotes of LLM eccentricities lies serious research questions about their knowledge. At heart is how LLMs are trained and continue to be trained. Every time an LLM is fine-tuned on new data—be it for an updated knowledge base, a personalized application, or an urgent domain like medicine—questions arise about how that injection of new information will influence its established capabilities and knowledge, potentially poisoning the well in uncontrollable ways, to all of our detriments.

The Ugly

Regardless whether you are more motivated by the lofty first goal, or more motivated to cure the second, dark poison, they are actually two sides of the same scientific coin: what learning means to an LLM: What's actually happening inside these models when they learn something new? And most importantly, can we control this process, so that we learn in a possibly desired way that we want & minimize negative side effects?

This fundamental issue is what drove our recent study at Google DeepMind, "How New Data Permeates LLM Knowledge and How to Dilute It." , to be a Spotlight at ICLR 2025 in Singapore.

Background

Large language models have become indispensable in many areas, from re...

root three

root three

root three

root three