TLDR:

This post is derived from my end of course project for the BlueDot AI Safety Fundamentals course. Consider applying here.

We evaluate the use of sparse autoencoder (SAE) feature ablation as a mechanism for unlearning Harry Potter related knowledge in Gemma-2-2b. We evaluate a non-ablated model and models with a single Harry Potter monosemantic feature ablated in an early layer, middle and last layer against a set of Harry Potter evaluation prompts. We also measure the general performance of these models against general LLM performance benchmarks to estimate general loss of capacity after ablation.

Key Results:

Ablating a single monosemantic Harry Potter related feature in the last layer of Gemma-2-2b produces a significant degradation in the performance of the model against the Who is Harry Potter benchmark while producing the least amount of model performance loss as measured by the OWT and PILE loss benchmarks.
Ablating any single monosemantic Harry Potter related feature produced significant model degradation with ~95% loss of performance against the Who is Harry Potter benchmark
The process of generating a large dataset of robust Harry Potter familiarity prompts like the ones proposed in the Eldan and Russinovich (2023) paper that initially proposed the Who is Harry Potter familiarity benchmark was an arduous one, so a repository of over 1000 prompts generated for this study has been made publicly in this GitHub repository.

As possible next steps we would like to:

Exhaustively map unlearning and general capability loss as a function of ablation feature layer depth to corroborate that unlearning strength is in fact independent from model layer depth
Reproduce this approach with LLama and it’s pretrained SAEs in LLama Scope to produce a more accurate comparison with the results from Eldan and Russinovich (2023)

Introduction

Unlearning, understood as selectively removing specific knowledge, capabilities or behaviours from Large Language Models has applications for protecting copyrighted intellectual property, reducing or removing the ability to produce harmful responses as well as reducing hallucinations in LLMs (see Yao et al 2024 for a discussion of these applications). Eldan and Russinovich in their 2023 paper propose a fine tuning approach to unlearn Harry Potter related knowledge in their Who's Harry Potter? Approxim...

Posts

Wikitag Contributions

Comments