LESSWRONG
LW

DawnLu

Message

Investigating Bias Representations in LLMs via Activation Steering

Produced as part of the SPAR program (fall 2023) under the mentorship of Nina Rimsky. Introduction Given recent advances in the AI field, it’s highly likely LLMs will be increasingly used to make decisions that have broad societal impact — such as resume screening, college admissions, criminal justice, etc. Therefore...

Jan 15, 2024•29

DawnLu

DawnLu — LessWrong

DawnLu

Message

Investigating Bias Representations in LLMs via Activation Steering

Jan 15, 2024•29

DawnLu

Investigating Bias Representations in LLMs via Activation Steering

DawnLu

Produced as part of the SPAR program (fall 2023) under the mentorship of Nina Rimsky.

Introduction

Given recent advances in the AI field, it’s highly likely LLMs will be increasingly used to make decisions that have broad societal impact — such as resume screening, college admissions, criminal justice, etc. Therefore it will become imperative to ensure these models don’t perpetuate harmful societal biases.

One way we can evaluate whether a model is likely to exhibit biased behavior is via red-teaming. Red-teaming is the process of “attacking” or challenging a system from an adversarial lens with the ultimate goal of identifying vulnerabilities. The underlying premise is that if small perturbation in the model can result in undesired... (read 1209 more words →)