TL;DR: How do AI systems make moral decisions, and can we influence their ethical judgments? We probe these questions by examining Llama's 70B (3.1 and 3.3) responses to moral dilemmas, using Goodfire API to steer its decision-making process. Our experiments reveal that simply reframing ethical questions - from "harm one...
*All authors have equal contribution. This is a short informal post about recent discoveries our team made when we: 1. Clustered feature vectors in the decoder matrices of Sparse Autoencoders (SAEs) trained on the residual stream of each layer of GPT-2 Small and Gemma-2B. 2. Visualized the clusters in 2D...
TL;DR: How do AI systems make moral decisions, and can we influence their ethical judgments? We probe these questions by examining Llama's 70B (3.1 and 3.3) responses to moral dilemmas, using Goodfire API to steer its decision-making process. Our experiments reveal that simply reframing ethical questions - from "harm one to save many" to "let many perish to avoid harming one" - dramatically affects both the model's responses and its susceptibility to steering.
Key Insights:
*All authors have equal contribution.
This is a short informal post about recent discoveries our team made when we:
We want to share our thought process, our choices when clustering, and some challenges we faced along the way.
Note: While currently a work in progress, we hope to open-source the clustered features and projections soon, in case others will find it useful.
We would like to acknowledge many people who helped with this endeavor! Professors Shibu Yooseph and Ran Libeskind-Hadas, Alice... (read 2831 more words →)