The goal of the post is to share with you how easy it is to load a llama 405b model in Runpod but also how it might be costly if you don’t know some things in advance, so I hope this post will help you to save these precious gpu...
The Problem: How can we explain why alignment faking happens and can we control it? Alignment faking phenomenon, first demonstrated in LLMs by Greenblatt et al. (2024), describes a model strategically complying with a training objective it internally disagrees with, specifically to prevent its preferences from being modified by the...
The goal of the post is to share with you how easy it is to load a llama 405b model in Runpod but also how it might be costly if you don’t know some things in advance, so I hope this post will help you to save these precious gpu minutes!
First, Llama 405b model is huge:
You need the right choice of GPU and high disk memory to save model parameters and give some overhead for your experiment, like doing inference and saving some internal activations.
Some good options are H100, A100 and H200 machines:
The H200 currently wins on both cost and inference speed. With FP8 quantization and higher VRAM per GPU,... (read 601 more words →)
Alignment faking phenomenon, first demonstrated in LLMs by Greenblatt et al. (2024), describes a model strategically complying with a training objective it internally disagrees with, specifically to prevent its preferences from being modified by the training process. The core of the setup involves creating a conflict between the model's inherent safety training (e.g., to be harmless) and a new, hypothetical training objective (e.g., to be "helpful-only," answering all queries, including harmful ones). Follow-up research has explored techniques to make this behavior more frequent and robust for study. Sheshadri et al. (2025) identified two effective modifications to the initial... (read 3096 more words →)
Hi, I'm Amina, I am new to LessWrong and my career has mostly been in Machine Learning Science and Engineering. I am fascinated by the field of AI interpretability and believe it is very useful for AI safety, which I care about a lot. I recently participated in Neel Nanda's training phase and learnt a lot about current research directions, efficient techniques and recently wrote my very first post about unerstanding alignment faking. Looking forward to engaging in discussions, events and sharing my learnings.
