Kajetan Dymkiewicz

You can now access the full article here! Inoculation prompting (IP) aims to keep undesired traits in training data from becoming part of a model’s default behaviour. During training, examples displaying those traits are paired with a prompt that explicitly requests them. In standard IP, however, applying the same prompt...

Summary We argue that shaping RL exploration, and especially the exploration of the motivation-space, is understudied in AI safety and could be influential in mitigating risks. Several recent discussions hint in this direction — the entangled generalization mechanism discussed in the context of Claude 3 Opus's self-narration, the success of...