by Narmeen Oozeer, Dhruv Nathawani, Nirmalendu Prakash, Amirali Abdullah
This work was done by Narmeen Oozeer as a research fellow at Martian, under an AI safety grant supervised by PIs Amirali Abdullah and Dhruv Nathawani. Special thanks to Sasha Hydrie, Chaithanya Bandi and Shriyash Upadhyay at Martian for suggesting researching generalized backdoor mitigations as well as extensive logistical support and helpful discussions.
TLDR:
- We show that representations across models of different sizes are weakly isomorphic when trained on similar data, and that we can "transfer" activations between them using autoencoders.
- We propose a technique to transfer safe behavior from one model to another through the use of steering vectors.
- Our representation transfer technique paves the way for transferring insights across
... (read 4785 more words →)