Sean Hacking

I fine-tuned a toy model to vomit back arbitrary tokens. I used them as Markers tied to specific subsets of its training data to connect training to generation. Training + testing cost $0.49 on an RTX 4090. Could help with interp and safety monitoring. Chain of Thought alignment monitoring is...

Main point: seeking information on this potential strategy for improving neural network interpretability, alignment, and reliability. Segmenting training data and injecting unique "barcoding" tokens into each data subset. These unique tokens should occasionally show up in generated outputs at a frequency correlated with the importance of each training subset to...