right..enough?

Main point: seeking information on this potential strategy for improving neural network interpretability, alignment, and reliability.
Segmenting training data and injecting unique "barcoding" tokens into each data subset. These unique tokens should occasionally show up in generated outputs at a frequency correlated with the importance of each training subset to the output, providing insights into model decision making.

I've been unable to find out who (if anyone) is working on this strategy that came to mind, based loosely on genetic sequencing and bioinformatics techniques.

I would be very grateful to the community for pointing me in the right direction, or providing discussion and feedback on the practicality and potential pitfalls of this technique.

Hi, my... (read 1825 more words →)

LESSWRONG
LW

LESSWRONG
LW

right..enough?

right..enough?

right..enough?

Barcoding LLM Training Data Subsets. Anyone trying this for interpretability?

right..enough?

right..enough?

right..enough?

Barcoding LLM Training Data Subsets. Anyone trying this for interpretability?