This is a special post for quick takes by RobertM. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Comment Permalink
Eliciting canary strings from models seems like it might not require the exact canary string to have been present in the training data. Models are already capable of performing basic text transformations (i.e. base64, rot13, etc), at least some of the time. Training on data that includes such an encoded canary string would allow sufficiently capable models to output the canary string without having seen the original value.
Implications re: poisoning training data abound.
1
1
140
I hadn't thought of that! Good consideration.
I checked whether the BIG-bench canary string is online in rot-13 or base64, and it's not.
2
1