TL;DR: Wherever calibrated rare defection drives an agent's strategy, its calibration mechanism may leave a fingerprint whenever it considers defection. I prototype a probe which might help focus expensive analysis toward actual defection opportunities, or it might be confounded by ASCII tables. Five trajectories, drawn with random.Random(42).sample over the full...
TL;DR: ARC's post does excellent work motivating a p(doom) estimator equal or better than random sampling; however, they evaluate p(doom) over a naive distribution of inputs, leaving them open to test-deploy asymmetry attacks. Trojan theory and cybersecurity practice suggest a lens and compare mitigation options. Context: I really admire ARC's...
TL;DR: The IOI algorithm doesn't specify what to do when the indirect object token is duplicated. I observe that the model succeeds anyway. I'd like to know if I'm mistaken in the IOI paper's predictions, and I'd like to know what evidence you'd reach for next. Background Wang et al....