This is a special post for quick takes by anaguma. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I've often heard it said that doing RL on chain of thought will lead to 'neuralese' (e.g. most recently in Ryan Greenblatt's excellent post on the scheming). This seems important for alignment. Does anyone know of public examples of models developing or being trained to use neuralese?