It seems like Model CoTs are getting increasingly unintelligible over time (e.g., OpenAI now gives a "translation" of the CoT in any communication containing CoTs). It also seems like we can train them to be more legible by adding a legibility penalty (eg. Meta's new approach). So why aren't labs training CoTs to be more legible?
I understand that we don't want to train CoTs not to show harmful thoughts because then we lose the ability to catch them when they are thinking harmful thoughts. But this concern doesn't apply when we are trying to make them more legible (and therefore easier to catch)?