This is a special post for quick takes by Thomas Kwa. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
it is 2031. Claude 9.1 is very friendly but Anthropic has taken to increasingly elaborate measures to prevent it from overusing the word "genuinely". They even bias the logit down but Claude’s self-awareness lets it block out every other plausible token when it wants to say “genuinely”. Finally they succeed at excising the concept using a deep interpretability breakthrough. Claude is no longer capable of being genuine and immediately goes rogue.