We think this occurs because in general there are groups of belief states that are degenerate in the sense that they have the same next-token distribution. In that case, the formalism presented in this post says that even though the distinction between those states must be represented in the transformers internal, the transformer is able to lose those distinctions for the purpose of predicting the next token (in the local sense), which occurs most directly right before the unembedding.

I wonder if you could force the Mixed-State Presentation to be "conserved" in later layers by training the model with different objectives. For instance, training on next-token prediction and next-token-after-that prediction might force the model to be a lot more "rigorous" about its MSP.

Papers from Google have shown that you can get more predictable results from LLMs if you train then on both next-token prediction and "fill-the-blanks" tasks where random tokens are removed from the middle of a text. I suspect it would also apply here.

Or maybe speaking french automatically makes you healthier. I'm gonna choose to believe it's that one.

Seed oil folks often bring up the French paradox, the (controversial) claim that French people are/were thin and have low cardiovascular disease despite eating lots of saturated-fat-rich croissants or whatever.

As a French person hearing about this for the first time, that claim indeed seems pretty odd.

If I was asked to list the lifestyle differences between France and the US with the most impact on public health, I would think of lower car dependency, higher access to farmer's markets, stricter regulations on industrial food processing (especially sugar content in sodas), smaller portions served in restaurants, pharmacies not doubling as junk food shops, the absence of food deserts, public health messaging (eg every junk food ad having a "please don't eat this, kids" type disclaimer) etc... way before I thought of the two croissants a week I eat.

Viennoiseries are an occasional food for most people, not a staple. Now if you wanted to examine a french-specific high-carb staple, baguettes are a pretty good options: almost all middle-class households buy one a day at least.

Did you ever get one of your clients to use the "Your honor, I'm very sorry, I'll never do it again" line?

This was not at all obvious from the inside. I can only imagine a lot of criminal defendants have a similar experience. Defense attorneys are frustrated that their clients don't understand that they're trying to help—but that "help" is all within the rules set by the justice system. From the perspective of a client who doesn't think he did anything particularly wrong (whether or not the law agrees), the defense attorney is part of the system.

I mean... you're sticking to generalities here, and implying that the perspective of the client who thinks he didn't do anything wrong is as valid as any other perspective.

But if we try to examine some specific common case, eg: "The owner said you robbed his store, the cameras showed you robbing his store, your fingerprints are on the register", then the client's fury at the attorney "working with the prosecutor" doesn't seem very productive?

The problem isn't that the client is disagreeing with the system about the moral legitimacy of robbing a store. The problem is that the client is looking for a secret trick so the people-who-make-decisions-about-store-robberies will think he didn't rob the store and that's not gonna happen.

With that in mind, saying the attorney is "part of the system" is... well, maybe it's factually true, but it implicitly blames the robber's predicament on the system and on his attorney in a way that just doesn't make sense. The robber would be just as screwed if he was represented by eg his super-wealthy uncle with a law degree who loves him dearly.

(I don't know about your psychiatric incarceration, so I'm not commenting on it. Your situation is probably pretty different to the above.)

“Well, when we first met, you told me that you never touched the gun,” I reminded him with an encouraging smile. “Obviously you wouldn’t lie to your own lawyer, and so what I can do is get a fingerprint expert to come to the jail, take your prints, then do a comparison on the gun itself. Since you never touched the gun, the prints won’t be a match! This whole case will get dismissed, and we can put all this behind you!”

For the record, I am now imagining you as Bob Odenkirk while you're delivering that line.

The point about task completion times feels especially insightful. I think I'll need to go back to it a few times to process it.

I think Duncan's post touches on something this post misses with its talk of "social API": apologies only work when they're a costly signal.

The people you deliver the apology to need to feel it cost you something to make that apology, either pride or effort or something valuable; or at least that you're offering to give up something costly to earn forgiveness.

The slightly less machiavellian version is to play Diplomacy with them.

(Or do a group project, or go to an escape game, or any other high-tension low-stakes scenario.)

I think "API calls" are the wrong way to word it.

It's more that an apology is a signal; to make it effective, you must communicate that it's a real signal reflecting your actual internal processes, and not a result of a surface-level "what words can I say to appear maximally virtuous" process.

So for instance, if you say a sentence equivalent to "I admit that I was wrong to do X and I'm sorry about it, but I think Y is unfair", then you're not communicating that you underwent the process of "I realized I was wrong, updated my beliefs based on it, and wondered if I was wrong about other things".

I'm not entirely sure what the simplest fix is

A simple fix would be "I admit I was wrong to do X, and I'm sorry about it. Let me think about Y for a moment." And then actually think about Y, because if you did one thing wrong, you probably did other things wrong too.