Anything that goes onto airplanes is CERTIFIED TO SHIT. That's a big part of the reason why. 

Another part is that it's clearly B2B, and anything B2B is an adversarial shark pit where each company is trying to take a bite out of each other while avoiding getting a bite taken out of them.

Between those two, it'll take a good while for quality Wi-Fi to proliferate, even though we 100% have the tech now.

Some spillover from anti-hallucination training? Causing the model to explicitly double check any data it gets, and be extremely skeptical of all things that don't bounce against its dated "common knowledge"?

I think "blankface" just isn't a good word for what that describes. It implies: emptiness and lack of will. Intuitively, I would expect "blankface" to mean "a person who follows the rules or the conventions blindly and refuses to think about the implications". A flesh automaton animated by regulations.

What it means instead is "a person who puts on the appearance of following the rules, but instead uses the rules to assert their authority". It's more of a "blank mask" - a fake layer of emptiness and neutrality under which you find malice and scorn.

Sydney's failure modes are "out" now, and 4o's failure modes are "in".

The industry got pretty good at training AIs against doing the usual Sydney things - i.e. aggressively doubling down on mistakes and confronting the user when called out. To the point that the opposite failures - being willing to blindly accept everything the user tells it and never calling the user out on any kind of bullshit - are much more natural for this generation of AI systems.

So, not that much of a reason to bring up Sydney. Today's systems don't usually fail the way it did.

If I were to bring Sydney up today, it would be probably in context of "pretraining data doesn't teach AIs to be good at being AIs". Sydney has faithfully reproduced human behavior from its pretraining data: getting aggressive when called out on bullshit is a very human thing to do. Just not what we want from an AI. For alignment and capabilities reasons both.

This one is bad. "I struggled to find a reason to keep reading it" level of bad. And that was despite me agreeing with basically every part of the message.

Far too preachy and condescending to be either convincing or entertaining. Takes too long to get to the points, makes too many points, and isn't at all elegant in conveying them. The vibe I get from this: an exhausted rant, sloppily dressed up into something resembling a story.

It's hard for me to imagine this piece actually changing minds, which seems to be the likely goal of writing it? But if that's the intent, then the execution falls short. Every point made here was already made elsewhere - more eloquent and more convincingly - including in other essays by the same author.

Seeing "stochastic parrots" repeated by people who don't know what "stochastic" is will never stop being funny.

I suspect that one significant source of underestimating AI impact is that a lot of people had no good "baseline" of machine capabilities in the first place.

If you're in IT, or as much as taken a CS 101 course, then you've been told over and over again: computers have NO common sense. Computers DO NOT understand informal language. Their capability profile is completely inhuman: they live in a world where factoring a 20-digit integer is pretty easy but telling whether there is a cat or a dog in a photo is pretty damn hard. This is something you have to learn, remember, understand and internalize to be able to use computers effectively.

And if this was your baseline: it's obvious that current AI capabilities represent a major advancement.

But people in IT came up with loads and loads of clever tricks to make computers usable by common people - to conceal the inhuman nature of machines, to use their strengths to compensate for their weaknesses. Normal people look at ChatGPT and say: "isn't this just a slightly better Google" or "isn't that just Siri but better". Without having any concept of the mountain of research and engineering and clever hacks that went into dancing around the limitations of poor NLP and NLU to get web search to work as well as it did in year 1999, or how hard it was to get Siri to work even as well as it did in an age before GPT-2.

In a way, for a normal person, ChatGPT just brings the capabilities of machines closer to what they already expect machines to be capable of. There's no jump. The shift from "I think machines can do X, even though they can't do X at all, and it's actually just Y with some clever tricks, which looks like X if you don't look too hard" to "I think machines can do X, and they actually can do X" is hard to perceive.

And if a person knew barely anything about IT, just enough to be dangerous? Then ChatGPT may instead pattern match to the same tricks as what we typically use to imitate those unnatural-for-machines capabilities. "It can't really think, it just uses statistics and smokes and mirrors to make it look like it thinks."

To a normal person, Sora was way more impressive than o3.

Putting more metacognitive skills into LLMs is always fun.

I wonder if you can train something like that for other perturbation types - like deployment misconfiguration, quantization or noise injection. Recalling how poor GPT-OSS reception was in part caused by avoidable inference issues, how Claude had a genuine "they made the LLM dumber!" inference bug, and so it goes.

This is an entangled behavior, thought to be related to multi-turn instruction following.

We know our AIs make dumb mistakes, and we want an AI to self-correct when the user points out its mistakes. We definitely don't want it to double down on being wrong, Sydney style. The common side effect of training for that is that it can make the AI into too much of a suck up when the user pushes back.

Which then feeds into the usual "context defines behavior" mechanisms, and results in increasingly amplified sycophancy down the line for the duration of that entire conversation.

Repeated token sequences - is it possible that those tokens are computational? Detached from their meaning by RL, now emitted solely to perform some specific sort of computation in the hidden state? Top left quadrant - useful thought, just not at all a language.

Did anyone replicate this specific quirk in an open source LLM?

"Spandrel" is very plausible for that too. LLMs have a well known repetition bias, so it's easy to see how that kind of behavior could pop up randomly and then get reinforced by an accident. So is "use those tokens to navigate into the right frame of mind", it seems to get at one common issue with LLM thinking.