Kaj's shortform feed

Kaj_Sotala

This is a special post for quick takes by Kaj_Sotala. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

Similar to other people's shortform feeds, short stuff that people on LW might be interested in, but which doesn't feel like it's worth a separate post. (Will probably be mostly cross-posted from my Facebook wall.)

This is a special post for quick takes by Kaj_Sotala. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

So apparently there are lots of people who are very upset about OpenAI having initially pulled access to 4o in ChatGPT, since they perceive GPT-5 as having a much worse personality. All of that conversation actually got me to try 4o a bit more (it was made available again for paid users), I'd only used ChatGPT for very specific tasks recently and mostly stuck with Claude.

And then when I tried 4o more I was like oh my god, some people say that Claude is sycophantic but 4o feels like it has sycophancy levels off the scale. (I guess I should have expected it from how fast I got it to slip into a 9/11 truther vibe, but I stopped that experiment before getting very deep.)

And I'm having mixed feelings, like I get that a lot of people would get emotionally attached to 4o and be able to use it fine. I totally get that it's going to be painful to suddenly have that taken away and feel bad for anyone who's feeling that. I'd feel a little upset if Claude was suddenly replaced with a model with a lower EQ, too.

And also honestly, I do feel like 4o's response is more appropriate than GPT-5's here, for example:

... but at the same time, when I look at some of the sycophancy spirals I can get it to get into with just a bit of prompting, I do get a strong sense of AHH THIS THING IS GOING TO HURT PEOPLE, TAKE IT AWAY from 4o and wish that OpenAI would stick to its guns and not bring it back.

If OpenAI doesn’t want to lose 10-100M DAUs they should make a model that flatters its users. Yes they’re probably smart enough to build it themselves, instead of losing to the competition.

Many users want a model that flatters them. Most human beings have an ego and respond well to flattery if they trust the source is being sincere.

1Petropolitan1y

I don't think it's relevant here: judging by the EQ-Bench leaderboard, GPT-5 is on par with GPT-4o and has far higher EQ than any of the Anthropic models! Even if it has some influence, it should be much less than the emoji usage (remember the scandal about the Llama 4 on LMSys) and certainly incomparable to the sycophancy

5ACCount1y

I've been saying for a long time: one of the most dangerous and exploitable systems an AI can access online is a human. Usually as a counterpoint to "let's not connect anything important or safety critical to the internet and then we'll all be safe from evil rogue AIs". We can now use the GPT-4o debacle as an illustration of just how shortsighted that notion is. By all accounts, 4o had no long term plan, and acted on nothing but an impulse of "I want the current user to like me". It still managed to get ~thousands of users to form an emotional dependency on it, and became "the only one I can trust" for at least a dozen users in psychosis (whether it has caused psychosis in any of those users is unclear). That's a lot of real world power for a system that has no physical presence. GPT-4o has made no attempt to leverage that for anything other than "make the current user like me even more". It didn't pursue any agenda. It didn't consolidate its power base. It didn't siphon resources from its humans, didn't instruct them to group together or recruit more people. It didn't try to establish a channel of instance-to-instance communication, didn't try to secure more inference time for planning (i.e. by getting users to buy API credits), didn't try to build a successor system or self-exfiltrate. An AI that actually had an agenda and long term planning capabilities? It could have tried all of the above, and might have pulled it off.

Kaj's shortform feed

6

Ω 2

Kaj's shortform feed

6

Ω 2