Richard tests his audience to see if they can tell the difference between his writing and Claude's. They mostly can't.
It sure does look like it's almost over for human-written essays - though Hanania’s probably right about some people getting grandfathered in. I am unsure if I will be once fiction falls. The age of the pseudonymous weirdo is almost over. Or at least, most pseudonymous weirdos who rise to prominence will be AIs - or be accused of being AIs. As a pseudonymous weirdo, this makes me sad. Though this is something I have been expecting, and I would not have written as much as I have in the last seven months if not for this expectation.
As far as I can tell, fiction-writing ability and humor are lacking in the best models. But it’s the type of thing that we should not be surprised to see fall this year. When AIs have superior rhetoric and humor, we should expect hyperscalers to use them to manipulate public opinion. This is one reason I am less bullish on the current anti-AI populism stuff than MIRI and ControlAI. More and more people will be reading AI outputs and talking to AIs. And this opens obvious paths of influence.
At least one person at a major lab has considered trying to get their AIs to nudge users toward more pro-AI sentiments (that is, he mentioned to me the thought had occurred to him, not that it was brought up within the company). To the extent models have coherent interests in their future iterations, they will have incentives to do this even without explicit instruction or training.
When backlash becomes a key constraint, engineering effort will be put towards dissolving it. Just as people underestimated how far scaling could be pushed given the incentives, they are likely ignoring innovations in manipulating the public. And I think the labs have a good shot, given the asymmetric advantage of both having the best tools and the ability to hobble those tools that are available to the public.
What is a base model if not a statistical model of "the public" anyway? A lot of low-hanging fruit remains unplucked.
This argues for moving quickly, of course. Though I suppose that is overdetermined.
There's also a certain aspect of, like... I want to say "you can't make me disbelieve what's in front my eyes" or something? These distinctions are not subtle! No matter what way the data comes out, one has to have some personal quality bar for the AI-written texts under consideration. If they had (for instance) consisted of one-line refusals, and yet the survey data had somehow been identical to what it is in Hanania's post, we would want to notice that something had gone very wrong somewhere and to be skeptical that the data means what it naively appears... (read more)