I’ve been thinking about the word ‘slop’ a lot recently. I have a strong aesthetic distaste for the word, and it lacks nuance - clearly there is huge variation in the quality of AI content, and at least some of it is actually quite good. I got an odd email the other day, one which highlighted to me that I have not fully thought through the implications of slop. It referenced a post I’d made about AI, and asked if I wanted to try an app that the sender had allegedly made, including offering me a free trial etc. It had one pretty big tell in terms of linguistic content and the general low-effort vibe of the email was also a giveaway that it was AI-generated, but that on its own wouldn’t be that interesting to me; this happens regularly already. What was more interesting about this email was that it was signed off with a real human name. I was curious, and so I looked up the domain that the sender was coming from. This revealed a company with two employees: a founder with various links to his Twitter, his personal website, his LinkedIn, etc., and the head of development, who had emailed me.

The latter didn’t have any links on her profile, and she had a fairly generic Japanese name. Interestingly, she also had a picture, notably smaller than the founder’s, such that it was quite hard to tell that it was obviously an AI-generated image. Using DevTools I could see the image at a larger size, and lo and behold, it was obviously AI-generated. Not only that, it was an extremely conventionally attractive Japanese woman. I thought it was pretty disingenuous that nowhere did the site or the email disclose that this was obviously an AI masquerading as a real person. I Googled the name of the employee - as I said, it was quite generic, so there were many results that obviously did not pertain to this AI persona, but there was one for an interview with a magazine that profiled aspirational and influential women. They had run a profile on this person, who clearly was not real. The picture they’d used on the profile was huge, 1600x1600px, and was just extremely obviously AI-generated. To be 100% sure, I ran it through Pangram, and yeah, 100% AI. Similarly, all the answers to the questions that had been conducted in the interview were, again, obviously AI-generated. I even wondered if the creator of this AI persona had also created the magazine to try and manufacture some kind of credibility. It didn’t appear to be that way, but with that being said, the magazine also didn’t appear to be a particularly respectable publication.

This was an interesting experience overall to me. I felt glad it was obvious that this whole thing was AI-generated from the get-go, but it also made me realise that soon it won’t be obvious, not only to me, but to the vast majority of people. It will be completely feasible to have AIs masquerading as real people, and to have them create companies. We already talk about AI running companies in the abstract, and when I’ve thought about it previously, I have imagined these companies as being different from regular human companies in some way. Sure, maybe they have some structural similarities to regular companies, but the kind of tasks, the kind of roles that are undertaken in that company are clearly very different because they’re suited to AI, and everything about them makes it obvious that they’re being undertaken by an AI. But maybe this won’t be the case. Maybe, actually, we will have these strange, uncanny-valley companies where everything is set up to try to look as much like a human company as possible.

I don’t really know what this means. It makes me feel uncomfortable. I’m worried that collectively we do not have the wisdom and the maturity to try to adapt to this, and I’m worried that there are people who will try to take advantage of this and use this to manipulate and scam and propagandise. For the past couple of years, I hadn’t been especially worried about a big increase in this kind of scam or manipulation. My sense was that the bottleneck to criminals running more of these attacks wasn’t one that could be solved with AI. It seemed like GPT-4 was sufficiently intelligent to scale up this kind of attack, if intelligence really was the bottleneck, and my understanding was that we didn’t see a big uptick. But now I’m starting to see this kind of thing in the wild, and indeed, these agents are reaching out to me on a regular basis. It worries me a lot.

I recently wrote a post where I said that I thought things might get pretty weird in the next couple of years, even before we hit RSI. But actually, I think things are already pretty weird, and they’re only going to get weirder.