To discredit writing because you suspect it’s AI is itself a sloppy heuristic—you should discredit writing because it’s bad.
A big reason for me, and which doesn't have to do with the quality, is that I can no longer trust that the human posting it actually had the sorts of thoughts that would be necessary to write the post.
And if they haven't had those thoughts, what's the point of engaging? [1]
Sure, maybe it's still worth engaging for persuasion or status-y type reasons, but that's usually not what interests me. Or just let me explicitly engage with the AI!
Imagine how obnoxious it would be if I had someone ghostwriting half of my comments and posts here... the fact that it's AI isn't even particularly relevant.
That's a great point. Seems like the category of 'informational writing' is not so homogeneous. There are at least:
* Evaluative contexts: "I am judging the author (are they worth engaging with? anything from cover letter screening to replying on blogposts), and their writing is the only available proxy to judge the author by."
* Expository contexts: "I am trying to learn an idea as best as I can."
Take the example of 'Falling in love feels good because of endogenous, not exogenous factors'. Even if written by AI, that could be useful + interesting in an expository context, for someone to whom that is new and subversive. But realizing it was written by AI would have a negative impact on any reader in an evaluative context.
I suppose I hadn't considered the distinction at all, and assumed a focus on expository writing -- though ironically, most blogposts are engaged with as evaluative instead!
Though, I think the point of the essay still stands, which is -- if AI was used well enough (as judged by a reader's independent evaluation of how interesting/good the idea is), that is still value delivered to the audience, which may not have existed otherwise.
Consider your footnote: "Imagine how obnoxious it would be if I had someone ghostwriting half of my comments and posts here" -- yes, it would be annoying to realize you were debating an automaton. But if you used AI to (1) write clarifying analogies (suppose your "ghostwriting half of my comments and posts here" example was the sort of example an AI could help surface), or (2) develop your opinions beyond what you previously held, I'm not sure a reader would get too annoyed!
But if you used AI to (1) write clarifying analogies (suppose your "ghostwriting half of my comments and posts here" example was the sort of example an AI could help surface), or (2) develop your opinions beyond what you previously held, I'm not sure a reader would get too annoyed!
Right, but neither of those cases necessitate using the AI's writing, which is the crucial distinction.
If I notice the writing is by AI, I don't know to what extent the purported author is using it to do their thinking for them. Maybe it's totally innocuous, like a non-native English speaker using it for translation. But it might also just be something lazily copied and pasted, not something they even read carefully. It scales much better for the lazy writers than the earnest writers.
Another thing is that it's easy to fool yourself into thinking the AI is just writing your own thoughts for you, but which turns out to be an illusion (I speak from experience).
Falling in love isn’t just about romance. It’s about discovering new parts of yourself.
If an adult wrote this couple of sentences, I could ask them what they were talking about, and maybe hear a story about their personal love life. That would almost certainly be interesting--even if expressed in clunky language.
If a nine-year old wrote the same, I wouldn't be so sure they knew what they were talking about, but I'd hold out hope for a charming anecdote about what they learned from their first crush.
But if I learned the nine-year-old had concluded it solely based on skimming all the books about love in their school library, then I would smile to myself and think, "Ah, how innocent: to write about love without having experienced it for oneself."
The official LessWrong stance on AI writing is quite insightful. A couple others have pointed at this but I'll try, too.
If an AI wrote something, it might well have also been the one that came up with or accepted the ideas. AI is terrible at that. It is a sophist, saying what sounds good and not what truly makes sense. And it's bad at telling the difference even if you ask it to.
Right now, human ideas beat AI ideas at the top end of the distribution. And that's what LessWrong is for.
So, I don't mind at all if you use AI to do your writing - as long as you somehow assure me that it had nothing to do with judging those ideas as valid and valuable.
I wish I had a link to that thread handy, it was quite insightful. I'd like ot use AI to help me write, but I see why it's a prejudice that's importantly accurate on average.
I assume you mean Policy for LLM Writing on LessWrong, thanks for directing me to that, it was interesting to read. It partly boils down to: a human must contribute value-add transformations at some point(s) of the AI writing 'supply chain', which I totally agree with.
I think its interesting you mentioned sophistry; I know the dictionary basically flattens it to "deception" but I've been interested for a while in its meaning as "style over substance". In particular, I think much of what makes someone feel subjective epiphany from an idea isn't usually the idea's substance, but by shaping its expression into a 'key' that fits their particular 'keyhole', or knowledge gap. Often this requires phrasing it as some form of "not X but opposite-of-X" (imagine my dismay when AI appropriated this after years of independent thinking on it...). Not to say substance doesn't matter. I think that often, the process of converting your idea-seed into this "not-but" form will frequently refine it into something clearer and better-scoped than it was before.
I think sophistry was originally a philosophical tradition that was heavily criticized for focusing on style over substance.
Interesting points on the not-but form, I'll try to try it!
Compare the reverse: “This sounds like AI. Is this writing bad?” That is paranoia.
The AI smell is a warning sign of problems with the writing itself. Sometimes, so loud a warning that it is not worth the effort to read any deeper. One does not seek an emperor in a village teahouse.
Also, if the text was written by AI, it means that none of your feedback will have an impact on author's next post. Your feedback is talking to a machine that isn't even listening.
To me that seems like the difference between text written by AI, and text written with AI. Selection as a steering mechanism for AI currently only works well with a human curator. This applies during writing (choosing between options, or choosing when to intervene with manual edits), but just as well applies after writing (seeing feedback on a piece, and learning how to improve your own curatorial instinct).
After that, the hope is that with future tools, curating a better past context window will itself lead to better future AI writing outputs. But I don't think the tools are there yet.
But the amount of AI writing in this post is… zero. Not for title ideation, not for promotional copywriting, not even for a single phrase, or even a fragment of a phrase anywhere in the piece.
Not quite. You explicitly quoted this as a piece of AI writing:
Falling in love isn’t just about romance. It’s about discovering new parts of yourself.
AI or human, that's a trite cliche. I'll take your word for it that you wrote the subsequent expansion into a whole paragraph without AI assistance, but the paragraph is no better than what an AI would come up with.
That's another hazard of AI slop: picking up its dull-witted habits by osmosis.
AI writing will gain usage because it has extraordinary capacity for good writing. But that only happens when you apply enough evolutionary selection pressure by exhibiting good taste.
AI has no capacity for good writing. (I am speaking of the here and now, not a hypothetical future.) You may set out to filter by good taste, but the process corrupts one's taste, and filters for people who did not have good taste to begin with.
To adapt Nietzsche, when you step into the abyss, the abyss steps into you.
Really appreciate the comment, I enjoyed engaging with it! See below:
Not quite. You explicitly quoted this as a piece of AI writing
Ahah, this made me laugh. Yes, that's true.
---
Falling in love isn’t just about romance. It’s about discovering new parts of yourself.
AI or human, that's a trite cliche. I'll take your word for it that you wrote the subsequent expansion into a whole paragraph without AI assistance, but the paragraph is no better than what an AI would come up with.
I think this illustrates my point.
---
That's another hazard of AI slop: picking up its dull-witted habits by osmosis.
I guess I can give you an advance preview of the next essay: this is the first point it makes. As in, this current essay covers how for reading AI writing, you should develop and trust your own taste / independent judgment of what counts as good, rather than using presence of AI as a proxy.
But when it comes to writing (topic of the next essay), the same prescription to 'develop and trust your taste' leads to a near-opposite perspective -- don't use AI. My closing line in this essay; "I handle [writing with AI] like poison"; segues into the following epigraph; a quote by Orwell written about the language of his time, but which maps nearly 1:1 for modern AI usage:
But if thought corrupts language, language can also corrupt thought. A bad usage can spread by tradition and imitation, even among people who should and do know better. The debased language that I have been discussing is in some ways very convenient. Phrases like a not unjustifiable assumption, leaves much to be desired, would serve no good purpose, a consideration which we should do well to bear in mind, are a continuous temptation, a packet of aspirins always at one’s elbow. Look back through this essay, and for certain you will find that I have again and again committed the very faults I am protesting against.
My bias against AI writing is one of those things built by experience. For example, I want to know something, so I run a web search. I land on a page that has the unmistakable flavor of AI. And I am virtually always disappointed by the content. It typically looks superficially good, but it's almost always low-effort bullshit. There will rarely be any actual insight, or facts beyond the most superficial and generic possible comments on the topic. And if the topic is even slightly off the beaten path, there will often be major errors and hallucinations. (For example, I was looking to find out how to upgrade the "class" of the newly introduced corvettes in No Man's Sky, and the AI articles I read were completely fictitious.)
I suspect that AI writing is usually generic slop because AIs are built to predict the "most likely token" at each step, which strongly biases their output towards mediocrity and predictability. And similarly, when they lack knowledge, they make up something plausible.
So my distaste for AI writing is a Bayesian phenomenon. I am estimating P(this writing is worth my time|this was obviously written by an AI). And as I keep encountering more awful slop, I keep updating that prediction downwards.
Note that this doesn't apply to non-native English speakers using AI to translate. There, the likelihood that the writing is worthwhile is based almost entirely on the original writing before translation.
All it would take to improve my opinion of AI writing would be to find myself regularly surprised and delighted by finding new, correct, and non-generic information in pieces obviously written by AIs.
I agree the most proliferate AI use case is SEO-spam/content-farming, and that almost no human input is invested to prune the mediocrity typically outputted. I see this as an alignment problem, rather than a problem with AI writing (ie, AI writing tools) in principle. Of course, AI writing out-of-the-box is bad, and a reasonable person should stop reading when they realize that anything they read fails to meet their quality bar.
My concern is when a Bayesian heuristic shifts to Bayesian epistemology, and people believe that "badness" is a property of AI writing, when in reality AI writing can become great. Particularly if you subscribe to a Popperian epistemology rooted in "alternating conjecture and criticism" as an engine for progress -- because you can choose to use AI tools in a way to emulate that cycle.
Regarding:
All it would take to improve my opinion of AI writing would be to find myself regularly surprised and delighted by finding new, correct, and non-generic information in pieces obviously written by AIs.
My view is we won't approach this anytime soon because the human curator's input, if done well, will transform the writing to no longer seem "obviously written by AI".
Currently, you tell ChatGPT what you want, then take what it gives. If you’re prudent, you’ll edit that. If you’re advanced, you might adjust the prompt and retry, throw on some custom instructions, or use handcrafted prompts sourced online. And if you’re really clever, you’ll ask the AI to write the prompt to ask itself. Yet after all that, the writing quality is almost always stilted anyway!
I don't think that summarizes best practice. I think it's an important step to ask the AI to ask you to clarify the argument and needs of your writing. You don't want a one-way conversation.
Yeah, that's good point. I was modeling stratifications of lay-usage I'm aware of; treating ChatGPT like a prose dispenser. But if I were to write a second edition of this post, I would include your point.
I think people know to turn AI's generative capabilities towards selection (ie, "edit this"). What you've described seems like: instead of turning AI's capability towards selection execution, its towards the selection process, (think, "working in the business" vs "working on the business"; the 'business' here is 'improving the writing'). Ie, you let the AI help you gain taste for what you even mean to say, and what you think is worth elaborating on or pruning away.
In any case, I think that builds naturally on the generation/selection framework established in this essay.
AI writing is not bad per se but it's a real concern for our cognitive capacities. Using our brains is important. I suggest to write as much as possible by ourselves.
I changed my caps lock key to a — key nearly a decade ago, and have done it on most of my keyboards/PCs ever since.
So seeing all the advice lately like "em dashes are a sign of AI writing" is a funny feeling to experience.
Isn't the consistent quality part of the problem? I associate smooth-talking with ability-to-bullshit. Good writers less so, but only because they're often personally interested in expressing something genuine. We can be bullshitted more easily now.
These days, it's status-boosting to notice when something was written by AI. Look at all those em dashes. Yep. Oh wow, “it’s not x, but y”; such a dead giveaway. I am so keen and discerning.
Shortform and longform content goes viral for flagging the signs of AI writing. Or rather, sins, because everyone is eager to oust the witches, is paranoid of the lowbrow-plagiarist accusations, and is self-righteous in deriding the verbal pollution.
But frankly, if a post is engaging or insightful, I don’t care if it was written by AI. Of course, I do care if it’s sloppy. But that’s just as evident when a human writes it au naturel.
To discredit writing because you suspect it’s AI is itself a sloppy heuristic—you should discredit writing because it’s bad. AI writing often is bad, but also often is not—you only ever notice when it’s done lazily. But blanket-discarding it has many problems.
First, the false positives. After two gentle em dashes, many people will quit reading and start hunting like the Terminator. Once you suspect AI’s invisible hand, you’ll start sensing it where it’s not. Now anything the piece could have taught you is tarnished.
Second, even if it was AI, constructions like “it’s not x, but y” (negative parallelisms) are idiomatic in English because insights always take the form of swapping misconceptions with the truth. E.g., these two topics are not disparate, but actually related, or an assumption I took for granted is not hidden, but actually legible; or is not static, but actually variable. So it’s unsurprising this structure gets rewarded by the AI models.
Take a writing sample from the article, 13 Signs You Used ChatGPT To Write That. It’s meant to show how “low-IQ” models are for blindly parroting negative parallelisms:
Falling in love isn’t just about romance. It’s about discovering new parts of yourself.
Yep, what a low-IQ—wait. That really is insightful!
That the pleasure of falling in love seems to stem externally from your relationship with your lover (you’re charmed by how attractive they are), but really, it stems internally from the new identities you inhabit (you enjoy becoming someone who flirts, who yearns, who feels alive).[1]
Which is to say, many AI writing tropes are human writing tropes, which often exist because we like them.
Lastly, it’s a mistake to lock in a disdain for AI writing now, while many of the faults are transitory: byproducts of nascent tools.
Currently, you tell ChatGPT what you want, then take what it gives. If you’re prudent, you’ll edit that. If you’re advanced, you might adjust the prompt and retry, throw on some custom instructions, or use handcrafted prompts sourced online. And if you’re really clever, you’ll ask the AI to write the prompt to ask itself. Yet after all that, the writing quality is almost always stilted anyway!
Quality in any domain emerges from evolution: iterative generation and selection. But given our current prompt-and-response interfaces, it’s clear we presently live in the impending past. Chatbots weren’t designed for high-quality content: they’re alright at generation (but can be incorrigible to steer), and horrible at enabling selection (there’s too much friction to (a) keep asking for (b) plenty of (c) tailored outputs to choose from).
One vision of the future is tree interfaces. In 2021 (a year before ChatGPT existed), a pseudonymous AI researcher named Janus designed Loom, software to explore branching paths of AI outputs.[2] It works by generating as little as one sentence at a time, yet up to a dozen or more alternatives at once. Choose one and repeat, or backtrack and trace another path.
The generation bottleneck is paradoxically solved by selection: don’t describe what you want; instead, prune a context window from which what you want will naturally flow.[3] You constrain possible futures only to those which plausibly follow from a high-quality past.
And the selection bottleneck is solved by generation: don’t try one-shotting a multi-paragraph response; instead, get plenty of options per sentence. You produce a greater density of decision points to exert your judgment in, each with more material for your judgment to express itself through.
AI becomes leverage on your judgment.
The future appears to be a world where creation cedes ground to curation. In their 2021 post Quantifying Curation, Janus finds:
It is an interesting and unintuitive property of large language models that their stochastic completions to the same prompt can vary from nonsense to super-human – we might instead expect an AI of infra-human capability to consistently produce infra-human content, the way a person with a weak understanding of a topic is unlikely to say something “accidentally” indistinguishable from an expert. But the learning curves of language models have very different properties than that of humans.
AI writing will gain usage because it has extraordinary capacity for good writing. But that only happens when you apply enough evolutionary selection pressure by exhibiting good taste.
However, an unwillingness to engage with AI writing is not an expression of your taste, but an insecurity with it. If the writing is good, it’s a guardrail against accidentally “falling for” or enjoying AI: you’re scared to think for yourself. If the writing is bad, then rejecting it for being AI is a shallow proxy that absolves you from articulating what is bad about it.
If anything, you should sublimate a disdain for AI into a disdain for poor writing generally; raising your standards. If you ever think: “This writing sounds bad. Is it AI?” Don’t even wait to investigate the answer; just drop the piece. That is principled. Compare the reverse: “This sounds like AI. Is this writing bad?” That is paranoia.
There’s nothing to fear once you realize that AI is just a force multiplier on your judgment. And that if someone’s AI writing looks bad, it’s not a reflection of AI’s abilities, but a reflection of the author’s low effort. AI is a tool, a human still has to use it well.
After this entire tractate, you may have been expecting the inevitable twist of the genre: And guess what? This post was itself… written by AI 🫳🎤.
But the amount of AI writing in this post is… zero. Not for title ideation, not for promotional copywriting, not even for a single phrase, or even a fragment of a phrase anywhere in the piece.
I handle it like poison.
Reading AI writing is completely different from writing with AI.
Part II is here: And Yet, Defend your Thoughts from AI Writing
Esther Perel, famed relationship therapist, has a related insight when she explains Why Happy People Cheat: “Sometimes when we seek the gaze of another, it’s not our partner we are turning away from, but the person we have become. We are not looking for another lover so much as another version of ourselves.” Similarly, Ye (formerly Kanye West), famed relationship therapist-needer, observed in his song Broken Road: “Love is a version of bein' a virgin again”. So AI landing on this point is rivalling our expert-level benchmarks on the human condition.
This also applies to humans. Rick Rubin opens his book The Creative Act with a quote by the painter Robert Henri: “The object isn’t to make art, / it’s to be in that wonderful state / which makes art inevitable.” The same happens when you try to invent a joke on the spot. AI can’t do it well, but nor could you. Yet both can make jokes all the time when immersed in some context, passively scanning for connections or subversions.