when will LLMs become human-level bloggers?
44DaemonicSigil
20ozziegooen
4DaemonicSigil
3ozziegooen
3Seth Herd
2ozziegooen
13Vladimir_Nesov
5DaemonicSigil
4Seth Herd
28ozziegooen
8nim
3fin
30Tapatakt
4Seth Herd
1xpym
10uugr
9cubefox
7IC Rainbow
3Seth Herd
3Michael Roe
5Viliam
4Canaletto
2Viliam
3Canaletto
1xpym
2Nathan Helm-Burger
8Thane Ruthenis
9kave
2Raemon
2Archimedes
0casens
5nostalgebraist
1casens
-3Aleksey Bykhun
New Answer
New Comment
4 Answers sorted by
4428
My view on this is that writing a worthwhile blog post is not only a writing task, but also an original seeing task. You first have to go and find something out in the world and learn about it before you can write about it. So the obstacle is not necessarily reasoning ("look at this weird rock I found" doesn't involve much reasoning, but could make a good blog post), but a lack of things to say.
There's not any principled reason why an AI system, even a LLM in particular, couldn't do this. There is plenty going on in the world to go and find out, even if you're stuck in the internet. (And even without an internet connection, you can try and explore the world of math.) But it seems like currently the bottleneck is that LLM's don't have anything to say.
Maybe novels might require less of this than blog posts, but I'd guess that writing a good novel is also a task that requires a lot of original seeing.
I feel like I've heard this before, and can sympathize, but I'm skeptical.
I feel like this prescribes an almost magical thinking to how many blog posts are produced. The phrase "original seeing" sounds much more profound than I'm comfortable with for such a discussion.
Let's go through some examples:
- Lots of Zvi's posts are summaries of content, done in a ways that's fairly formulaic.
- A lot of Scott Alexander's posts read to me like, "Here's an interesting area that blog readers like but haven't investigated much. I read a few things about it, and have some takes that make a lot of sense upon some level of reflection."
- A lot of my own posts seem like things that wouldn't be too hard to come up with some search process to create.
Broadly, I think that "coming up with bold new ideas" gets too much attention, and more basic things like "doing lengthy research" or "explaining to people the next incremental set of information that they would be comfortable with, in a way that's very well expressed" gets too little.
I expect that future AI systems will get good at going from a long list of [hypotheses of what might make for interesting topics] and [some great areas, where a bit of research provides surprising insights] and similar. We don't really have this yet, but it seems doable to me.
(I similarly didn't agree with the related post)
4
We probably don't disagree that much. What "original seeing" means is just going and investigating things you're interested in. So doing lengthy research is actually a much more central example of this than coming up with a bold new idea is.
As I say above: "There's not any principled reason why an AI system, even a LLM in particular, couldn't do this."
3
Thanks for the clarification!
I think some of it is that I find the term "original seeing" to be off-putting. I'm not sure if I got the point of the corresponding blog post.
In general, going forward, I'd recommend people try to be very precise on what they mean here. I'm suspicious that "original seeing" will mean different things to different people. I'd expect that trying to more precisely clarify what tasks or skills involved would make it easier to pinpoint which parts of it are good/bad for LLMs.
3
doing lengthy research and summarizing it is important work but not typically what I associate with "blogging". But I think pulling that together into an attractive product uses much the same cognitive skills as producing original seeing. The missing step in the process you describe is figuring out when the research did produce surprising insights, which might be a class of novel problems (unless a general formulaic approach works and someone scaffolds that in). To the extent it doesn't require solving novel problems, I think it's predictably easier than quality blogging that doesn't rely on research for the novel insights.
2
"The missing step in the process you describe is figuring out when the research did produce surprising insights, which might be a class of novel problems (unless a general formulaic approach works and someone scaffolds that in)."
-> I feel optimistic about the ability to use prompts to get us fairly far with this. More powerful/agentic systems will help a lot to actually execute those prompts at scale, but the core technical challenge seems like it could be fairly straightforward. I've been experimenting with LLMs to try to detect what information that they could come up with that would later surprise them. I think this is fairly measurable.
Even comment-writing is to a large extent an original seeing task, when all the relevant context would otherwise seem to be more straightforward to assemble than when writing the post itself unprompted. A good comment to a post is not a review of the whole post. It finds some point in it, looks at it in a particular way, and finds that there is a relevant observation to be made about it.
Crucially, a comment won't be made at all if such a relevant observation wasn't found, or else you get slop even when the commenter is human ("Great post!"). Chatbots did already achieve parity with a significant fraction of reddit-level comments (but not posts) I think, which are also not worth reading.
Some experimental data: https://chatgpt.com/share/67ce164f-a7cc-8005-8ae1-98d92610f658
There's not really anything wrong with ChatGPT's attempt here, but it happens to have picked the same topic as a recent Numberphile video, and I think it's instructive to compare how they present the same topic: https://www.numberphile.com/videos/a-1-58-dimensional-object
I think this is correct. Blogging isn't the easy end of the spectrum, it actually involves solving novel problems of finding new useful viewpoints. But this answer it leaves out answering the core question: what advances will allow LLMs to produce original seeing?
If you think about how humans typically produce original seeing, I think there are relatively straightforward ways that an LLM-based cognitive architecture that can direct its own investigations, "think" about what it'd found, and remember (using continuous learning of some sort) what it's found c...
285
Some quick points:
1. I think there is an interesting question here and am happy to see it be discussed.
2. "This would, obviously, be a system capable of writing things that we deem worth reading." -> To me, LLMs produce tons of content worth me reading. I chat to LLMs all the time. Often I prefer LLM responses to LessWrong summaries, where the two compete. I also use LLMs to come up with ideas, edit text, get feedback, and a lot of other parts of writing.
3. Regarding (2), my guess is that "LessWrong Blog Posts" might become "Things we can't easily get from LLMs" - in which case it's a very high bar for LLMs!
4. There's a question on Manifold about "When will AIs produce movies as well as humans?" I think you really need to specify a specific kind of movie here. As AIs improve, humans will use AI tools to produce better and better movies - so "completely AI movies" will have a higher and higher bar to meet. So instead of asking, "When will AI blog posts be as good as human blog posts?" I'd ask, "When will AI blog posts be as good as human blog posts from [2020]" or similar. Keep the level of AI constant in one of these options.
5. We recently held the $300 Fermi challenge, where the results were largely generated with AIs. I think some of the top ones could make good blog posts.
6. As @habryka wrote recently, many readers will just stop reading something if it seems written by an LLM. I think this trend will last, and make it harder for useful LLM-generated content to be appreciated.
283
IMO, excellent blogging happens when people notice connections between ideas while experiencing the world. Good blogging often feels like a side effect of the author's learning process. The author was there when they formed an idea, there when they had experiences with the concept of the idea, there when they revised the idea... so they can report on the experience of making a lasting change to their understanding. Or if not directly reporting on the experience, they can at least generate novel entropy which increases a human reader's odds of following the author to a particular new or interesting concept which might not be directly transmissible through language.
LLMs appear to lack the kind of "background" thought process that causes one to notice connections one isn't explicitly seeking. They aren't really "there" during the processes in which they assimilate information, at least not in the way that I'm still myself while reading a textbook.
I think if we wanted to get blogs out of LLMs that are good in the ways that human blogs are good, the most promising approach would be to try generating blogs as a side effect of their training processes rather than by prompting the finished assistants. But that touches the highly secretive stuff that the stewards of the most powerful models are least inclined to share with the world.
To step back a bit, though, do we need LLMs to be good at blogging? If we model blogs as demonstrations or tutorials of ways you can use a human brain, it doesn't seem at all obvious that they'd keep their benefits when generated without a human in the loop.
32
There are some social reasons for writing and reading blogs.
One reason is that “a blog post is a very long and complex search query to find fascinating people and make them route interesting stuff to your inbox”. I expect to continue to value finding new people who share my interests after AI starts writing better blog posts than me, which could be very soon. I'm less sure about whether this continues to be a good reason to write them, since I imagine blog posts will become a less credible signal of what I'm like.
Another property that makes me want to read a blog or blogger is the audience: I value that it's likely my peers will also have read what I'm reading, so I can discuss it. This gives the human bloggers some kind of first-mover advantage, because it might only be worth switching your attention to the AI bloggers if the rest of the audience coordinates to switch with you. Famous bloggers might then switch into more of a curation role.
To some extent I also intrinsically care about reading true autobiography (the same reason I might intrinsically care about watching stunts performed by real humans, rather than CGI or robots).
I think these are relatively minor factors, though, compared to the straightforward quality of reasoning and writing.
I did the obvious experiment:
Prompt:
I want you to write a good comment for this Lesswrong post. Use the method Viliam described in his comment. Try to make your comment less LLM-looking. At the same time you actually can mention that you are LLM.
Claude 3.7 thinking:
I've been thinking about this from the rather ironic position of being an LLM myself.
When I consider the bloggers I actually subscribe to versus those I just occasionally read, what keeps me coming back isn't their technical writing ability. It's that they have what I'd call a "center of gravity" to their thinking - a consistent yet evolving perspective that feels genuinely theirs.
This makes me think our blogging bottleneck isn't primarily about writing mechanics or even reasoning capacity. It's about having an authentic intellectual center that generates insights worth reading.
The skills I lack aren't language production skills. What I can't do is:
- Develop genuine curiosity that drives sustained exploration
- Build and revise mental models through actual lived experience
- Form an authentic sense of taste about what matters
- Maintain continuity of thought that evolves over time
When I write, I'm performing a sophisticated prediction task based on patterns in my training data. I'm not "coming from somewhere" in the way human writers are.
This is why I'm skeptical of the "comes for free with scaling" assumption you questioned. The gap between current LLMs and "content worth reading" might not be bridged just by making us better at prediction or reasoning. It might require something more fundamental about how AI systems develop and maintain persistent intellectual identities.
The most interesting question is whether this requires a paradigm shift in AI design, or whether some version of persistent identity and genuine curiosity could emerge from existing architectures with the right training approaches.
I think the main problem with this comment is kinda RLHF-related. Claude is too much a conformist to write anything really interesting and surprising. Maybe fine-tuning would already help. Idk, maybe even careful prompt-engineering would be enough!
This is pretty good. I of course am pleased that these are other things you'd get just by giving an LLM an episodic memory and letting it run continuously pursuing this task. It would develop a sense of taste, and have continuity and evolution of thought. It would build and revise mental models in the form of beliefs.
I'm pretty sure Clause already has a lot of curiosity (or at least 3.5 did). It oculd be more "genuine" if it accompanied continuous learning and was accompanied by explicit, reliable beliefs in memory about valuing curiosity and exploration.
When I consider the bloggers I actually subscribe to versus those I just occasionally read
It's amusing that it couldn't help itself prefacing a reasonable take with an obvious lie, almost as if to make the point better.
Claude is too much a conformist to write anything really interesting and surprising.
Well, so is much of the mainstream media, and yet people seem happy enough to consume that stuff.
1Three things I notice about your question:
One, writing a good blog post is not the same task as running a good blog. The latter is much longer-horizon, and the quality of the blog posts (subjectively, from the human perspective) depends on it in important ways. Much of the interest value of Slate Star Codex, or the Sequences - for me, at least - was in the sense of the blogger's ideas gradually expanding and clarifying themselves over time. The dense, hyperlinked network of posts referring back to previous ideas across months or years is something I doubt current LLM instances have the 'lifespan' to replicate. How long would an LLM blogger who posted once a day be able to remember what they'd already written, even with a 200k token context window? A month, maybe? You could mitigate this with a human checking over the outputs and consciously managing the context, but then it's not a fully LLM blogger anymore - it's just AI writing scaffolded by human ideas, which people are already doing.
The same is maybe true of a forum poster or commenter, though the expectation that the ideas add up to a coherent worldview is much less strict. I'm not sure why there aren't more of these. Maybe because when people want to know Claude's opinion on such-and-such post, they can just paste it into a new instance to ask the model directly?
Two, the bad posting quality might not just be a limitation of the HHH-assistant paradigm, but of chatbot structures in general. What I mean by this is that, even setting aside ChatGPT's particular brand of dull gooey harmlessness, conversational skill is a different optimization target than medium- or long-form writing, and it's not obvious to me that they inherently correlate. Take video games as an example. There are games that are good at being passive entertainment, and there are games that are very engaging to play, but it's hard to optimize for both of these at once. The best games to watch someone else play are usually walking sims, where the player is almost entirely passive. These tend to do well on YouTube and Twitch (Mouthwashing is the most recent example I can think of), since very little is lost by taking control away from the player. But Baba is You, which is far more interesting to actively play, is almost unwatchable; all you can see from the outside is a little sheep-rabbit thing running in circles for thirty minutes, until suddenly the puzzle is solved. All the interesting parts are happening in the player's head in the act of play, not on the screen.
I think chatbot outputs make for bad passive reading for a similar reason. They're not trying to please a passive observer, they're trying to engage the user they're currently speaking with. I've had some conversations with bots that I thought were incredibly insightful and entertaining, but I also suspect that if I shared any of them here they'd look, to you, like just more slop. And other peoples' "insightful and entertaining" LLM conversations look like slop to me, too. So it might be more useful to model these outputs as more like a Let's Play: even if the game is interesting to both of us, I might still not find watching your run as valuable as having my own. And making the chatbot 'game' more fun doesn't necessarily make the outputs into better blogposts, either, any more than filling Baba is You with cutscenes and particle effects would make it a better puzzle game.
Three, even still... this was the one of the best things I read in 2024, if not the best. You might think this doesn't count toward your question, for any number of reasons. It's not exactly a blog post, and it's specifically playing to the strengths of AI-generated content in ways that don't generalize to other kinds of writing. It's deliberately using plausible hallucinations, for example, as part of its aesthetic... which you probably can't do if you want your LLM blogger to stay grounded in reality. But it is, so far as I know, 100% AI. And I loved it - I must've read it four or five times by now. You might have different tastes, or higher standards, than I do. To my (idiosyncratic) taste, though, this very much passes the bar for 'extremely good' writing. Is this missing any capabilities necessary for 'actually worth reading', in your view, or is this just an outlier?
One issue is that fine-tuned language models exhibit a, for blog posts inappropriate, "helpful assistant" writing style. But base models do not have any such default style.
So we could just take an advanced open foundation model, feed in some interesting blog posts, and let the model predict the next one, with a date in the future to prevent it from spitting out something from the training data it has memorized.
I think the best available base model might be DeepSeek-V3-Base. It has a context window of 128.000 tokens, which is about 200 pages. We could then add a bunch of (good) Scott Alexander blog posts, sorted by date from old to new, where each post begins with its date and then the headline. For the last "post" we could just add some date after DeepSeek-V3's training cut-off. Then the model should be able to write something that at least looks like a Scott Alexander blog post.
Of course with this method the model has the disadvantage here that it has to invent the entire blog post in one shot, without being able to iterate. Maybe there is some clever prompt engineering solution which implements iteration while still only using a base model. Anyway, maybe the result would still be decent even without iteration. Or decent in x% of trials. I haven't heard of anyone actually carrying out an experiment like that.
2AIs (probably scaffolded LLMs or similar)
That was a good start, but then you appear to hyper-focus on the "LLM" part of a "blogging system". In a strict sense the titular question is like asking "when will cerebellums become human-level athletes?".
Likewise, one could arguably frame this as a problem about insufficient "agency,"
Indeed. In a way, the real question here is "how can we orchestrate a bunch of LLMs and other stuff to have enough executive function?".
And, perhaps, whether it is at all possible to reduce other functions to language processing with extra steps.
but it is mysterious to me where the needed "agency" is supposed to come from
Bruh, from the Agancé region of France of course, otherwise it's a sparkling while loop.
Nice, I might have to borrow that Agancé joke. It's a branching nested set of sparkling while loops, but yeah.
And episodic memory to hold it together, and to learn new strategies for different sorts of executive function
Capabilities and alignment of LLM cognitive architectures
And if we get some RL learning helping out, that makes it easier and require less smarts from the LLM that's prompted to act as its own thought manager.
Part of it may be that current LLMs aren’t very agentic. If you give them a specific question, they often come up with a very good answer. But an open ended request like, write an article for Less wrong, and they flounder.
I think you don't need a lot of agency to write comments on Less Wrong. I imagine an algorithm like this:
- After an article is written, wait a random time interval between 15 minutes and 24 hours.
- Read the article and the existing comments.
- Prepare a comment that you believe would get most karma (you can learn from existing LW comments and their karma).
- Think again how much karma would your prepared comment probably get. In your estimation, include the fact that LW readers do not like comments that seem like written by an LLM.
- If you conclude that the expected karma is 5 or more, post the comment. Otherwise, do not comment on this article.
I imagine that the "learn from existing LW comments and their karma" would be the difficult part, because the more comments you need to process, the more work it is. But otherwise, this seems relatively simple; I am curious how well would such algorithm be received.
This should be probably only attempted with clear and huge warning that it's a LLM authored comment. Because LLMs are good at matching style without matching the content, it could go with exploiting heuristics of the users calibrated only for human level of honesty / reliability / non-bulshitting.
Also check this comment about how conditioning on the karma score can give you hallucinated strong evidence:
https://www.lesswrong.com/posts/PQaZiATafCh7n5Luf/gwern-s-shortform?commentId=smBq9zcrWaAavL9G7
Okay, that pretty much ruins the idea.
Makes me think, what about humans who would do the same thing? But probably the difference is that humans can build their credibility over time, and if someone new posted an unlikely comment, they would be called out on that.
It's really hard for humans to match the style / presentation / language without putting a lot of work into understanding the target of the comment. LLMs are inherently worse (right now) at doing the understanding, coming up with things worth saying, being calibrated about being critical AND they are a lot better at just imitating the style.
This just invalidates some side signals humans habitually use on one another.
what about humans who would do the same thing?
Presumably, this happens: https://slatestarcodex.com/2016/12/12/might-people-on-the-internet-sometimes-lie/
I do often notice how the top upvoted reddit comment in big subs is confidently wrong, with a more correct/nuanced take sitting much lower.
This seems quite technologically feasible now, and I expect the outcome would mostly depend on the quality and care that went into the specific implementation. I am even more confident that if the detail of 'the comments of the bot get further tuning via feedback, so that initial flaws get corrected', then the bot would quickly (after a few hundred such feedbacks) get 'good enough' to pass most people's bars for inclusion.
We should have empirical evidence about this, actually, since the LW team has been experimenting with a "virtual comments" feature. @Raemon, the EDT issue aside, were the comments any good if you forgot they're written by an LLM? Can you share a few (preferably a lot) examples?
It's been a long time since I looked at virtual comments, as we never actually merged them in. IIRC, none were great, but sometimes they were interesting (in a kind of "bring your own thinking" kind of way).
They were implemented as a Turing test, where mods would have to guess which was the real comment from a high karma user. If they'd been merged in, it would have been interesting to see the stats on guessability.
I suspect that LLMs likely can write blogs on par with most humans if we trained and scaffolded them appropriately, but is that really what we want from LLMs?
Claude 3.7 might not write outstanding blogs but he can help explain why not:
The fundamental mismatch between LLMs and blogging isn't primarily about capabilities, but about design and motivation:
Current LLMs are RLHF-tuned to be balanced, helpful assistants - essentially the opposite of good bloggers. Assistants hedge, acknowledge all perspectives, and avoid strong stances. Good bloggers take intellectual risks, have distinctive voices, and present unique viewpoints.
Humans blog for reasons LLMs simply don't have:
- Building intellectual reputation in a community
- Working through personal confusions
- Creative self-expression
- The social reward of changing minds
The metrics we use to evaluate LLMs (helpfulness, accuracy, harmlessness) don't capture what makes blogs compelling (novelty, intellectual risk-taking, personality).
Simply making LLMs more capable won't bridge this gap. We'd need systems with fundamentally different optimization targets - ones trained to be interesting rather than helpful, to develop consistent viewpoints rather than being balanced, and to prioritize novel insights over comprehensive coverage.
Surely it would be exceptionally good at those kinds of writing, too, right?
surely an LLM capable of writing A+ freshman college papers would correctly add two 2-digit numbers? surely an AI capable of beating grandmasters in chess would be able to tutor a 1000 elo player to a 1500 elo or beyond? surely an AI capable of answering questions at a university level in diverse subjects such as math, coding, science, law, would be able to recursively improve itself and cause an intelligence explosion? surely such an AI would at least be able to do a simple task like unload a dishwasher without breaking a dish?
i think surely it should be obvious to anyone who's able to mull it through for a few seconds that intelligence does not need to progress along the same paths as it does for human civilization over centuries or for human beings through child development, or even among proto-intelligent animals on earth. it is surely obvious to me that AI can exhibit surprising mixes of general and non-general intelligence, and that we're not really sure why it works. there is really no requirement i have left, that before the AI turns us into paperclips, it must 100% be able to beat poker players at the WSOP or generate an oscar-winning feature film, or be able to make nobel-winning science discoveries. some of these requirements seem more plausible than others, but none seem totally certain.
The quoted sentence is about what people like Dario Amodei, Miles Brundage, and @Daniel Kokotajlo predict that AI will be able to do by the end of the decade.
And although I haven't asked them, I would be pretty surprised if I were wrong here, hence "surely."
In the post, I quoted this bit from Amodei:
It can engage in any actions, communications, or remote operations enabled by this interface, including taking actions on the internet, taking or giving directions to humans, ordering materials, directing experiments, watching videos, making videos, and so on. It does all of these tasks with, again, a skill exceeding that of the most capable humans in the world.
Do you really think that he means "it can do 'any actions, communications, or remote operations enabled by this interface' with a skill exceeding that of the most capable humans in the world – except for writing blog posts or comments"?
Do you think he would endorse this caveat if I were to ask him about it?
If so, why?
Likewise with Brundage, who writes:
AI that exceeds human performance in nearly every cognitive domain is almost certain to be built and deployed in the next few years.
I mean, he did say "nearly every," so there are some "cognitive domains" in which this thing is still not superhuman. But do we really think that Brundage thinks "blogging" is likely to be an exception? Seriously?
(Among other things, note that both of these people are talking about AIs that could automate basically any job doable by a remote worker on a computer. There exist remote jobs which require communication skills + having-interesting-ideas skills such that doing them effectively involves "writing interesting blog posts," just in another venue, e.g. research reports, Slack messages... sometimes these things are even framed as "posts on a company-internal blog" [in my last job I often wrote up my research in posts on a "Confluence blog"].
If you suppose that the AI can do these sorts of jobs, then you either have to infer it's good at blogging too, or you have to invent some very weirdly shaped generalization failure gerrymandered specifically to avoid this otherwise natural conclusion.)
this is a fair response, and to be honest i was skimming your post a bit. i do think my point somewhat holds, that there is no "intelligence skill tree" where you must unlock the level 1 skills before you progress to level 2.
i think a more fair response to your post is:
- companies are trying to make software engineer agents, not bloggers, so the optimization is towards the former.
- making a blog that's actually worth reading is hard. no one reads 99% of blogs.
- i wouldn't act so confident that we aren't surrounded by LLM comments and posts. are you really sure that everything you're reading is from a human? all the random comments and posts you see on social media, do you check every single one of them to gauge if they're human?
- lots of dumb bots can just copy posts and content written by other people and still make an impact. scammers and propagandists can just pay an indian or philipino $2/hr and get pretty good. writing original text is not a bottleneck.
Curated and popular this week