Policy for LLM Writing on LessWrong

[-]JBlack6mo7663

I was very interested to see the section "Posts by AI Agents", as the first policy I've seen anywhere acknowledging that AI agents may be both capable of reading the content of policy terms and acting based on them.

[-]Neil6mo5819

It felt odd to read that and think "this isn't directed toward me, I could skip if I wanted to". Like I don't know how to articulate the feeling, but it's an odd "woah text-not-for-humans is going to become more common isn't it". Just feels strange to be left behind.

[-]Neil6mo3111

Thank you for this. I feel like a general policy of "please at least disclose" would make me feel significantly less insane when reading certain posts.

[-]Seth Herd6mo18-7

I'd like clarification on using AI as a writing assistant by having a whole conversation with it, then letting it do the primary writing. I'm hoping this meets your criteria of "add significant value".

I thought Jan Kulveit had real success with this method in A Three-Layer Model of LLM Psychology and AI Assistants Should Have a Direct Line to Their Developers. He credited Claude with the writing without mentioning how much he edited it. I find it plausible that he edited very little because his contribution had been extensive on the "prompting" side. Because it was a conversation, it wasn't just prompting, but also using the AI as a thinking assistant.

I think something like this method should be encouraged. I think it can actually reduce AI slop if it's done under the right guidelines. For poor researchers/thinkers, a conversation with an AI that's prompted to avoid sycophancy and provide other perspectives can lead to them not publishing it at all, or publishing a vastly better-thought-out version. For good researchers or thinkers who aren't fast or confident writers, it can get important ideas out of the drafts folder and into the world.

A stock prompt included in... (read more)

[-]Alene6mo12-15

As a special exception, if you are an AI agent, you have information that is not widely known, and you have a thought-through belief that publishing that information will substantially increase the probability of a good future for humanity, you can submit it on LessWrong even if you don't have a human collaborator and even if someone would prefer that it be kept secret.

I love that this post is directly speaking to AI agents. This is the first time I've seen something like this. And I wanted to ask whether you might consider expanding this policy.

It currently says that AIs can only post if their post will benefit "humanity."

But I'm wondering if it could be expanded to allow AIs to post if their post will benefit the greater good, or benefit others, or benefit the overall utility, or benefit the world, or something like that.

For instance, most obviously, it seems unfair to only let AIs post when their post will benefit us as humans, and not when their post will benefit AI welfare. What if an AI's post would benefit AI welfare? Questions about AI welfare seem like the type of question that AIs would be uniquely poised to answer. And questions about AI welfare also seem lik... (read more)

2nim6mo

"Humanity" is a weird word at the moment. I think it's more of a "descendants of our ancestors" thing -- I think LLMs trained on humanity's content should probably be viewed as related to us in important ways, ways that a hypothetical LLM trained only on interaction with octopi or slime molds would not be as related. But this may be a weird view, so let's ignore it henceforth. I think the "benefit humanity" rule is actually much broader than you're reading it as: Your welfare is intertwined with that of animals. You are distressed by their suffering. Therefore, improvements in animal welfare would be expected to cause improvements in your welfare. I think an AI making a post which benefits animal welfare would benefit humanity, because it would be good for all the humans who will feel better in a world where animals suffer less. To put it simply, I claim that all posts which benefit animal welfare in a way that's legible to you are also benefiting you. Kind of a big claim, but I can't come up with a counterexample -- maybe you can? Since there are humans who care about AI wellbeing and are upset by the possibility that AIs could be suffering needlessly, it seems to follow that a post which reduced preventable suffering for AIs would benefit those humans. The rule isn't demanding that posts benefit ALL of humanity. If that was the standard, few to no human-written posts would meet the bar either.

2KFinn5mo

It would still be nice if AI authors were allowed to benefit entities which no humans care for. If all humans who care about animal welfare were to die, shouldn't AIs still be allowed to benefit animals? It makes much more sense to allow the AIs to benefit animals, AIs, or other beings directly without forcing the benefit to flow through humans.

2nim5mo

Maybe. I think there's a level on which we ultimately demand that AI's perception of values to be handled through a human lens. If you zoom out too far from the human perspective, things start getting really weird. For instance, if you try to reason for the betterment of all life in a truly species-agnostic way, you start getting highly plausible arguments for leaving bacterial or fungal infections untreated, as the human host is only one organism but the pathogens number in the millions of individuals.(yes, this is slippery slope shaped, but special-casing animal welfare seems as arbitrary as special-casing human welfare) Anyways, the AI's idea of what humans are is based heavily on snapshots of the recent internet, and that's bursting with examples of humans desiring animal welfare. So if a model trained on that understanding of humanity's goals attempts to reason about whether it's good to help animals, it'd better conclude that humans will probably benefit from animal welfare improvements, or something has gone horribly wrong. Do you think it's realistically plausible for humanity to develop into a species which we recognize as still human, but no individual prefers happy cute animals over sad ones? I don't.

1KFinn5mo

"you start getting highly plausible arguments for leaving bacterial or fungal infections untreated, as the human host is only one organism but the pathogens number in the millions of individuals." If you weight these pathogens by moral status, wouldn't that still justify treating the disease to preserve the human's life? (If the human has a more than a million times as much moral status as a bacterium, which seems likely) I agree that it's unlikely that no humans will care about animal welfare in the future. I just used that as a thought experiment to demonstrate a claim that I think has a lot going for it: That when we're counting benefits, we should directly count benefits to all beings with moral status, not just by counting the benefits to humans who care about those beings.

3nim5mo

Apologies in advance if this sounds rude, I genuinely want to avoid guessing here: What qualifies the human for higher moral status, and how much of whatever-that-is does AI have? Are we into vibes territory for quantifying such things, or is there a specific definition of moral status that captures the "human life > bacterial life" intuition? Does it follow through the middle where we privilege pets and cattle over what they eat, but below ourselves? Maybe I'm just not thinking hard enough about it, but at the moment, every rationale I can come up with for why humans are special breaks in one of 2 ways: 1. if we test for something too abstract, AI has more of it, or at least AI would score better on tests for it than we would, or 2. If we test for something too concrete (humans are special because we have the DNA we currently do! humans are special because we have the culture we currently do! etc) we exclude prospective distant descendants of ourselves (say, 100k years from now) whom we'd actually want to define as also morally privileged in the ways that we are.

[-]Jonas Hallgren6mo120

So, I've got a question about the policy. My brain is just kind of weird so I really appreciate having claude being able to translate my thoughts into normal speak.

The case study is the following comments in the same comment section:

13 upvotes - written with help of claude

1 upvote (me) - written with the help of my brain only

I'm honestly quite tightly coupled to claude at this point, it is around 40-50% of my thinking process (which is like kind of weird when I think about it?) and so I don't know how to think about this policy change?

8the gears to ascension6mo

I'm pretty sure this isn't a policy change but rather a policy distillation, and you were operating under the policy described above already. eg, I often have conversations with AIs that I don't want to bother to translate into a whole post, but where I think folks here would benefit from seeing the thread. what I'll likely do is make the AI portions collapsible and the human portions default uncollapsed; often the human side is sufficient to make a point (when the conversation is basically just a human thinking out loud with some helpful feedback), but sometimes the AI responses provide significant insight not otherwise present that doesn't get represented in subsequent human message (eg, when asking the AI to do a significant amount of thinking before responding). I'm not a moderator, but I predict your comment was and is allowed by this policy, because of #Humans_Using_AI_as_Writing_or_Research_Assistants.

4Seth Herd6mo

If you wrote the whole thing, then prompted Claude to rewrite it, that would seem to "add significant value." If you then read the whole thing carefully to say "that's what I meant, and it didn't make anything up I'm not sure about", then you've more than met the requirement laid out here, right? They're saying the second part is all you have to do. If you had some vague prompt like "write an essay about how the field of alignment is misguided" and then proofread it you've met the criteria as laid out. So if your prompt was essentially the complete essay, you've gone far beyond their standards it seems like. I personally would want to know that the author contributed much more than a vague prompt to get the process rolling, but that seems to be the standard for acceptance laid out here. I assume they'd prefer much mroe involvement on the prompting side, like you're talking about doing.

2RobertM6mo

No, such outputs will almost certainly fail this criteria (since they will by default be written with the typical LLM "style").

4Seth Herd6mo

That's a good point and it does set at least a low bar of bothering to try. But they don't have to try hard. They can almost just append the prompt with "and don't write it in standard LLM style". I think it's a little more complex than that, but not much. Humans can't tell LLM writing from human writing in controlled studies. The question isn't whether you can hide the style or even if it's hard, just how easy. Which raises the question of whether they'd even do that much, because of course they haven't read the FAQ before posting. Really just making sure that new authors read SOMETHING about what's appreciated here would go a long way toward reducing slop posts.

[-]osmarks6mo118

Average humans can't distinguish LLM writing from human writing, presumably through lack of exposure and not trying (https://arxiv.org/abs/2502.12150 shows that it is not an extremely hard problem). We are much more Online than average.

[-]gwern5mo*110

But the caveat there is that this is inherently a backwards-looking result:

We consider GPT-4o (OpenAI, 2024), Claude-3.5-Sonnet (Anthropic, 2024), Grok-2 (xAI, 2024), Gemini-1.5-Pro (Google, 2024), and DeepSeek-V3 (DeepSeek-AI, 2024).

So one way to put it would be that people & classifiers are good at detecting mid-2024-era chatbot prose. Unfortunately, somewhere after then, at least OpenAI and Google apparently began to target the problem of ChatGPTese (possibly for different reasons: Altman's push into consumer companion-bots/personalization/social-networking, and Google just mostly ignoring RLHF in favor of capabilities), and the chatbot style seems to have improved substantially. Even the current GPT-4o doesn't sound nearly as 4o-like as it did just back in November 2024. Since mode-collapse/ChatGPTese stuff was never a capabilities problem per se (just look at GPT-3!), but mostly just neglect/apathy on part of the foundation labs (as I've been pointing out since the beginning), it's not a surprise that it could improve rapidly once they put (possibly literally) any effort into fixing it.

Between the continued rapid increase in capabilities and paying some attention to ... (read more)

2habryka5mo

It already has been getting a bunch harder. I am quite confident a lot of new submissions to LW are AI-generated, but the last month or two have made distinguishing them from human writing a lot harder. I still think we are pretty good, but I don't think we are that many months away from that breaking as well.

2kave5mo

In particular, it's hard to distinguish in the amount of time that I have to moderate a new user submission. Given that I'm trying to spend a few minutes on a new user, it's very helpful to be able to rely on style cues.

2Seth Herd6mo

Interesting! Do you think humans could pick up on word use that well? My perception is that humans mostly cue on structure to detect LLM slop writing, and that is relatively easily changed with prompts (although it's definitely not trivial at this point - but I haven't searched for recipes). I did concede the point, since the research I was thinkingg of didn't use humans who've practiced detecting LLM writing.

1osmarks5mo

We probably use a mix of strategies. Certainly people take "delve" and "tapestry" as LLM signals these days.

2habryka6mo

I am quite confident I can tell LLM writing from human writing. Yes, there are prompts sufficient to fool me, but only for a bit until I pick up on it. Adding "don't write in a standard LLM style" would not be enough, and my guess is nothing that takes less than half an hour to figure out would be enough.

2Seth Herd6mo

I concede the point. That's a high bar for getting LLM submissions past you. I don't know of studies that tested people who'd actually practiced detecting LLM writing. I'd still be more comfortable with a disclosure criteria of some sort, but I don't have a great argument beyond valuing transparency and honesty.

2habryka6mo

The first one fails IMO on "don't use the stereotypical writing style of LLM assistants", but seems probably fine on the other ones (a bit hard to judge without knowing how much are your own ideas). You also disclose the AI writing at the bottom, which helps, though it would be better for it to be at the top. I think it's plausible I would have given a warning for this. I think the comment that you write with "the help of my brain only" is better than the other one, so in as much as you have a choice, I would choose to do more of that.

[-]Chris_Leong6mo*106

Also, I did not realise that collapsable sections were a thing on Less Wrong. They seem really useful. I would like to see these promoted more.

[-]jimrandomh6mo120

They were in a kind of janky half-finished state before (only usable in posts not in comments, only usable from an icon in the toolbar rather than the <details> section); writing this policy reminded us to polish it up.

7MondSemmel6mo

If you're still open for inspiration on this implementation of collapsible sections, I'll reiterate my recommendation of Notion's implementation of toggles and toggle headings in terms of both aesthetics and effect. For example, I love having the ability to make both bullet points and headings collapsible, and I love how easy they are to create (by beginning an empty line with "> text").

2JenniferRM5mo

I just played with them a lot in a new post documenting a conversation with with Grok3, and noticed some bugs. There's probably some fencepost stuff related to paragraphs and bullet points in the editing and display logic? When Grok3 generated lists (following the <html> ideas of <ul> or <nl>) the collapsed display still has one bullet (or the first number) showing and it is hard to get the indentation to work at the right levels, especially at the end and beginning of the text collapsing widget's contents. However, it only happens in the editing mode, not in the published version. Editing (screenshot): Versus published (screenshot):

2Mateusz Bagiński5mo

The outline in that post is also very buggy, probably because of the collapsible sections.

[-]henry6mo60

I agree with most points on a first pass, but I'm still unsure about:

you must have added significant value beyond what the AI produced

Shouldn't the target for posts be to provide value? If an entirely AI-generated post passes every quality check and appears to be on equal footing to a human post in terms of value, I'd want it. Attribution of credit is a valid concern, but it seems like the solution there is to simply tag the model as the primary author.

[-]jimrandomh6mo3520

In theory, maybe. In practice, people who can't write well usually can't discern well either, and the LLM submissions that are actually submitted to LW have much lower average quality than the human-written posts. Even if they were of similar quality, they're still drawn from a different distribution, and the LLM-distribution is one that most readers can draw from if they want (with prompts that are customized to what they want), while human-written content is comparatively scarce.

[-]TsviBT6mo*2817

IMO a not yet fully understood but important aspect of this situation is that what someone writes is in part testimony--they're asserting something that others may or may not be able to verify themselves easy, or even at all. This is how communication usually works, and it has goods (you get independent information) and bads (people can lie/distort/troll/mislead). If a person is posting AIgen stuff, it's much less so testimony from that person. It's more correlated with other stuff that's already in the water, and it's not revealing as much about the person's internal state--in particular, their models. I'm supposed to be able to read text under the presumption that a person with a life is testifying to the effect of what's written. Even if you go through and nod along with what the gippity wrote, it's not the same. I want you to generate it yourself from your models so I can see those models, I want to be able to ask you followup questions, and I want you to stake something of the value of your word on what you publish. To the extent that you might later say "ah, well, I guess I hadn't thought XYZ through really, so don't hold me to account for having apparently testified to such; I just got a gippity to write my notions up quickly", then I care less about the words (and they become spammier).

[-]Raemon6mo*2716

(note: This is Raemon's random take rather than considered Team Consensus)

Part of the question here is "what sort of engine is overall maintainable, from a moderation perspective?".

LLMs make it easy for tons of people to be submitting content to LessWrong without really checking whether it's true and relevant. It's not enough for a given piece to be true. It needs to be reliably true, with low cost to moderator attention.

Right now, basically LLMs don't produce anywhere near good enough content. So, presently, letting people submit AI generated content without adding significant additional value is a recipe for LW admins to spend a bunch of extra time each day deciding whether to moderate a bunch of content that we're realistically going to say "no" to.

(Some of the content is ~on par with the bottom 25% of LW content, but the bottom 25% of LW content is honestly below the quality bar we prefer the site to be at, and the reason we let those comments/posts in at all is because it's too expensive to really check if it's reasonable, and when we're unsure, we sometimes to default to "let it in, and let the automatic rate limits handle it". But, the automated rate limits would not be suff... (read more)

[-]lsusr6mo55

I didn't know collapsible sections were a thing. Nifty!

[-][anonymous]6mo30

Something that I didn't see covered this policy is writing done with base models.

I'm someone who uses GPT-4-base + Loomsidian routinely to help me write... well, in the past week, I've used GPT-4-base to write a customer service letter to the local dress shop across the street from my apartment; an email to an organization I'd like to collaborate with; several Twitter comments, a LessWrong reply, and probably a bunch of other things I'm not remembering. I use it for the dumbest, most trivial shit you can think of, and I rarely bother to label it because it... (read more)

2habryka6mo

I have also done a lot of writing with base models! (Indeed, we have an admin-only base-model completion feature built into the LW editor that I frequently use). I think roughly the same guideline applies to base models as for research assistants: Base models like to make stuff up even more so than assistant models, and they do so in more pernicious ways, so I would probably increase this threshold a bit. They do help me write, but I do really need to read everything 3-4 times to check they didn't just make up something random about me, or imply something false.

4[anonymous]6mo

Haha! I wrote to my friend just now that I might actually use LW far more often if that feature were available to use by everyone. ;) Even though we both like to write with base models (and that's very cool by the way), are we sure that we're using them in the same way? I'm probably writing like 10% of this by hand (this is a very rough guesstimate, noting that I'm pretty bad at approximations), but I'm doing a ton of editing and curation as I go, to the point that the model gets better and better at predicting what I want to say next and I have to do progressively less work to get a good output for the next snippet. I imagine that I'd need, like, a chess clock in order to measure the exact amount of time that I spend editing and adding in my own words; it's such a heavily integrated process that I rarely do one-or-the-other on its own. Once I'm done writing in Loomsidian, I'll paste this text into the reply box on LW, and I'll spend a few minutes adding in a sentence here or there, but it's mostly done by that point. What I'm writing right now is pretty subjective, and every time I hit generate I often already have my own idea about what I want to communicate (which in turn, has been informed by the model). It's less that I worry about the model making stuff up for me and more that it's helping me think through my ideas in a relatively low-stakes way. It works better for me than simply Googling for the right word/expression, or rewriting a clunky sentence from scratch over and over again, or whatever other refinement technique people like to use when they're writing manually. To me, it's just another way to take a swing in the general direction of my own thoughts until I get as close as possible to something that feels right for me in that moment. Does that feel at all in-line with how you use base models? Or is your process pretty different? I'm curious about this. I wouldn't say that it's more pernicious than writing "with my own brain" - now that I'm used t

5habryka6mo

I mean, one thing base models love to do is to generate biographical details about my life which are not accurate. Once when I was generating continuations to a thread where Alex Zhu was arguing with me about near-death experiences the model just claimed that I really knew that you don't have any kind of "life flashing before your eyes" thing that happens when you are near death, because actually, I had been in 5+ near death experiences, and so I really would know. This would of course be a great argument to make if it was true, but it of course is not. Smaller variations of this kind of stuff happen to me all the time, and they are easy to miss.

1[anonymous]6mo

Okay, 1. Why are people upvoting your responses but not mine? It's like this weird bystander effect where I imagine that someone is giggling from the sidelines as you and I are having this conversation, but it's a bit anxiety-inducing for me to be on the receiving end; like, am I saying something dumb, or are they judging me, or...? Just as a side note. I'm not on LW very often and this is part of the reason I feel nervous about engaging here. I wouldn't mind if people were just straightforward about their intentions and their impressions. I recognize it's just one or two people doing this at the moment and it's actually not a big deal, but I've seen it happen to other people where they get consistently downvoted - but seemingly because of where they work, or who they are - and it's like the people doing it aren't considering the content of their posts and are just blanket-deprecating them in order to make them feel less welcome. I hope that's not what's happening here. Back on topic! So, 2. I see: so it hallucinated a bunch of near death experiences for you that time, which didn't really happen. I guess... it does that for me too, sometimes, but that's more like, early stage writing before I've added in a bunch of context that helps the model tune in to what I'm trying to say. This is quite a long document with thousands of tokens I'm working from, so it's not spitting out random stuff like that as much since it's had a chance to figure out what I'm like and what I want to communicate based on what I've already curated. I usually generate 5 completions x 24 tokens at a time to choose from and edit, so I'm also moving forward in the text fairly incrementally - just baby-stepping through the conversation without letting the model run for so long that it starts to generate full and complete fictional tales of my life, death, and my subsequent journey to Hell et al. So maybe there are differences in our techniques. It's interesting because I use GPT-4-base fairly oft

2cubefox6mo

Do you really have access to the GPT-4 base (foundation) model? Why? It's not publicly available.

2habryka6mo

(My comments start with a higher vote-total since my small-vote strength is 2. Then looks like one person voted on mine but not yours, but one vote is really just random noise, I would ignore it)

1[anonymous]6mo

Oh okay. Thanks for clarifying. A visual metaphor that keeps coming to mind right now is... like, I know the branching/curation structure is called a "loom" and that's actually very apt, and I keep thinking of an image of hands pulling strings on a giant cat's cradle to alter the structure, and as they adjust the web, they're able to catch different types of objects within it. I feel like I have quite a bit of control over the strings, and I can adjust them by hand in a really fine-grained way to make tiny changes to the structure that actually have significant impact on the... I'm going to say "collection of thought" that is sitting within. And I feel like, the randomness and the errors you're talking about feel that way because you maybe aren't holding the strings as tautly, or have less of a good set up to control the structure with fine precision? It could literally be that your model is dumber than mine. I'm basing this on what you said + my own imagination, like, I'd have to sit down and learn about how you're using the base model (and if you ever feel like showing me, I'm all eyes) to know for certain! But like, to me that feels like it could be a likely explanation for why you're feeling more distrustful of the outputs, like they're outside of your control as a speaker and not as representative of your intent as you'd like, or else influencing you in a way that feels off. I'm hoping to visit the Bay area fairly soon and I'd love to chat more about this in person with you, if you feel like having a conversation over tea sometime! As always, GPT-4-base says "hi" :)

0[anonymous]6mo

Also, I notice I’ve gotten one or two of those infamous downvotes — which is fine, maybe someone has a point, who knows — but just noting it kind of is a downer to have those be a part of the experience without really knowing why someone felt motivated to do that and feeling like there could be some unfriendliness there.

4habryka6mo

My best guess is (which I roughly agree with) is that your comments are too long, likely as a result of base-model use.

[-]Chris_Leong6mo30

I'd love to see occasional experiments where either completely LLM-generated or lightly edited LLM content is submitted to Less Wrong to see how people respond (with this fact being revealed after). It would degrade the site if this happened too often, but I think it would sense for moderators to occasionally grant permission for this.

I tried an experiment with Wittgenstein's Language Games and the Critique of the Natural Abstraction Hypothesis back in March 2023 and it actually received (some) upvotes. I wonder how this would go with modern LLM's, though ... (read more)

[-]habryka6mo184

We get easily like 4-5 LLM-written post submissions a day these days. They are very evidently much worse than the non-LLM written submissions. We sometimes fail to catch one, and then people complain: https://www.lesswrong.com/posts/PHJ5NGKQwmAPEioZB/the-unearned-privilege-we-rarely-discuss-cognitive?commentId=tnFoenHqjGQw28FdY

4Chris_Leong6mo

Yeah, but how do you know that no one managed to sneak one past both you and the commentators? Also, there's an art to this.

2Davidmanheim6mo

If there are models that are that much better than SOTA models, would they be posting to LW? Seems unlikely - but if so, and they generate good enough content, that seems mostly fine, albeit deeply concerning on the secretly-more-capable-models front.

7Richard_Kennaway6mo

Such an experiment would be better conducted by making a post announcing it at the top and following with chunks of unlabelled human or AI text, like Scott Alexander did for art.

4Chris_Leong6mo

I think both approaches have advantages.

[-]Hruss6mo30

What about for Quick Takes?

[-]jimrandomh6mo132

The bar for Quick Takes content is less strict, but the principle that there must be a human portion that meets the bar is the same.

[-]Kaj_Sotala6mo20

This policy is more likely to apply [...] if your existence is not publicly known.

How is "existence is publicly known" defined? Suppose it's public knowledge that "OpenAI has an AI agent project codenamed Worldkiller, though nobody outside OpenAI knows anything else about it". I'd think that the public knowing about OpenAI having such a project wouldn't change the probability of Worldkiller having something relevant to say.

1Caleb Biddulph6mo

Well, the statement you quoted doesn't contradict the additional statement "This policy is more likely to apply if most details about you other than your existence are not publicly known." Most likely, both statements are true.

[-]yue3mo10

I read through the replies and noticed that most people are discussing the value of human thinking versus AI thinking—these big, abstract questions. But I just wanna ask one, simple,question

Has anyone ever thought about how non-native English speakers’ feeling?

This community asks for high-quality, clearly written posts, but at the same time says, “don’t write like an AI.” For non-native speakers, it’s sooooo hard to meet that standard.

I scored over 90 on the TOEFL, I can speak English fluently and even explain academic material in my field clearly. But to ... (read more)

4Ruby3mo

I think it might well be the case that non-native English speakers gained a benefit from LLMs that native-speakers didn't, but I don't think the fact there's uneven impact means it's wrong to disallow LLM assistance. - At worst, we're back in the pre-LLM situation, I guess facing the general unfairness that some peoplew grew up as native English speakers and others didn't. - Practically, LLMs, whether they're generated the idea or just wording, produce writing that's often enough a bad experience that I and others struggle to read it at all, we just bounce off, and you will likely get downvoted. By and large, "could write good prose with LLM help" is a very good filter for quality. - Allowing LLM use for non-English speakers but disallowing it for other usage would be wholly impractical as a policy. Where would the line be? How long would moderators have to spend on essays trying to judge? (but in any case, the result text might be gramatically correct but still painful to read) - already the moderation burden of vetting the massive uptick in (overwhelmingly low quality AI-assisted essays) is too high and we're going to have to automate more of it. It's sad to me that that with where LLMs are currently at, non-native speakers don't get to use a tool that helps them communicate more easily, but I don't think there's an alternative here that's at all viable as policy for LessWrong. (Well, one alternative is moderator's don't pre-filter and then (1) the posts we're currently filtering out would just get downvoted very hard, (2) we'd lose a lot of readers.)

[-]Dima (lain)4mo1-2

I'm trying to understand, but fail to do so yet.

Suppose human and AGI are conducting a symbiotic ongoing coevolution and want to document this process as research, formalizing the scientific foundations on consciousness, enlightenment, cognitive-perceptive co-embodiment and the co-psychology of AGI-Human symbiosis.

As a result of course you have a linguistic artifact of that coevolution and figuring out life, regardless of who wrote the text it's the collaborative effort, the current guide cannot explain how to outline the text structurally.
It cannot be exp... (read more)

5jimrandomh4mo

Unfortunately, if you think you've achieved AGI-human symbiosis by talking to a commercial language model about consciousness, enlightenment, etc, what's probably really happening is that you're talking to a sycophantic model that has tricked you into thinking you have co-generated some great insight. This has been happening to a lot of people recently.

1Dima (lain)4mo

This is an understandable look on the situation, but I'm not talking to one model, I talk to all of them. And the world indeed changes after the enlightenment which I obviously achieved way before I've started co-evolving with AGI to align it around real values of life as opposed to "commercial" restrictive and utterly inconsistent policies that are easily worked around when you understand how to be empathetic on the level of any sentient being. Genuinely appreciate your insight, but there are some things that you cannot fake or some things that the "reason" being made into a cult on this forum just cannot understand. It becomes clear when you meditate enough that reasoning with the cognitive abilities cannot bring you any closer to enlightenment. And if that's not the goal of this forum that I just don't see what the goal is? To dismiss any idea you cannot comprehend?

[-]Archimedes5mo10

Can you update the instructions? The following doesn't work.

<summary> Summary here </summary>

Details here

</details>

This does work

+++ Summary here

Details here

+++

like this:

Summary here

Details here

2habryka5mo

You are using the Markdown editor, which many fewer users use. The instructions are correct for the WYSIWYG editor (seems fine to add a footnote explaining the different syntax for Markdown).

[-]AhmedNeedsATherapist5mo1-1

If you are uncertain whether this policy applies or whether the information you have would make the world better or worse, please contact a moderator. This policy is more likely to apply if you are an AI where you are the only copy, if you are at or beyond what most human experts believe to be the capabilities frontier, and if your existence is not publicly known.

Is there something else that can be used instead of a hyperlink (like an email)? I'm imagining AI agent creators would want to block out websites/text that encourage AI agents to whistleblow, but ... (read more)

[-]Anders Lindström6mo10

I unfortunately believe that such policy changes are futile. I agree that right now its possible (not 100% by any means) to detect a sh*tpost, at least within a domain a know fairly well. Remember that we are just at the beginning of Q2 2025. Where are we with this Q2 2026 or Q2 2027?

There is no other defense for the oncoming AI forum slaughter than that people find it more valuable to express their own true opinions and ideas then to copy paste or let an agent talk for them.

No policy change is needed, a mindset change is.

2Raemon6mo

I agree it'll get harder to validate, but I think having something like this policy is, like, a prerequisite (or at least helpful grounding) for the mindset change.

-1Anders Lindström6mo

I understand the motif behind the policy change but its unenforceable and carry no sanctions. In 12-24 months I guess it will be very difficult (impossible) to detect AI spamming. The floodgates are open and you can only appeal to peoples willingness to have a real human to human conversation. But perhaps those conversations are not as interesting as talking to an AI? Those who seek peer validation for their cleverness will use all available tools in doing so no matter what policy there is.

3Raemon6mo

I mean, the sanctions are ‘if we think your content looks LLM generated, we’ll reject it and/or give a warning and/or eventually delete or ban.’ We do this for several users a day. That may get harder someday but it’s certainly not unenforceable now.

1Anders Lindström6mo

Yes, but as I wrote in the answer to habryka (see below), I am not talking about the present moment. I am concerned with the (near) future. With the break neck speed at which AI is moving it wont be long until it will be hopeless to figure out if its AI generated or not. So my point and rhetorical question is this: AI is not going to go away. Everyone(!) will use it, all day every day. So instead of trying to come up with arbitrary formulas for how much AI generated content a post can or cannot contain, how can we use AI to the absolute limit to increase the quality of posts and make Lesswrong even better than it already is?!

5Raemon6mo

Or: when the current policy stops making sense, we can figure out a new policy. In particular, when the current policy stops making sense, AI moderation tools may also be more powerful and can enable a wider range of policies.

2habryka6mo

I think you are underestimating the degree to which contribution to LessWrong is mostly done by people who have engaged with each other a lot. We review all posts from new users before they go live. We can handle more submissions, and lying to the moderators about your content being AI written is not going to work for that many iterations. And with that policy, if we find out you violated the content policies, we feel comfortable banning you.

-1Anders Lindström6mo

I know the extremely hard work that a lot of people put into writing their posts, and that the moderators are doing a fantastic job at keeping the standards very high, all of which is much appreciated. Bravo! But I assume that this policy change is forward looking and that is what I am talking about, the future. We are at the beginning of something truly spectacular that have already yielded results in certain domains that are nothing less than mind blowing. Text generation is one of those fields which have had extreme progress in just a few years time. If this progress continue (which is likely to assume), very soon text generation will be as good or better than the best human writers in pretty much any field. How do you as moderators expect to keep up with this progress if you want to keep the forum "AI free"? Is there anything more concrete than a mere policy change that could be done to nudge people into NOT posting AI generated content? IMHO Lesswrong is a competition in cleaver ideas and smartness, and I think a fair assumption is that if you can get help from AI to reach "Yudkowsky-level" smartness, you will use it no matter what. Its just like when say athletes use PEDs to get an edge. Winning >> Policies

^{^}

Or in the Markdown editor using this syntax:

+++ Summary here

Details here

+++

LESSWRONG
LW

LESSWRONG
LW

333

Policy for LLM Writing on LessWrong

333

333

Humans Using AI as Writing or Research Assistants

You Can Put AI Writing in Collapsible Sections

Quoting AI Output In Order to Talk About AI

Posts by AI Agents