I think you may underestimate the quantitative side of the things. Yes, we all make mistakes sometimes, and by being corrected we learn. But if you get a critical mass of fools, not only most comment will contain mistakes, but even most replies to them will contain mistakes. How are people supposed to learn in an environment where most feedback they get is wrong?
Similarly, we all break social norms sometimes. But that is different from having an environment where the rules are broken so often that breaking the rules becomes the social norm, and following them is the weird exception.
Less Wrong is mostly a community moderating itself. We have a few moderators, and sometimes they take direct action. But most of the moderation is done using the upvotes and downvotes, otherwise the moderators would burn out quickly. The problem with a self-moderating community is that the more it needs moderating, the less it is capable of doing that properly. If the fools get votes, they won't only post foolish content; they will also upvote it.
Quantity has a quality all of its own. I think you're absolutely correct, and you point out a good reason why self-moderation can be insufficient upon reaching this "critical mass". My benefit is that ours is not a forum-based platform but mostly chat, so it's much more likely for at least one moderator to see each message or at least the most obviously wrong ones. Would you say that, as the quantity increases, effective moderation becomes key?
Yeah, my model is that there only so much time a moderator wants to spend moderating, and if the quantity increases, then the time requirements will increase, until it is too much, and the moderator burns out. Even worse, the moment when the moderator gives up does not necessarily have to be the moment when the time requirements exceed some critical threshold; it may also be something unexpected than randomly happens on the moderator's side -- for example, the moderator gets fired from his job, or gets sick, or needs to take care about someone else getting sick; simply, something external will take away the extra attention he was giving to the forum so far.
Ideally, you would have a system where the moderator is not needed for its everyday functionality, only to solve exceptional situations. (By definition, exceptional situations happen rarely.) Then you can have moderators that won't burn out.
A possible alternative is to have multiple moderators, but that opens another can of worms, if the moderators do not agree with each other about some details.
I think that Less Wrong is an example of a well-designed system. It helps that as a community we have the technical expertise to implement some technical solutions (even going to such extremes are rewriting the entire codebase from scratch), and enough money to pay someone to work on that full-time. So I understand that such solution is not available for everyone. (The Less Wrong code is freely available, but you still need some technical expertise to install it.) We have the community that votes, moderators have all kinds of software tools that make their jobs easier, and if a need arises, they can get new tools. That makes it sustainable.
(An example of a bad system that seems good is Reddit. It seems to work okay at first sight, but actually moderators gradually burn out, and there are a few power-hungry people who volunteer to act as a replacement, until you get hundreds of subreddits that are moderated by the same small group of people. No one is aware of this, until something blows up, and suddenly you get the same message censored across hundreds of seemingly unconnected subreddits.)
I tried to moderate a community (other than Less Wrong) once, and I gave up soon, because it was too much work to keep updating and configuring the software to avoid new exploits, while manually banning a few dedicated crazy people who kept manually creating new and new accounts (only to post the same kind of message and get banned instantly -- but that always came a few hours lates, and required an intervention on my end; they kept doing this from different IP addresses).
So this is another problem: the difficulty of moderation does not increase continuously with the number of users or the volume of messages, but rather sooner or later you attract a crazy person with too much free time, and suddenly your have 10x more work overnight. (Which is also what happened to Less Wrong at one moment, and why the codebase had to be rewritten, because the existing tools just were not sufficient to fight that one individual, and it was difficult to implement new tools in the old code.)
I think that in general this is a serious problem that doesn't have a good solution, yet. All communities seem one dedicated attack away from ruin, they are just lucky that the moment didn't come yet. Maybe I am overly pessimistic here. But I would recommend to keep the requirements on the moderators close to zero, so that they can have their hands free for some exceptional action when the critical moment comes.
As a new user, it's hard to know where to start, and how to contribute to a community. being a Good Samaritan by nationality, I was reading through the guides and posts pertaining to the LessWrong community. One article that stood out to me is the "Well-Kept Gardens Die By Pacifism" post. The rhetoric revolves around the fool, and where the fool goes, (intellectual) communities die. It resonated with me. I manage a community that's large on paper, but in practice often devoid of content that excites experts. Indeed, now devoid of the content that attracted (and was made by) the experts that grew the community in the first place, long before I joined.
Is our community dead? Even a community overtaken by fools can still be alive, albeit with little recollection of its former self. If a Well-Kept Garden dies by pacifism, the death of such a Garden is more like a return to nature. Overgrown, covered in weeds, and barren in places where nothing can grow. The community is not dead. There is still movement. Yet returning it to its former glory, or better yet, growing something beautiful in its own right; new and fresh for the modern eye, will take significant efforts. Efforts in pruning and espalier. And in bringing in new, fertile soil.
Before taking on this painstaking responsibility as the self-appointed gardener, I think it wise to ask myself what I can learn from the current state of the community. How did we get here, and what can I do to prevent this slow abandonment by our most valued members. And this is where I set out to question the fool.
Because I don't believe in the fool.
There is no fool
Obviously, online trolling is a real thing. More common than trolling, and likely vastly more deathly owing to its insidious and seemingly tolerable nature (it should not be tolerated) are indolence and deceit. Explicit malice can be counteracted, swiftly and confidently. But incomplete information in question asking and lazy hand-waving in replies is not always so easily spotted. And deceit, deceit prides itself on its stealth. "Perhaps if I add this to the discussion, I'll be thought of as more than I am", deceit thinks.
So how can I say there is no fool. If you're from the community, you'll know I've done more than my fair share of warning, kicking, and banning. Surely, they were fools. And they were, but they were not the fool. They were not the death of the community. They were not those that abused the moderation team's pacifism. They have been dealt with. And as such they have no impact (anymore) on the community.
Fine then, I hear you say, but that's not what Eliezer Yudkowsky was talking about. The fool can be anyone who wanders into a community and through their interactions negatively affects the quality of the discussion. Be it greatly or mildly, any ongoing discussion is irrevocably worse off for their contribution. More egregious still, when two fools find each other, and their potential for harm is maximized.
But there is no fool. The quality of a discussion cannot be measured, only felt. And felt, especially so, by the most invested members. The stronger one's expertise, the easier to notice signs of incompetence. Indeed, in the eyes of the world's leading experts, aren't we all fools? But if you've made a habit of talking to experts, often you'll find that they are humble, especially about the domain they are most knowledgeable on. They know, after all, what it was like to learn for themselves. They are eager to talk about their domain for they became an expert out of passion and love.
In the presence of such experts, one inevitably feels like a bull in a china shop. You know that any contribution you may have, would have been better expressed by the experts. But the community thrives on interaction. Without contributing to the conversation, you cannot err, and, as stated by the first words I read on LessWrong, without erring, you cannot grow. So you add your two cents. Make the conversation slightly worse-off for it. And learn from it.
The worst fool, as stated, is one who is just articulate enough that the experts "feel obliged to respond, and correct misapprehensions". But in many cases, this is indistinguishable from someone who is just coming around, has put in significant effort, but is simply mistaken. And indeed, this nuance is likely why so many "former inhabitants of the garden" feel obliged to respond in the first place.
So if the most damage is done by relatively benign but incompetent (mis)contributions, how can we ever enact an action other than pacifism against the fool? Allowing for no-mistakes, no-fun, on-topic conversation only is not keeping a well-kept garden, it's keeping a walled garden. The experts in my community weren't always experts. They weren't always right either, still aren't, and butted heads not infrequently. As their expertise was not always "communications and public relations", sometimes disagreement was more blunt than necessary. But this is precisely what turned a patch of seedlings and saplings into the well-kept garden that I remember. At a glance, we cannot tell a fool from an eventual expert.
If we want to welcome new members, if we want to wield the secateurs rather than the chainsaw, we have to act as if there is no fool. Purely rhetorical, perhaps the idea of the fool is valid. But we cannot diagnose it, or at least not early when spotting it is most effective. And even if we could, drawing a hard line is almost impossible. What is a conversation-derailing joke in one discussion may spark deeper reflection at the satire in another.
Maybe I'm the fool
And yet I am not writing this article because there is not a very real problem. I like to be very tolerant of people's behavior. As much as I enjoy rare and exotic plants, I admit with some guilt that I am also particular to very simple plots of wildflowers and common herbs. The problem is that some of these are invasive species, or would simply drown out the light and steal the nutrients of other flora if given the chance. The danger is very real. A discussion gets poisoned, established members get fed up and turn silent. Silent members leave, and all that's left is the weeds. Maybe I'm the fool for tolerating these herbs in (part of) my garden.
This article is here because I struggle with a solution. Pacifism implies having the choice of acting or not. If you're not the one wielding the banhammer, you can't tend to the garden. In fact, interacting with the fool will just derail the conversation. In fact, even if there is no fool, and unbeknownst to you it really is just a future expert, current rookie. The conversation can still get derailed. I don't struggle with pacifism. I struggle with not having the option of pacifism, despite wielding the banhammer.
An as-of-yet unmentioned alternative is to simply prevent the issue altogether. If, in many conversations, the fool cannot be distinguished from, forgive the wordplay, the budding expert, we simply have to change the context of the conversation such that they can be distinguished. Weeds, in the right context, are not weeds, but valuable cooking ingredients or beautiful wildflowers. There is no fool, if we can place it in the right context.
Maybe I'm being idealistic. But I think clearly demarcating what is for what, and strictly adhering to it, goes a long way. What is a garden, after all, if not a set of clearly demarcated and structured patches of flora. With sufficient clarity, an act of wandering into a conversation that is out of your depth is either an obvious act of trolling, or a clumsy but endearing learning moment. Either way, there is no longer a fool. In the words of a wise professor, there is a time and place for everything. If we can keep expert collaborative efforts separate from the aspirant's learning and stumbling, and both safe from the jester's wit, I believe each part of the community can bloom to its full potential.
We're all fools
Infractions will be made, conversations interrupted, and current inhabitants disturbed. Both the LessWrong community and my community have been affected by the recent surge in popularity in AI. We have noticed a flood in low-quality posts, both from honest attempts to learn as well as from lazy efforts of what I can only understand as an innate need to talk about "the next hot thing". For some of us, it has affected our jobs. For others, just our community or hobby. Regardless, if you want to grow or protect a community in this environment, we're in the same boat.
Perhaps we're fools for even attempting this, and it can feel like having a full time job in bailing water from a ship that is going under from this deluge. At the same time, fresh water is a necessity for life, and maybe it's an opportunity for growth. If we can tend to our garden now, maybe in a few years it will be bigger and brighter than ever.
Let's inspire confidence in each other and our moderators, not to get rid of the fools, but to be certain about what goes where. Children are fools. Toddlers are worse. All we can do is set out rules that foster growth, and be consequent. That way, every "fool" gets the opportunity to one day be an expert. LessWrong already has many tools implemented for this, such as allowing for self-set moderation guidelines per post. Going forward, I'll definitely be spending effort not just on "hunting down fools", but more so on improving guidelines and making more clearly demarcated spaces.
By consistently creating a space within the space that is protected and aimed only at the highest quality, we can encourage collaborative inquiry there, while leaving a spot for our new and still learning members. Nobody has to be a fool forever.
In short
The death of a community happens through tiny infractions that worsen the ongoing dialogue. These infractions can be especially off-putting to members with high quality contributions, ultimately causing them to leave. I argue that there is not one type of person that does this, and lazy/foolish input usually cannot easily be distinguished from well-intended mistakes or inexperience.
Instead of being on the lookout for fools who would hurt the community, I think (for my community at least) it is more fruitful to spend efforts in demarcating and clarifying the various communal virtual spaces. This allows low-quality content to stand out more in spaces where it does not belong, which enables moderation by making it is easier to draw hard lines. Simultaneously, we can still welcome some level of foolishness in other threads or channels where it is ok to learn.