So, here is the question: why didn't we apply the don't feed the troll meme, as would probably happen much sooner on most forums?
To your question, my own question: Why would we apply the "don't feed the troll" meme? There is no reason to believe it works. There's no academic study of least trollish places on the web, nor are there reams of data with which we can analyze best responses. Interestingly, the most trollish places on the internet tend to rely entirely on the "don't feed the troll" meme. While the least visibly trollish places on the web all use distributed moderation systems along with an active defense by all the posters there to combat trolling.
Using 4chan's method of ignoring trolls when we could be following Stackoverflow's method of obliterating trolls is insane. It's like asking for advice on passing our driving test from the girl who failed the test 12 times. Sure she's got more experience than anyone, but that's only because she so bad at it. We have no reason to believe that "don't feed the trolls" works. Going further, I'd say we have good evidence to believe it doesn't work. We should first determine what deters and eliminates trolls best, instead of aspiring to an untested ideal.
"Don't feed the trolls" is just a meme that may or may not work. Nothing more and nothing less.
I agree that 'don't feed the trolls' is an untested folk theory that we shouldn't necessarily assume works, but your argument seems to assume that the expected result of not feeding trolls is no trolls at all, which I'm fairly sure is a strawman. The point of not feeding the trolls is, as far as I can tell, to minimize how disruptive they are when they appear. It's more analogous to "if you cut yourself, apply first aid and consider seeing a doctor" than "if you want to avoid scurvy, make sure to get some vitamin C in your diet".
The problem on most forums is that people say "don't feed" while continue feeding (not necessarily the same people are engaged in both parts). I believe that the not feeding policy works because (1) I don't remember seeing a really obnoxious trollish exchange not feeded by non-trolls (itself a weak argument, since I don't frequent troll habitats often), (2) it corresponds to my proto-model of troll motivation, which is seeking attention (also a weak argument, I don't really understand trolls) and (3) the trolls need something to react to, and the responses to their debate contributions provide more material and thus opportunities (this is a bit stronger argument, it seems almost self-evident).
The continued simultaneous existence of trolls and the no-feeding policy doesn't say much and is well compatible with the policy being effective. Note that:
The poster in question at least wouldn't be immediately obvious as a troll to outsiders reading only a small part of the discussion, more proactive ways to deal with posters like that would seem to carry a serious risk of making less wrong appear even more cultish (group-think).
For whatever measures we take we should first consider how much ammunition against less wrong they offer, how likely they are to cost us genuinely valuable contributions due to seeming closed to dissent, and whether the expected magnitude of intended effects is worth that.
That includes using the word "troll". After the various facts about their behavior and motivations there is no additional fact as to whether they are a troll. Using the word troll might easily lead people who only took a quick look to come away with the impression that we generally dismiss non-bayesians as trolls, whereas talking about how to prevent endless discussions not aimed at resolving disagreements seems less dangerous that way.
That includes using the word "troll". After the various facts about their behavior and motivations there is no additional fact as to whether they are a troll. Using the word troll might easily lead people who only took a quick look to come away with the impression that we generally dismiss non-bayesians as trolls, whereas talking about how to prevent endless discussions not aimed at resolving disagreements seems less dangerous that way.
Indeed. I'm not even sure the user in question was a troll by intention (even if they were one functionally) - being persistent and dense beyond reason is a highly plausible trait of participants in Internet philosophy discussions, after all, particularly when the participant has their very own site all about what they're talking about.
That is, the label "troll" assigns intent in a way that is not actually all that relevant to the problem, which is the behaviour, when you can accurately describe the problematic behaviour.
"I prefer trolls to cranks, because trolls sometimes rest." - Alexandre Dumas (fils) (loose translation)
I don't think the "trolls" started that way - in fact claiming to be trolling (or that they don't care, etc) is a common defensive response, especially on the internet, after which appearances and cognitive dissonance do lead to some actual trolling.
They were just people who disagreed with with the majority, with the unfortunate need to frame everything in terms of their favorite topic. Sure their arguments were well below normal LW standards, but we have high standards. They were normal, pre-rationality-training arguers, and not talking to normal people doesn't appeal to me at all.
Your first suggestion, "we are unable to recognize trolls", is a valid one. I personally didn't think of the downvoted user as a troll, just as someone who supported an incorrect epistemology and had a couple of confused ideas. It was only after said user failed to learn from several clear, well-written replies that it became clear that ver presence on LW was a bad thing.
A really good troll would be able to maintain a high karma level even while wasting Users' time on trivial matters -- and this forum has been able to fend off even that kind, so I would not regard it as worrisome that an occasional less-obvious troll like User:[withheld] succeeds.
How about 6.: arguing for the lurkers' benefit? That argument was voluminous, repetitious, and random sampling of the low-voted comments turned out to only be useful for verifying that their scores weren't capricious, but a few of the upvoted comments were worth my time to find and read; if a few dozen other people felt the same then they might have also been worth the authors' time to write.
For what it's worth -- I had one exchange with the person in question pretty early on, decided based on their response that the conversation wasn't going to go anywhere useful, and dropped it.
But I mostly refrained from downvoting them initially because there were people I respected who were continuing the discussion with every indication that it was being productive, and I value productive discussion even if I'm not getting anything out of it personally. (There are a lot of exchanges on this site that kinda sound like gibberish to me, and in at least some cases I'm fairly confident that this is because I don't understand the issue well enough to participate usefully in the discussion even as an observer, not because the participants are in fact spouting gibberish.)
After a very short while I stopped reading any of the comments on that thread except to sample them every once in a while to see if they were going anywhere, and it seemed pretty clear that they weren't. At that point I started downvoting everyone involved on the grounds that I want less high-volume discussion that makes no progress.
No general lesson here, just another data point.
As someone who was probably doing more feeding than most, I'd like to apologise here.
In my case it was primarily the 'didn't recognise troll' problem, I'm not very good at distinguishing the more eloquent and seemingly reasonable type of troll from honest commenters who disagree with me. I also have a strong aversion to just walking out on a debate without reaching any kind of agreement, mainly because it annoys me a lot when other people do it.
I will try not to fall into a similar pattern again, but since I'm not all that good at noticing it any 'don't feed the troll' PMs are appreciated.
I'm in a similar situation - I can, technically, recognize that someone fits the profile of 'troll', but my brain doesn't like to actually use that information for anything. (It's not just an issue with trolls, it's a general inability to use different scripts for people based on categorizing them.) What I've found works, in my case, is to be more aware of subtleties in peoples' behavior, rather than trying to categorize them. I still wind up feeding trolls sometimes (and in fact tend to enjoy doing so in those cases), but if someone is being logically rude or otherwise offensive, that's a thing worth noticing and reacting to whether they're 'a troll' or not.
Another trick that's useful sometimes is backing out of the argument a bit and looking at the bigger picture. Sometimes that shows patterns that aren't otherwise obvious, and that can make it clear that the person isn't worth continuing to deal with. For example, in the most recent case, the individual was defying the evidence in an irrational and not very obvious way. Once I noticed that, it seemed obvious to me that without a more thorough understanding of when that is and isn't a reasonable thing to do, they weren't going to stop doing it to any evidence that we gave them, and thus the argument at hand was not going to resolve anything, so it was pointless. It's a lot easier to walk away in a situation like that, when you can see that it's impossible to actually get the goal you were aiming for.
1. We are unable to recognise trolls for lack of training.
2. We are unwilling to deal with trolls.
3. Sunken costs.
4. Best rebuttal contest.
5. Trollish arguments are fun.
It's good to recognize how tempting these are. In fact, my impression was that the exact same mental bugs were motivating the "troll" in this case to continue with the conversation. As far as motivation for participating in that specific conversation was concerned, the only difference between the "troll" and his interlocutors was that the interlocutors knew for sure that they had a large approving audience.
There's another issue here- a lot of my comments in reply (almost all of them) were upvoted. Some were upvoted quite high. I interpreted this as a sign that people were interested in the subject when it seems in retrospect I should have interpreted those upvotes more as "agree" or "well-argued" and not "want more". Being more clear about what people mean when they upvote might help.
Upvoted for admirable restraint in not linking or naming.
Several times the troll mentioned he was forced to slow down posting by the site. Was this because of low karma? If so, can we just penalise people more for massively negative karma?
Replying to people who are wrong or systematically wrong is not a problem, so long as we keep the good Less Wrong tradition of addressing the statements with excruciating seriousness, like this.
The problem appears when people who systematically produce wrong or low-quality content don't slow down in modes in which they get downvoted. Global Karma level and 10 minute delay don't address this problem directly.
Perhaps downvotes should act as a cooler measure, temporary ban points, for example:
That system would also ban people for some time after they set up a poll. Perhaps it would be better to let K bet the total Karma of all comments, not just negative Karma comments, so good comments could offset bad ones.
... all symptoms of trolling were present, and still, people were repeatedly patiently explaining what's wrong with the author's position. Which reaction is, I must admit, sort of amazing ...
Amazing only if you assume that educating that trollish author is the purpose of the response. Frequently, though, one responds imagining an audience much larger than a single troll. Sometimes one writes experimentally, for oneself, seeking feedback from the community as to whether one's own viewpoint finds resonance with other people.
...it is hard to deny that the
I'm unfamiliar with this particular instance, but I've engaged trolls on Fark before, for these reasons:
1) Some trolls just give up when taken calmly seriously at face value, as opposed to getting hit with indignation.
2) Bored at work.
3) They provide such wonderful strawmen against which to clarify my own thoughts and sharpen my arguments.
4) In some discussions (political ones especially), Poe's Law applies, oftentimes real people hold real opinions that are as blunt as trolls' opinions.
Responding to curi was good training. I found myself discovering, for myself, what levels parts of my understanding were at - automatically seeing exactly where he went wrong, or being able to derive from my web of beliefs a principle that was violated... and I could go on.
Point is, I fed the trolls because it was good exercise.
There are some more constructive reasons for arguing with the incorrigible. One is to persuade the audience, rather than your interlocutor -- anything you say on a public stage is addressed as much to the general readership as to the individual being explicitly addressed. Another is to practise your skill at arguing the material.
Not that these justify going on and on indefinitely, but they are worth putting in the balance.
- Trollish arguments are fun.
As something of a troll, depending on what is meant by that at least, it is often quite interesting to debate with people that do not hold your point of view and are capable of making decent arguments. Too often one runs into arguments where both sides don't know what they are talking about in the slightest, which is terribly frustrating.
As for the Karma, I have had for a while enough karma to make a top level post, getting it isn't particularly hard. I would have much higher karma if I didn't occasionally post controversi...
Good post, thanks.
I note you don't list as a hypothesis "even though the interlocutor was a troll, their arguments had enough merit to be worth the effort of rebutting". I didn't read most of the threads; I take it if I had, I'd know that wasn't a good hypothesis? Or does this come under the "Best rebuttal contest" heading?
Under the usual rules, trolls are to be treated like zombies: they emit messages, but their words don't reflect what they actually think, but a sort of fake-thinking designed to deceive you. Or they are outlaws: responding in good faith towards them is considered bad behavior, under the "don't feed the troll" principle.
"We are actors; we are the opposite of people. So? We need an audience." — Rosencrantz and Guildenstern are Dead
If what you care about is arriving at accurate beliefs about the world under discussion rather than about the...
We could hide downvoted posts from "recent posts" (make them available only to those who already know the url) and hide downvoted comments from "recent comments". And hide all children of downvoted posts/comments from "recent comments" too. That would discourage feeding.
So, here is the question: why didn't we apply the don't feed the troll meme, as would probably happen much sooner on most forums? I have several hypotheses on that.
There have been a few comments of that type, and I've sent private messages to some participants saying just that. Public "don't feed the troll" can be an encouragement.
Part of this is probably due to the feeling that "we can't let this outrageous statement stand". We feel the need to respond to a bad argument, even if there is no particular benefit to doing so.
If you knew in advance that you were wrong and your comment was meaningless, why did you make it?
We could hide downvoted posts from "recent posts" (make them available only to those who already know the url) and hide downvoted comments from "recent comments". And hide all children of downvoted posts/comments from "recent comments" too. That would discourage feeding.
Hiding downvoted posts is already available as an individual preference, which lots of people have turned off. Do not want.
One of the best achievements of the LessWrong community is our high standard of discussion. More than anywhere else, people here are actively trying to interpret others charitatively, argue to the point, not use provocative or rude language, apologise for inadvertent offenses while not being overtly prone to take offense themselves, avoid their own biases and fallacies instead seeking them in others, and most importantly, find the truth instead of winning the argument. Maybe the greatest attribute of this approach is its infectivity - I have observed several newcomers to change their discussing habits for better in few weeks. However, not everybody is susceptible to the LW standards and our attitude produces somewhat bizarre results when confronted with genuine trolls.
Recent posts about epistemology1 have all generated large number of replies; in fact, the discussions were among the largest in the last few months. People have commented there (yes, I too am guilty) even if it was clear that the author of the posts doesn't actually react to our arguments. After he was rude and had admitted to do it on purpose. After commiting several fallacies, after generating an unreasonable amount of text of mediocre to low quality, after saying that he is neither trying to convince anyone nor he is willing to learn anything nor he aims for agreement. In short, perhaps all symptoms of trolling were present, and still, people were repeatedly patiently explaining what's wrong with the author's position. Which reaction is, I must admit, sort of amazing - but on the other hand, it is hard to deny that the whole discussion was detrimental to the quality of LW content and was mostly a waste of time.
So, here is the question: why didn't we apply the don't feed the troll meme, as would probably happen much sooner on most forums? I have several hypotheses on that.
1. We are unable to recognise trolls for lack of training. The first hypothesis is quite improbable, given that the concerned troll was downvoted to oblivion2, but still possible. There are not many trolls on LW and perhaps it is difficult to believe that someone is actively seeking that sort of confrontation. I have never understood the psychology of trolls - I try to avoid combative arguments instinctively and find it hard to imagine why somebody would intentionally try to create one. Perhaps a manifestation of the typical mind fallacy combines with compartmentalisation here: although we consciously know that there are trolls out there (as this is hard to ignore), when meeting one our instict tells us that the person cannot be so much different from us.
2. We are unwilling to deal with trolls. The second theory is that although we know that a person isn't sincere, we cherish our standards of discussion so strongly that we still try to respond kindly and maintain a civil debate, or at least one side of the debate. If it is the case, it is not automatically a bad policy. Our rationality is limited and we always operate under the threat of self-serving biases. Some quasi-deontological rule of kindness in debates, even if it is an overkill, may be useful in the same way presumption of innocence is useful in justice.
3. Sunken costs. Once the debate has started, our initial investments feel binding. It is unsettling to quit an argument admitting that it was completely useless and we have lost an hour of our life for nothing. Sunken costs fallacy is well known and widespread, there is no reason to expect we are immune.
4. Best rebuttal contest. An interesting fact is that not only the number of replies was fairly large, but also lot of replies were strongly upvoted. It leads me to suspect that those replies weren't in fact aimed at the opponent in the discussion, but rather intended to impress the fellow LessWrongers. Once the motivation is not "I want to convince my interlocutor" but rather "I can craft an extraordinarily elegant counter-argument which until now didn't appear", the attitude of the opponent doesn't matter. The debate becomes an exercise in arguing, a potentially useful practice maybe, but with many associated dangers.
5. Trollish arguments are fun. I include this possibility mainly for completeness since I don't much believe that significant number of LW users enjoy pointless arguments. But still, there is something fascinating in fallacious arguments. They are frustrating to follow, for sure, especially for a rationalist, but I cannot entirely leave out of consideration the appeal of seeing biases and fallacies in real life, as opposed to mere reading about them in a Kahneman and Tversky paper.
Whatever of the above hypotheses is correct, or even if none of them is correct, I don't doubt that on reflection most of us would prefer to have less irrational discussions. The karma system works somehow, but slowly, and cannot prevent the trollish discussions from gaining momentum if people continue in their present voting patterns. One of the problems lies in upvoting the rebuttals which gives additional motivation for people to participate. There seem to be two main strategies of voting: "I want to see more/less of this" and "this deserves more/less karma than it presently has". The first strategy seems marginally better for dealing with trolls, but both strategies should work better when applied in context. Even a brilliant reply should not be upvoted when placed in an irrational debate: first, it is mostly wasting of resources, and more, we certainly want to see less irrational debates. I don't endorse downvoting good replies, if only because the troll could interpret it as support for his cause. But leaving them on zero seems to be a correct policy.
1 I am not going to link to them because I don't want to generate more traffic there; one of those posts figures already on the 4th place when you Google lesswrong epistemology. Neither I write down the precise topic or the name of the author explicitly, which I hope decreases the probability of his appearing here.
2 In fact, the downvoting, even if massive, came relatively late, with the person in question being able to post on the main site after several days.