Interest In Conflict Is Instrumentally Convergent
20Viliam
8Screwtape
3samuelshadrach
3Viliam
6CronoDAS
1Richard Horvath
4Viliam
14NunoSempere
9Said Achmiz
7Screwtape
8Said Achmiz
12CronoDAS
6CronoDAS
3Said Achmiz
2CronoDAS
3Said Achmiz
2Screwtape
2Said Achmiz
2Screwtape
3Said Achmiz
2Screwtape
3Said Achmiz
6Screwtape
3Hastings
2Screwtape
1dirk
2Said Achmiz
2[comment deleted]
2CronoDAS
3Said Achmiz
4Screwtape
2Said Achmiz
5Viliam
4Said Achmiz
18Said Achmiz
2Jiro
2Said Achmiz
2Jiro
2Said Achmiz
2Jiro
2Said Achmiz
2Jiro
2Said Achmiz
2Jiro
2Said Achmiz
2Jiro
2Said Achmiz
2dirk
2NunoSempere
2Said Achmiz
3NunoSempere
2Said Achmiz
2Screwtape
2Said Achmiz
2Screwtape
9Said Achmiz
5d_el_ez
3WillPetillo
1samuelshadrach
New Comment
This is a horrible situation, where excessive knowledge of some bad action X could be evidence of being:
- the kind of bad actor who does X or plans to do it, or
- a person who was falsely accused of doing X, or
- someone who spends a lot of effort on protecting themselves or others from X, or
- a nerd who happens to be obsessed with X.
Taking all of them them together, we get a group that has the best knowledge of X, and which is dangerous to approach, because too many of them are bad actors. (Also, there is a possible overlap between the groups.)
Even worse, if you decide to avoid approaching the group and just study X yourself... you become one of them.
However, having zero knowledge about X makes you an easy victim. So what are we supposed to do?
I guess the standard solution is something like "try to learn about X without coming into too much contact with the teachers (learn from books, radio, TV, internet, or visit a public lecture), and keep your knowledge about X mostly private (do things that reduce the chance of X happening to you, maybe warn your friends about the most obvious mistakes, but do not give lectures yourself)" or sometimes "find a legible excuse to learn about X (join the police force)". Which, again, is something that the bad actor would be happy to do, too.
They say that the former criminals make the most efficient cops. I believe it also works the other way round.
I guess Jordan Peterson would say that you cannot become stronger without simultaneously becoming a potential monster. The difference between good and bad actors is how much the "potential" remains potential. Weak (or ignorant) people are less of a threat, but also less of a help.
It could help if you know the person for a long time, so you could see how the excessive knowledge of X manifests in their actual life. Difficult to do for an online community.
Like I said, I don't have a solution. At least, not one I'm confident and certain of. I have other essays in the pipeline with (optimistically) pieces of it.
I don't think it's doomed. Most security experts a bank would reasonably hire are not bank robbers, you know? I assume that's true anyway, I'm not in that field but somehow my bank account goes un-robbed.
Checking where wildly different spheres agree seems promising. The source of advice here that I trust the most comes from a social worker who I knew for years who hadn't heard of the rationalist community, and I asked them instead of them unprompted (or as part of an argument) starting to tell me how it should work. Put another way, getting outside perspectives is helpful- if a romantic partner seems like they might be pressuring you, describe it to a friend and see what they say.
It's part of why I spent a while studying other communities, looking to see if there was anything that say, Toastmasters and the U.S. Marines and Burning Man and Worldcon all agreed about.
Most security experts a bank would reasonably hire are not bank robbers, you know? I assume that's true anyway,
If you're good at it, you can purchase the knowledge without giving them a position of power. Intelligence agencies purchase zero days from hackers on black market. Foreign spies can be turned using money to become double agents.
Most security experts a bank would reasonably hire are not bank robbers, you know?
Yes, it would be useful to know how exactly that happens.
I suspect that a part of the answer is how formal employment and long-term career changes the cost:benefit balance. Like, if you are not employed as a security expert, and rob a bank, you have an X% chance of getting Y money, and Z% chance of ending up in prison. If you get hired as a security expert, that increases the X, but probably even more increases the Z (you would be the obvious first suspect), and you probably get a nice salary so that somewhat reduces the temptation of X% chance at Y. So even if you hire people who are tempted to rob a bank, you kinda offer them a better deal on average?
Another part of the answer is distributing the responsibility, and letting the potential bad actors keep each other in check. You don't have one person overseeing all security systems in the bank without any review. One guy places the cameras, another guy checks whether all locations are recorded. One guy knows a password to a sensitive system (preferably different people for different sensitive systems), another guy writes the code that logs all activities in the system. You pay auditors, external penetration testers, etc.
There is also reputation. If someone worked in several banks, and they those banks didn't get robbed, maybe it is safe to hire that person. (Or they play a long con. Then again, many criminals probably don't have patience for too long plans.) What about your first job? You probably get a role with less responsibility. And they probably check your background?
...also, sometimes the banks do get robbed; they probably do not always make it public news. So I guess there is no philosophically elegant solution to the problem, just a bunch of heuristics that together reduce the risk to the acceptable level (or rather, we get used to whatever is the final level).
So... yeah, it makes sense to learn the heuristics... and there will be obvious objections... and some of the heuristics will be expensive (in money and/or time).
I think the amount of cash a bank loses in a typical armed robbery really isn't that large compared to the amounts of money the bank actually handles - bank robbers are a nusiance but not an existential threat to the bank.
The actual big danger to banks comes from insiders; as the saying goes, the best way to rob a bank is to own one.
Any opinion on this regarding being a somewhat good solution?
https://www.lesswrong.com/posts/Q3huo2PYxcDGJWR6q/how-to-corner-liars-a-miasma-clearing-protocol
Trying to summarize the method:
- list all know facts
- list all competing theories
- make an M×N table, highlight places where the fact contradicts the theory
- require an explanation for each such place in the current theory
- if a new theory is made, add a new column to the table, and evaluate all cells in the new column
I guess this mostly avoids the failure mode when someone uses an argument A to support their theory X, later under the weight of evidence B switches to a theory Y (because B was incompatible with X, but is compatible with Y), and you fail to notice that A is now incompatible with Y... because you vaguely remember that "we talked about A, and there was a good explanation for that".
The admitted disadvantage is that it takes a lot of time.
Makes me empathize with the defender :), but let me tell you, being interrogated in an airport for six hours trying to convince a US immigration agent that I'm an oddball not a danger is not fun
All of this sounds reasonable, on the surface…
And yet I notice that the view that “people who have opinions about how [whatever] should be done are unusually likely to be bad actors who want me to do [whatever] in such a way as to benefit them, therefore I should be suspicious of their motives and suggestions” is memetically adaptive. Whenever you come across this idea, it is to your benefit to immediately adopt it—after all, it means that you will thenceforth need to spend less effort evaluating people’s suggestions and opinions, and have a new and powerful reason to reject criticism. And the idea protects itself: if someone suggests to you that this whole perspective is misguided and harmful, well, aren’t they just maliciously trying to undermine your vigilance?
Anyhow, I am not a meetup czar, so I don’t have to make the decisions that you make. And I don’t go to many meetups, so I am more or less unaffected by those decisions. I do have a bit of experience running communities, though; and, of course, the usual plethora of experience interacting with people who run communities. My own view, on the basis of all of that experience, is this:
Community members should default to the assumption that you are basically the KGB.
Your own approach and policies should work unproblematically even if everyone assumes that you are basically the KGB. (This is especially true if you are not the KGB at all.)
And if your approach to running a community is predicated on the members of that community not treating you as if you are the KGB, then you are definitely the KGB.
And yet I notice that the view that “people who have opinions about how [whatever] should be done are unusually likely to be bad actors who want me to do [whatever] in such a way as to benefit them, therefore I should be suspicious of their motives and suggestions” is memetically adaptive. Whenever you come across this idea, it is to your benefit to immediately adopt it—after all, it means that you will thenceforth need to spend less effort evaluating people’s suggestions and opinions, and have a new and powerful reason to reject criticism. And the idea protects itself: if someone suggests to you that this whole perspective is misguided and harmful, well, aren’t they just maliciously trying to undermine your vigilance?
I have a couple of thoughts here. One is that I don't think this is true for most values of [whatever]. If someone has suggestions about the venue, or the meetup activities, or announcement platforms, I don't think this dynamic is in play. If I get advice on job searching or algebra homework or the best way to bake a loaf of sourdough, I'm not getting nearly as much adverse selection as for conflict resolution from within the community I'm involved in. Who has a motive to subtly sabotage my sourdough?
If someone read this essay and came away with a full general counterargument against listening to advice on any subject, my guess is there's a big reading comprehension failure happening.
It isn't as clearly a failure of reading comprehension if someone comes away with the idea that they shouldn't listen to any advice on handling conflict specifically, though I think that would also be incorrect. Finding people who are trustworthy, good at handling it well, and willing to teach you is wonderful. I've been trying to learn the most from sources well outside the rationalist community, but I think there is good advice to be had. Just, not uncritically trusted?
Also, some people seem to think this class of problem should be easy. For those people I want to make the point that it is (at least sometimes) an adversarial situation.
Who has a motive to subtly sabotage my sourdough?
Probably nobody, but then again, your sourdough is probably not impinging on anyone’s interests, either. Baking a loaf of sourdough doesn’t really come with opportunities to exploit other people for your own gain, etc. So of course there’s not going to be much controversy.
But whenever there is controversy, usually due to the existence of genuinely competing interests, then motives for sabotage become plausible, whereupon it immediately becomes tempting to declare that those who think that you ought to be doing things differently are just trying to sabotage you.
Finding people who are trustworthy, good at handling it well, and willing to teach you is wonderful. I’ve been trying to learn the most from sources well outside the rationalist community, but I think there is good advice to be had. Just, not uncritically trusted?
Also, some people seem to think this class of problem should be easy. For those people I want to make the point that it is (at least sometimes) an adversarial situation.
I agree, it certainly is an adversarial situation—and not only sometimes, but most of the time. And I agree that you should not uncritically trust advice that you hear from any sources. In fact, you shouldn’t even trust advice that you hear from yourself.
Consider your bank example again. You might think: “hmm, that guy has an odd amount of knowledge of, and/or interest in, internal bank practices and security and so on; suspicious!”. Then you learn that he works at a bank himself, so it turns out that his knowledge and interest aren’t suspicious after all—great, cancel that red flag.
No! Wrong! Don’t cancel it! Put it back! Raise two red flags! (“An analysis by the American Bankers Association concluded that 65% to 70% of fraud dollar losses in banks are associated with insider fraud.”) Suspect everyone, especially the people you’ve already decided to trust!
But of course “suspect” is exactly the wrong word here. If you’re having to suspect people, you’ve already lost.
Consider computer security. I ask about the security software that your company is using to protect your customers’ data—could I see the code? Which cryptographic algorithms do you use? You’re suspicious; what do I need this information for? Who should be allowed to have this sort of knowledge?
And of course the right answer is “absolutely everyone”. It should be fully public. If your setup is such that it even makes sense to ask this question of “who should be allowed to know what cryptographic algorithm we use”, then your security system is a complete failure and nobody should trust you with so much as their mother’s award-winning recipe for potato salad, much less any truly sensitive data.
The way to ensure that you don’t accidentally give the wrong person insider access to your system is to construct a system such that nobody can exploit it by having insider access.
(Another way of putting this is to say that selective methods absolutely do not suffice for ensuring the trustworthiness and integrity of social systems.)
The same is true for the problem of “from whom to take advice on conflict resolution”. You should not have to figure out the motives of the advice-giver or to decide whether to trust their advice. Your procedure for evaluating advice should work perfectly even if the advice comes from your bitter enemy who wishes nothing more than to see you fail. And then you should apply that same procedure to what you already believe and the practices you are already employing—take the advice that you would give to someone, and ask what you would think of it if it had come to you from someone of whom you suspected that they might be your worst and most cunning enemy. Is your evaluation procedure robust enough to handle that?
If it is not, then any time spent thinking about whether the source of the advice is trustworthy is pointless, because you can’t very well trust someone else more than you trust yourself, and you evaluation procedure is too weak to guard against your own biases. And if it is robust enough, then once again it is pointless to wonder whom you should trust, because you don’t have to trust anyone—only to verify.
And of course the right answer is “absolutely everyone”. It should be fully public. If your setup is such that it even makes sense to ask this question of “who should be allowed to know what cryptographic algorithm we use”, then your security system is a complete failure and nobody should trust you with so much as their mother’s award-winning recipe for potato salad, much less any truly sensitive data.
This makes sense for computer security, but for biosecurity it doesn't work, because it's a lot harder to ship a patch to people's bodies than to people's computers. The biggest reason there has never been a terrorist attack with a pandemic-capable virus is that, with few exceptions (such as smallpox), we don't know what they are.
A: My understanding is that the U.S. Government is currently funding research programs to identify new potential pandemic-level viruses.
K: Unfortunately, yes. The U.S. government thinks we need to learn about these viruses so we can build defenses — in this case vaccines and antivirals. Of course, vaccines are what have gotten us out of COVID, more or less. Certainly they’ve saved a ton of lives. And antivirals like Paxlovid are helping. So people naturally think, that’s that’s the answer, right?
But it’s not. In the first place, learning whether a virus is pandemic capable does not help you develop a vaccine against it in any way, nor does it help create antivirals. Second, knowing about a pandemic-capable virus in advance doesn’t speed up research in vaccines or antivirals. You can’t run a clinical trial in humans on a new virus of unknown lethality, especially one which has never infected a human — and might never. And given that we can design vaccines in one day, you don’t save much time in knowing what the threat is in advance.
The problem is there are around three to four pandemics per century that cause a million or more deaths, just judging from the last ones — 1889, 1918, 1957, 1968 and 2019. There’s probably at least 100 times as many pandemic-capable viruses in nature — it’s just that most of them never get exposed to humans, and if they do, they don’t infect another human soon enough to spread. They just get extinguished.
What that means is if you identify one pandemic-capable virus, even if you can perfectly prevent it from spilling over and there’s zero risk of accidents, you’ve prevented 1/100 of a pandemic. But if there’s a 1% chance per year that someone will assemble that virus and release it, then you’ve caused one full pandemic in expectation. In other words, you’ve just killed more than 100 times as many people as you saved.
you can’t very well trust someone else more than you trust yourself
In certain domains, I absolutely can and will do this, because "someone else" has knowledge and experience that I don't and could not conveniently acquire. For example, if I hire lawyers for my business's legal department, I'm probably not going to second-guess them about whether a given contract is unfair or contains hidden gotchas, and I'm usually going to trust a doctor's diagnosis more than I trust my own. (The shortfalls of "Doctor Google" are well-known, so although I often do "do my own research" I only trust it so much.)
In certain domains, I absolutely can and will do this, because “someone else” has knowledge and experience that I don’t and could not conveniently acquire.
And how do you choose who the “someone else” is?
Honestly? By going to the list of doctors that my health insurance will pay for, or some other method of semi-randomly choosing among licensed professionals that I hope doesn't anti-correlate with the quality of their advice. There are probably better ways, but I don't know what they are offhand. ::shrug::
If you were accused of a crime and intended to plead not guilty, how would you choose a defense attorney, assuming you weren't going to use a public defender?
So you trust yourself to decide how to select a doctor; you trust your decision procedure, which you have chosen.
If you were accused of a crime and intended to plead not guilty, how would you choose a defense attorney, assuming you weren’t going to use a public defender?
I’d ask trusted friends for recommendations, because I trust myself to know whom to ask, and how to evaluate their advice.
I would be delighted to have the social equivalent of a zero trust conflict resolution system that everyone who interacted with it could understand and where the system could also maintain confidentiality as needed. I'm in favour of the incremental steps towards that I can make. In the abstract, I agree the procedure for evaluating advice should work even if it comes from bitter enemies. I do not think my personal evaluation procedure is currently robust enough to handle that, though tsuyoku naritai, someday maybe it will be.
The main context I encounter these problems is in helping local ACX meetup organizers. Some of them first found the blog a few months ago, ran a decent ACX Everywhere that blossomed into a regular meetup group, and then a conflict happened. I want good advice or structures to hand to them, and expecting them to be able to evaluate my advice to that standard seems unreasonable. It's likely that at least one and possibly all of the local belligerents will have suggestions, and those suggestions will conveniently favour the advice-giver.
One way to read this essay, which I would endorse as useful, is as one useful answer to the question "why do all the people in this conflict I find myself in all have such different ideas of the procedure we should use to resolve it?"
I’m in favour of the incremental steps towards that I can make. In the abstract, I agree the procedure for evaluating advice should work even if it comes from bitter enemies. I do not think my personal evaluation procedure is currently robust enough to handle that, though tsuyoku naritai, someday maybe it will be.
Yes, but in the absence of this, every other approach is doomed to failure. And the severity of the failure will be inversely proportional to how seriously the people with the power and authority take this problem, and to how much effort they put into addressing it.
I want good advice or structures to hand to them, and expecting them to be able to evaluate my advice to that standard seems unreasonable.
Respectfully, I disagree. I think that this is the only standard that yields workable results. If it cannot be satisfied even approximately, even in large part (if not in whole), then better not to begin.
I'm trying to come up with people that I think actually reach the standard you're describing. I think I know maybe ten, of which two have any time or interest in handling meetup conflicts.
I do agree there's some big failures that can happen when the people with authority to solve the problem take it very seriously, put a lot of effort into addressing it, and screw up. I don't agree that relationship is inversely proportional; if I imagine say, a 0 effort organizer who does nothing vs a 0.1 effort organizer who only moderates to say "shut up or leave" to a attendees who keep yelling their political opponents should be killed, this seems like an improvement. There's a lot of low hanging fruit here.
It's possible "even approximately, even in large part" covers a much greater range than I'm interpreting it as and your standard is lower than it sounds. If not, I think we're at an impasse of a disagreement. I think that if nobody does any conflict resolution at all unless they are that good of an evaluator, all but a vanishingly small number of spaces will become much worse. We're talking on LessWrong, I do not think the moderators here are at that level, and yet the space is much improved relative to other places. Seems like 4chan decided better not to begin, and I like LessWrong more.
I do agree there’s some big failures that can happen when the people with authority to solve the problem take it very seriously, put a lot of effort into addressing it, and screw up. I don’t agree that relationship is inversely proportional; if I imagine say, a 0 effort organizer who does nothing vs a 0.1 effort organizer who only moderates to say “shut up or leave” to a attendees who keep yelling their political opponents should be killed, this seems like an improvement. There’s a lot of low hanging fruit here.
Er, sorry, I think you might’ve misread my comment? What I was saying was that the more seriously the people with the power and authority take the problem, the better it is. (I think that perhaps you got the direction backwards from how I wrote it? Your response would make sense if I had said “directly proportional”, it seems to me.)
I think that if nobody does any conflict resolution at all unless they are that good of an evaluator, all but a vanishingly small number of spaces will become much worse. We’re talking on LessWrong, I do not think the moderators here are at that level, and yet the space is much improved relative to other places. Seems like 4chan decided better not to begin, and I like LessWrong more.
“Better not to begin” wouldn’t be “4chan”, it would be “nothing”.
I agree that the moderators on Less Wrong aren’t quite at the level we’re talking about, but they’re certainly closer than most people in most places. (And many of what I perceive to be mistakes in moderation policy are traceable to the gap between their approach, and the sort of approach I am describing here.) At the very least, it’s clear that the LW mods have considerable experience with having to evaluate advice that does, in fact, come from their (our) enemies.
Er, sorry, I think you might’ve misread my comment? What I was saying was that the more seriously the people with the power and authority take the problem, the better it is. (I think that perhaps you got the direction backwards from how I wrote it? Your response would make sense if I had said “directly proportional”, it seems to me.)
"And the severity of the failure will be inversely proportional to how seriously the people with the power and authority take this problem, and to how much effort they put into addressing it."
Hrm. Yes, I seem to have read it differently, apologies. I think I flipped the sign on "the severity of the failure" where I interpreted it as the failure being bigger the more seriously people with power and authority took the problem.
“Better not to begin” wouldn’t be “4chan”, it would be “nothing”.
I agree that the moderators on Less Wrong aren’t quite at the level we’re talking about, but they’re certainly closer than most people in most places.
Yeah. I prefer having LessWrong over having nothing in its place. I even prefer having LessWrong over having nothing in the place of everything shaped like an internet forum.
Do the LW mods pass your threshold for good enough it's worth beginning? I think a lot of my incredulity here comes from trying to figure out how big that gap is, though in terms of the specific problem I'm trying to solve I think I need to take as a premise that I start with whatever crop of ACX organizers I'm offered by selection effects.
Do the LW mods pass your threshold for good enough it’s worth beginning?
Well… hard to say. The LW mods now pass that threshold[1], but then again they’re not beginning now; they began eight years ago.
I think a lot of my incredulity here comes from trying to figure out how big that gap is, though in terms of the specific problem I’m trying to solve I think I need to take as a premise that I start with whatever crop of ACX organizers I’m offered by selection effects.
Yes… essentially, this boils down to a pattern which I have seen many, many times. It goes like this:
A: You are trying to do X, which requires Y. But you don’t have Y.
B: Well, sure, I mean… not exactly, no. I mean, mostly, sort of… (a bunch more waffling, eventually ending with…) Yeah, we don’t have Y.
A: So you can’t do X.
B: Well, we have to, I’m afraid…
A: That’s too bad, because you can’t. As we’ve established.
B: Well, what are we going to do, just not do X?
A: Right.
B: Unacceptable! We have to!
A: You are not going to successfully do X. That will either be because you stop trying, or because you try but fail.
B: Not doing X is not an option!
B tries to do X
B fails to do X, due to the lack of Y
A: Yep.
B: Well, we have to do our best!
A: Your best would be “stop trying to do X”.
B ignores A, continues trying to do X and predictably failing, wasting resources and causing harm indefinitely (or until external circumstances terminate the endeavor, possibly causing even more harm in the process)
In this case: a bunch of people who are completely unqualified to run meetups are trying to run meetups. Can they run meetups well? No, they cannot. What should they do? They should not run meetups. Then who will run the meetups? Nobody.
Now, while reading the above, you might have thought: “obviously B should be trying to acquire Y, in order to successfully do X!”. I agree. But that does not look like “do X anyway, and maybe we’ll acquire Y in the process”. (Y, in this case, is “the skills that we’ve been discussing in this comment thread”.) It has to be a goal-directed effort, with the explicit purpose of acquiring those skills. It can be done while also starting to actually run meetups, but only with an explicit awareness and serious appreciation of the problem, and with serious effort being continuously put in to mitigate the problem. And the advice for prospective meetup organizers should tackle this head-on, not seek to circumvent it. And there ought to be “centralized” efforts to develop effective solutions which can then be taught and deployed.
You might say: “this is a high bar to clear, and high standards to meet”. Yes. But the standards are not set by me, they are set by reality; and the evidence of their necessity has been haunting us for basically the entirety of the “rationalist community”’s existence, and continues to do so.
Approximately, anyway. There’s a bunch of mods, they’re not all the same, etc. ↩︎
Well… hard to say. The LW mods now pass that threshold[1], but then again they’re not beginning now; they began eight years ago.
My sense is that if the mods had waited to start trying to moderate things until they met this threshold, they wouldn't wind up ever meeting it. There's a bit of, if you can't bench press 100lbs now, try benching 20lbs now and you'll be able to do 100lbs in a couple years, but if you just wait a couple years before starting you won't be able to then either.
Ideally there's a way to speed that up and among the ideas I have for that is writing down some lessons I've learned in big highlighter. I'm pretty annoyed at how hard it is to get a good feedback loop and get some real reps in here.
Yes… essentially, this boils down to a pattern which I have seen many, many times. It goes like this:
...
In this case: a bunch of people who are completely unqualified to run meetups are trying to run meetups. Can they run meetups well? No, they cannot. What should they do? They should not run meetups. Then who will run the meetups? Nobody.
There are circumstances where trying and failing is very bad. If someone is trying to figure out heart surgery, I think they should put the scalpel down and go read some anatomy textbooks first, maybe practice on some cadavers, medical school seems a good idea. I do not think meetups are like this and I do not think the majority of the organizers are completely unqualified; even if they're terrible at the interpersonal conflict part they're often fine at picking a location and time and bringing snacks. That makes them partially qualified.
The -2std failure case is something like, they announced a time and place that's inconvenient, then show up half an hour late and talk over everyone, so not many people come and attendees don't have a good time. This is not great and I try to avoid that outcome where I can, but it's not so horrible that I'd give up ten average meetups to prevent it. Worse outcomes do happen where I do get more concerned.
It's possible you have a higher bar or a different definition of what a rationalist meetup aught to be? I'm on board with a claim something like "a rationalist meetup aught to have some rationality practiced" and in practice something like (very roughly) a third of the meetups are pure socials and another third are reading groups. Which, given my domain is ACX groups, isn't that surprising. Conflict can come for them anyway.
Hrm. Maybe a helpful model here is I'm trying to reduce the failure rate? The perfect spam filter bins all spam and never bins non-spam. If someone woke up, went to work, and improved the spam filter such that it let half as much spam through, that would be progress. If because of my work half the [organizers that would have burned out/ attendees who would have been sadly driven away/ maleficers who would have caused problems] have a better outcome, I'll call it an incremental victory.
And there ought to be “centralized” efforts to develop effective solutions which can then be taught and deployed.
waves Hi, one somewhat central fellow, trying to develop some effective solution I can teach. I don't think I'm the only one (as usual I think CEA is ahead of me) but I'm trying. I didn't write much about this for the first year or two because I wasn't sure which approaches worked and which advisors were worth listening to. Having gone around the block a few times, I feel like I've got toeholds, at least enough to hopefully warn away some fools mates.
fwiw, these are what I'd say a 2std failure case of a rationalist meetup looks like
https://www.wired.com/story/delirious-violent-impossible-true-story-zizians/
https://variety.com/2025/tv/news/julia-garner-caroline-ellison-ftx-series-netflix-1236385385/
https://www.wired.com/story/book-excerpt-the-optimist-open-ai-sam-altman/
(Ways my claim could be false: there could have been way more than 150 rationalist meetups, so that these are lower than 2 std, or these could not have, at any point in their development, counted as rationalist meetups, or ziz, sam, and eliezer could have intended these outcomes, so these don't count as failures)
I think of Ziz and co as less likely than 2std out, for about the reasons you give. I tend to give 200 as the rough number of organizers and groups, since I get a bit under that for ACX Everywhere meetups in a given season. If we're asking per-event, Dirk's ~5,000 number sounds low (off the top of my head, San Diego does frequent meetups but only the ACX Everywheres wind up on LessWrong, and there are others like that) but I'd believe 5,000~10,000.
You're way off on the number of meetups. The LW events page has 4684 entries (kudos to Said for designing GreaterWrong such that one can simply adjust the URL to find this info). The number will be inflated by any duplicates or non-meetup events, of course, but it only goes back to 2018 and is thus missing the prior decade+ of events; accordingly, I think it's reasonable to treat it as a lower bound.
There are circumstances where trying and failing is very bad. If someone is trying to figure out heart surgery, I think they should put the scalpel down and go read some anatomy textbooks first, maybe practice on some cadavers, medical school seems a good idea. I do not think meetups are like this and I do not think the majority of the organizers are completely unqualified; even if they’re terrible at the interpersonal conflict part they’re often fine at picking a location and time and bringing snacks. That makes them partially qualified.
FWIW, my experience is that rationalist meetup organizers are in fact mostly terrible at picking a location and at bringing snacks. (That’s mostly not the kind of failure mode that is relevant to our discussion here—just an observation.)
Anyhow…
The −2std failure case is something like, they announced a time and place that’s inconvenient, then show up half an hour late and talk over everyone, so not many people come and attendees don’t have a good time. This is not great and I try to avoid that outcome where I can, but it’s not so horrible that I’d give up ten average meetups to prevent it. Worse outcomes do happen where I do get more concerned.
All of this (including the sentiment in the preceding paragraph) would be true in the absence of adversarial optimization… but that is not the environment we’re dealing with.
(Also, just to make sure we’re properly calibrating our intuitions: −2std is 1 in 50.)
It’s possible you have a higher bar or a different definition of what a rationalist meetup aught to be? I’m on board with a claim something like “a rationalist meetup aught to have some rationality practiced” and in practice something like (very roughly) a third of the meetups are pure socials and another third are reading groups.
No, I don’t think that’s it. (And I gave up on the “a rationalist meetup aught to have some rationality practiced” notion a long, long time ago.)
Is it better to have a rationality meetup with a baseline level of unskilled ad-hoc conflict resolution, or no rationality meetup? (You can ask the same question about any social event potentially open to the public.) I imagine the answer would depend on the number of people expected to attend - informal methods work better for groups of 10 than for groups of 100.
Not sure how referential "you" vs general "you" you're using when you're talking about assuming some "you" is the KGB. I do think it's useful to build a system which does not assume the watchman is perfectly trustworthy and good. In my own case, one of the first things I did once I started to realize how tricky this part of my role might be was write down a method for limited auditing of myself. That said:
Your own approach and policies should work unproblematically even if everyone assumes that you are basically the KGB. (This is especially true if you are not the KGB at all.)
I'm not sure how literally to take the "unproblematically" adverb here. If you're being literal, then I disagree; part of my thesis here is that sometimes there will be as many problems as enemy action can cause, and they will be able to cause some problems.
(If you're on the lookout for a fully general counterargument, here's one I haven't found a way around! This theory treats occasional strident complaints about the way a resolution system is operating as very little evidence that the system is operating badly, because one would expect occasional bad actors to try shaking everyone's trust in the system even if it was a good system. And yes, that is such a suspicious theory for me in particular to put forward. Dunno what to tell you here.)
And yes, that is such a suspicious theory for me in particular to put forward.
Indeed. But really, I wouldn’t say “suspicious”, exactly; I’d say “yes, it makes perfect sense that you would say this”. This isn’t even an accusation, or anything like that. It’s just the logical outcome of the setup.
The question is, can a bad actor shake everyone’s trust in the system? If they can, then is it really a good system?
The best answer to “should I trust you[r system]?” isn’t “yes, you should, and here is why”. It’s “you don’t have to”.
It’s “you don’t have to”.
My current best guess about what you are trying to say is something like this: "People should give up on the idea of making systems that are resilient against bad actors on both sides. You should just give unlimited power to one side (the moderator, the meetup czar, the police...) and that's it. Now at least the system is resilient against bad actors on one side."
EDIT: Never mind, after reading your other comments, I guess you believe that community moderation can be solved by an algorithm. Ok, I might believe it if you show me the code.
My current best guess about what you are trying to say is something like this: “People should give up on the idea of making systems that are resilient against bad actors on both sides. You should just give unlimited power to one side (the moderator, the meetup czar, the police...) and that’s it. Now at least the system is resilient against bad actors on one side.”
Uh… no. Definitely not.
I guess you believe that community moderation can be solved by an algorithm.
… what in the world?
No, I don’t believe anything like this.
Honestly, it would be hard to get further from my views on this subject than what you’ve described…
Here is some commentary on my views on this subject, framed with a bit of literary analysis. (This was inspired by a semi-recent rereading of a pair of Alexander Wales stories, namely Shadows of the Limelight and The Metropolitan Man. NOTE: This comment contains spoilers… well, a spoiler… but only a very mild and general one.)
The key point is: Superman is evil.
For anyone who hasn’t read The Metropolitan Man: it’s a Superman fanfic, wherein Superman is revealed to be evil. That is: it’s not an AU fic where Superman, instead of being good like he normally is, is instead evil; rather, it’s just the regular Superman, but the story shows (or so I claim! it’s not made explicit, of course) that regular Superman is evil.
Now, this is hardly a novel insight, but the trick is articulating just what the nature of that evil is; but I’ve finally managed to do so, to my own satisfaction. The problem with Superman (as depicted in The Metropolitan Man—and also, of course, many, many canon Superman stories) is the chain of reasoning that goes:
“With great power comes great responsibility” -> “With great responsibility comes great moral leeway”
In other words, you are responsible for the lives of many people, the fate of a nation/planet/whatever, or some other great and important thing; therefore you cannot be expected to scrupulously follow rules that normal people are held to, and are allowed to break the law, to lie, to imprison innocents, or whatever else.
But why do you have that great responsibility? Well, because you have great power, so you were obligated to take on that responsibility.
But then result of this is: “With great power comes great moral leeway”.
Which is just “might makes right”.
So, through this concept of moral obligation, you have reasoned yourself into being exempt from the rules because you’re very powerful; and of course, being powerful, you also can’t have those rules enforced upon you—very convenient!
That’s Superman in a nutshell. (Also many other superheroes, but Superman is the most famous example.)
The correct solution (morally speaking) is, of course, to deny the first step: no, with great power does not come great responsibility—except the responsibility of being careful not to use that power to do harm. Anything other than that, over and above any responsibilities that any person has, is strictly supererogatory.
And of course one does not have to be Superman to encounter this problem. Whenever you gain power, there is a very great temptation to use this power; and it is a dangerous delusion, to think that the only problematic temptation is the temptation to use your power to aggrandize or enrich yourself directly. The temptation to use your power to benefit others is even more perilous.
So what is to be done?
If you find yourself with power, do not begin by thinking “how can I use this to do good”. Think first:
- How can I ensure that I do not use this to do harm?
- How can I share this power, so that I am not the only one who has it? Can I set things up so that everyone has it?
- Does anyone need to have this power? Can I set things up so that nobody has it?
(Note that these are not numbered—I am not saying that you need to think of these things in that order. Think of them all, simultaneously.)
(There are also auxiliary questions, like “If someone whom I didn’t trust at all were to gain this power, what would I wish they’d do with it?”, and “Suppose that I become crazy and/or evil tomorrow; what should I do to prevent bad-future-me from abusing this power?”. Note that if the above-listed primary questions are addressed effectively, these auxiliary questions also disappear almost entirely.)
(The astute reader may notice parallels between the above perspective and certain dichotomies among different types of economic systems, approaches to the construction of complex abstract systems, etc.)
But then result of this is: “With great power comes great moral leeway”.
Which is just “might makes right”.
That doesn't follow. The result of combining these two things is that with great power comes great moral leeway when doing specific things. He can violate the law by breaking into a home to catch a villain, but he can't if he just sees someone's big TV and wants to take it for himself. He doesn't get moral leeway to do things just because he personally benefits from them, and "might makes right" is typically assumed to include that. Furthermore, he doesn't get unlimited leeway bounded by people's ability to stop him, like "might makes right" also assumes. If he has to destroy property to save lives, he can, but he still needs to take reasonable steps to minimize property damage.
(And he only gets moral leeway on a stricter standard than utilitarianism. He can't steal that TV just because he gets more utility from it than its current owner. He can't even steal it to sell it, donate the money to charity, and help 100 malaria victims.)
That doesn’t follow. The result of combining these two things is that with great power comes great moral leeway when doing specific things. He can violate the law by breaking into a home to catch a villain, but he can’t if he just sees someone’s big TV and wants to take it for himself. He doesn’t get moral leeway to do things just because he personally benefits from them, and “might makes right” is typically assumed to include that.
I’m sure that you know quite well that people with power are exceptionally skilled at finding justifications for how they use their power, even if what they’re actually doing is enriching and aggrandizing themselves and suppressing dissent.
Furthermore, he doesn’t get unlimited leeway bounded by people’s ability to stop him, like “might makes right” also assumes. If he has to destroy property to save lives, he can, but he still needs to take reasonable steps to minimize property damage.
Who decides what is reasonable? The answer is that it’s the person with the power. And others go along with this—because they want to be on his side, or because they fear him, or simply because they “like the strong horse” (which is, of course, nothing more than an evolved response that encodes the preceding two game-theoretic considerations).
And this is what we see in superhero comics (and animated television shows, etc.). When do we see Superman exhibit a careful consideration of what steps are reasonable to take to minimize property damage? The answer is surely not “never”—simply because Superman has been written so many times by so many different writers—but that’s not the typical portrayal.
(And he only gets moral leeway on a stricter standard than utilitarianism. He can’t steal that TV just because he gets more utility from it than its current owner. He can’t even steal it to sell it, donate the money to charity, and help 100 malaria victims.)
Well, utilitarianism is not a very strict standard at all for this sort of thing, so that’s a very low bar to clear. In any case, what does Superman want with someone’s TV? The fact is that when you have that much power, you don’t need to steal people’s TVs, so the fact that you don’t do so is not indicative of anything. The real test is whether Superman (or whoever) does, or refrains from doing, something that really would benefit him—and that is unlikely to involve any petty material goods…
P.S. For a TV to yield 100 malaria victims’ worth of revenue, it would have to be sold for something like half a million dollars. Not exactly a common consumer product…
I’m sure that you know quite well that people with power are exceptionally skilled at finding justifications for how they use their power, even if what they’re actually doing is enriching and aggrandizing themselves and suppressing dissent.
But Superman isn't, in typical Superman stories. Superman just doesn't do the kind of things that would be done by someone who's really in it for themselves but is trying to justify it as needed for the greater good. (Subject to the same caveats as you--I'm sure that over 80+ years of comics, someone's written Superman that way a few times, but that's unusual.)
You could argue that actual real world people would and that Superman is thus unrealistic, but in terms of "is Superman evil" and "is Superman just using 'might makes right'", the answer is no, even if real world people would act differently.
I'm also skeptical of even this much. Not all real people are the same, after all. I could lie like Cade Metz with no consequence other than some disdain, but I have not, in fact, done so.
When do we see Superman exhibit a careful consideration of what steps are reasonable to take to minimize property damage?
Probably most of the time. Few Superman stories show people complaining about the damage Superman causes, and I don't think that's because Superman would beat them up. Just because there isn't a word balloon saying "I must minimize the damage!" doesn't change this.
But Superman isn’t, in typical Superman stories. Superman just doesn’t do the kind of things that would be done by someone who’s really in it for themselves but is trying to justify it as needed for the greater good.
The point I am making is that “might makes right, and right is this ideology that I have which involves various things that I claim are good” is still “might makes right”. Superman isn’t “in it for himself”, he’s in it for other people—which in fact is much worse.
When do we see Superman exhibit a careful consideration of what steps are reasonable to take to minimize property damage?
Probably most of the time. Few Superman stories show people complaining about the damage Superman causes, and I don’t think that’s because Superman would beat them up. Just because there isn’t a word balloon saying “I must minimize the damage!” doesn’t change this.
Well of course the stories don’t show people complaining about it, because the writers don’t want their audience to be thinking about this stuff! And that’s because people who do start thinking about it quite often come to the conclusion that actually, the way that superheroes behave w.r.t. property damage is actually quite seriously messed up.
What we see of the behavior depicted in most superhero stories is that property damage is routinely excessive. I mean, this is a well-known trope (and often parodied/deconstructed/etc., too).
The point I am making is that “might makes right, and right is this ideology that I have which involves various things that I claim are good” is still “might makes right”. Superman isn’t “in it for himself”, he’s in it for other people—which in fact is much worse.
Central examples of might makes right have many prominent aspects that are absent from what Superman does, even if in some technical sense it's "might makes right". Being in it for other people is one of those, but I mentioned several others.
Well, I already responded to those other points also, so I am not sure what else to say to this…
At this point, I can’t tell if you’re just objecting to my terminology, or what… do you just dislike the use of the term “might makes right” here? Is there some other formulation which you feel better captures my point? Or is your objection deeper somehow…?
do you just dislike the use of the term “might makes right” here?
Your term carries with it an inaccurate connotation.
(And if you didn't intend that connotation, it doesn't make sense as a criticism of Superman or a reason to call him evil anyway.)
What is the connotation?
EDIT: And what term would you prefer for the phenomenon which I described?
The connotation is all that stuff I described. "Might makes right" implies that it is being done for personal gain, not for good, and that there aren't limits to it beyond the use of more might.
And what term would you prefer for the phenomenon which I described?
You could call him a one man nanny state. But I would disagree that even this accurately describes Superman. Just like Superman doesn't casually do bad things for personal gain, he doesn't casually do them to benefit others. Stories where Superman runs the world as a dictatorship are explicitly meant as Superman being evil.
The connotation is all that stuff I described. “Might makes right” implies that it is being done for personal gain, not for good, and that there aren’t limits to it beyond the use of more might.
I see. Well, uh… I disagree. “Might makes right” implies literally none of those things, in my opinion.
And what term would you prefer for the phenomenon which I described?
You could call him a one man nanny state. But I would disagree that even this accurately describes Superman.
I agree that calling Superman a “one man nanny state” would be inaccurate, and I would certainly not use any such term.
Just like Superman doesn’t casually do bad things for personal gain, he doesn’t casually do them to benefit others.
Indeed; nor did I claim otherwise.
I agree that calling Superman a “one man nanny state” would be inaccurate, and I would certainly not use any such term.
It's inaccurate for how Superman typically behaves, but it's accurate in that it matches your claims about him. You said:
The point I am making is that “might makes right, and right is this ideology that I have which involves various things that I claim are good” is still “might makes right”. Superman isn’t “in it for himself”, he’s in it for other people—which in fact is much worse.
The term for a government which harms people for their own good is a nanny state. Superman doing similar things all on his own using his great power makes him a one man nanny state.
Superman doing similar things all on his own using his great power makes him a one man nanny state.
It does not.
A “nanny state” style government is not literally the only scenario in which someone does something to someone else for that person’s own good.
people who have opinions about how [whatever] should be done are unusually likely to be bad actors who want me to do [whatever] in such a way as to benefit them
But the inference is correct, no, since you are discarding the probability mass on "innocent normie", no?
I am not sure I follow. Could you say more? What do you mean by saying that I am discarding that probability mass?
Thanks, yes.
- The original post introduced a difference between the professionals/the curious/the oddballs/normal people/bad actors.
- People who have opinions about how [whatever] should be done are not normies (where [whatever] relates to defenses against bad actors)
- Normies are innocent but also the large majority of people.
- When learning that someone has opinions about [whatever], the prior probability of being a bad actor switches from #bad actor/(#professionals +#the curious +#the oddballs + #normal people + #bad actors) to #bad actor/(#professionals +#the curious +#the oddballs + #bad actors)
- That is, you are discarding this probability: #normal people/(#professionals +#the curious +#the oddballs + #normal people + #bad actors)
- Thus, the prior probability of someone interested in [whatever] being a bad actor rises.
I'm not saying anything the post isn't saying, I'm just pointing out that the forecasting/simple Bayesian tradition of knowledge really agrees with this post. You then have further arguments around the virtue of orienting the world around happy paths and normies, but still.
Uh… sure, that’s true enough, but this logic requires that we first accept the OP’s categorization scheme—which is part of precisely the meme that I am referring to!
OP notes that “These categories absolutely overlap and intersect some of the time”. This is true, but the trouble is that taking this caveat seriously means discarding the logic of the argument.
Consider an alternate set of categories:
- Professionals, who might also be bad people.
- The curious, many of whom are also oddballs, and who might also be bad people.
- The oddballs, who might also be bad people.
- Normal people, who might also be bad.
Hmm. Doing a Bayesian calculation on this might be tricky. Perhaps we can separate out some of those, like so:
- Professionals who are good people.
- Professionals who are bad people.
- The curious and/or oddballs who are good people.
- The curious and/or oddballs who are bad people.
- Normal people who are good.
- Normal people who are bad.
We learn that someone has opinions about [whatever]. We now discard categories #5 and #6. But unbeknownst to us, the ratio of bad:good in the overall population was actually lower than the ratio of bad:good among normal people. So learning this fact should reduce our subjective probability of the person being bad.
Is this possible? Well, for example, suppose that it’s 1975, and you learn that a certain person has opinions about how psikhushkas should work. In particular, this individual thinks that said institutions should work differently than they do in fact work. What should you conclude about this person?
In the 1960s Soviet psychiatry, particularly Serbsky Institute Director Dr. Andrei Snezhnevsky, introduced the concept of "sluggish schizophrenia", a special form of the illness that supposedly affects only social behavior, with no effect on other traits: "most frequently, ideas about a struggle for truth and justice are formed by personalities with a paranoid structure", according to the Serbsky Institute professors.[9][10]
The logic in the OP is easily recognizable as the logic of every police force, every security service, and every authoritarian enforcement organization. It’s the logic that says “if you’re not one of us, then you’re either a clueless normie who will unthinkingly submit to our authority, or else you’re probably a criminal—unless you can, with great effort and in the face of considerable skepticism, prove to us (and yes, the burden of proof is entirely on you) that you’re one of the rare harmless weirdoes (emphasis on the ‘harmless’; if you give any hint that you’re challenging our authority, or make any move toward trying to change the system, then the ‘harmless’ qualifier is immediately stripped from you, and you move right back into the ‘criminal’ category)”.
(That “professionals” are much more likely than anyone else to be bad actors is another fact that drastically undermines the OP’s thesis—and this blind spot is not an accident. It’s just that “professionals” simply means “the ingroup”—“one of us”. As in, “You know the score, pal! If you’re not cop, you're little people.”)
1That is not the argument I'm trying to make.
The argument I'm trying to make is that conflict resolution is hard in a particular way that approximately nothing else in running events or communities is hard; it's potentially adversarial, therefore taking the advice of people with strongly held and seemingly sensible advice can be a trap.
The bullet pointed personas are not a load bearing part of this thesis. If it would help, try dropping everything from "let's be reductive" to "what they might do about it". I think the only other part I'm directly referencing them is the parenthesis in section III about how the position of overseeing the resolution process is a position of interest for a bad actor, therefore you might even try random lot.
There is a mistake I'm trying to warn about where someone thinks conflict handling should be simple, why don't we all sit down and come up with a simple setup, and I do not think they are actually envisioning how any part of this will work in the face of someone with a mind to get away with something. There is a second mistake I'm trying to warn about where someone makes confident assertions about how conflict handling should work, and people do not notice the ulterior motive. I recognize the weirdness around me trying to point out the second mistake - as I said in the post, "Okay, but why should you trust me? Professional interest since complaint handling is part of my role, but good question and don't be satisfied by that." I am trying design a setup that would work even if I was a problem even as I was designing the setup.
I do not think the logic you're talking about is the logic I'm using. I will very cheerfully bet you my hundred bucks against your dollar that every police force does not criminalize everyone who makes any move towards trying to change the system, unless you're using some non-standard definitions of police force and criminalize. I'm going to assume that "every" is colloquial, not literal, and that paragraph is rhetoric, but even if the section in quotes was how police forces and authoritarian enforcement organizations were, I don't think their logic ends by suggesting constant vigilance against themselves.
That “professionals” are much more likely than anyone else to be bad actors is another fact that drastically undermines the OP’s thesis—and this blind spot is not an accident.
- That would undermine my thesis if my thesis was that everyone should just trust the professionals, which seems to be the thing you claim is my thesis. That's not my thesis, and bad actors turning out to be more likely "professionals"[1] than any other category would not undermine my actual thesis.
- I am not sure whether "this blind spot is not an accident" is suggesting that I'm making a good faith effort but a predictable mistake, or that I am deliberately leaving information which I know to be true and relevant out in an effort to make my argument stronger, or some third thing. Would you please clarify this? As you might imagine, if it's the second I'm going to say that's not what I'm trying to do.
- ^
I do think it would help to have a better name for this bucket, since in my contexts a lot of people aren't getting paid a regular paycheck to do this and don't have a lot of training or background. The obvious and central examples are- divorce lawyers are, CEA's community health team comes to mind as well- but a lot of the time it's a local meetup organizer in some random town that's had an issue.
The idea that situations/problems that involve conflicts, and require resolving conflicts, are more challenging than situations/problems that don’t involve conflicts, is trivial. If someone thinks that conflict handling should be simple then of course that person is an idiot. If this were all that you were saying, then it would hardly be worthy of a post.
However. In the OP you write:
Now let’s say that the bad actor is not an idiot. They have considered what and who might stop them, and what they might do about it.
It is an obvious, straightforward move to accuse whatever system is responsible for catching them of being corrupt and the people running that system to be horrible or incompetent.
But a more realistic scenario would be:
“Now let’s say that the system is already full of bad actors (as it probably is). They have considered what and who might stop them, and what they might do about it. The system will, of course, be corrupt, and the people running that system will be horrible or incompetent. It is an obvious, straightforward move to promote memes that prevent this from being rectified.”
(You might protest that sure, this happens, but you know that you are not a bad actor, right? Even if nobody else can be sure of this, you at least can! And you’re talking to people who also know that they are not bad actors. And I say that even this is false. You don’t know this. The people in your target audience—other organizers, etc.—also don’t know this about themselves. And certainly nobody else should take your (or their) word for it.)
You write:
I recognize the weirdness around me trying to point out the second mistake—as I said in the post, “Okay, but why should you trust me? Professional interest since complaint handling is part of my role, but good question and don’t be satisfied by that.”
But in fact this should read more like this:
“Okay, but why should you trust me? Good question; the answer is that you definitely shouldn’t trust me—especially since complaint handling is part of my role, so I have a professional interest, which makes me exceptionally likely to be a bad actor. Do not trust! Verify! If you can’t verify, assume treachery until proven otherwise!”
I am trying design a setup that would work even if I was a problem even as I was designing the setup.
This is an admirable goal and I applaud it. My comments are aimed precisely at this goal also. You can read them as saying “your setup does not and cannot succeed at this, so long as you take the approach described in the OP”.
I will very cheerfully bet you my hundred bucks against your dollar that every police force does not criminalize everyone who makes any move towards trying to change the system, unless you’re using some non-standard definitions of police force and criminalize.
I… didn’t claim that any police force criminalizes everyone who makes any move towards trying to change the system, so… no bet! (This is especially a nonsensical formulation since police forces do not criminalize anything; governments—in systems like that of the United States, specifically legislatures—criminalize things. But police forces have many, many tools available to them to apply differential treatment on the basis of evaluated threat level. Heck, even actual criminals can be harrassed and pressured in many ways other than arresting, indicting, and convicting them of a crime.)
I’m going to assume that “every” is colloquial, not literal, and that paragraph is rhetoric, but even if the section in quotes was how police forces and authoritarian enforcement organizations were, I don’t think their logic ends by suggesting constant vigilance against themselves.
It’s not literal only to the extent that I anticipate reasonable disagreements about what qualifies as a “police force”, “security service”, or “authoritarian enforcement organization”. Otherwise, I can’t think of any exceptions.
And their logic certainly doesn’t end by suggesting constant vigilance against themselves. Of course it doesn’t! They’re the ingroup; why should they guard against themselves?
(Although I will point out that the pattern is fractal. The cops treat “civilians” in this way, but then cops-within-the-cops organizations like “Internal Affairs” divisions and the like treat regular cops in this way. Similar patterns involving the relationship of ordinary people, spies, and counter-intelligence agencies are also well known. I don’t know offhand of any examples of this pattern going up another level, but I expect that they exist.)
I do think it would help to have a better name for this bucket, since in my contexts a lot of people aren’t getting paid a regular paycheck to do this and don’t have a lot of training or background. The obvious and central examples are- divorce lawyers are, CEA’s community health team comes to mind as well- but a lot of the time it’s a local meetup organizer in some random town that’s had an issue.
To be clear, this is how I assumed you were using the term as well.
I am not sure whether “this blind spot is not an accident” is suggesting that I’m making a good faith effort but a predictable mistake, or that I am deliberately leaving information which I know to be true and relevant out in an effort to make my argument stronger, or some third thing. Would you please clarify this? As you might imagine, if it’s the second I’m going to say that’s not what I’m trying to do.
First one.
EDIT: I really should clarify this one, sorry. It’s not the second one or any third thing—that’s what my response meant. My comment about “this blind spot is not an accident” was not meant as an accusation of deliberate bad faith argument in your post, in other words—nothing like that!
But your first interpretation isn’t quite right either, because it takes an entirely too mistake-theoretic approach. It would be more correct to say something like: the blind spot in question is a predictable consequence of the logic and incentives of the situation, such that it might be that your intentions are pure, or it might be that your intentions are not pure, but either way the result is ultimately the same, and the distinction usually moot in any case (due to self-deception).
(I am happy to assume perfectly good intentions on the part of actual-you, the actual person I am talking to, for the purposes of this and similar discussions. It’s just that we have to keep in mind the possibility of a hypothetical-you who is in the same situation and is writing the same things but who does not have good intentions.)
I disagree your scenario is more realistic.
“Now let’s say that the system is already full of bad actors (as it probably is). They have considered what and who might stop them, and what they might do about it. The system will, of course, be corrupt, and the people running that system will be horrible or incompetent. It is an obvious, straightforward move to promote memes that prevent this from being rectified.”
I think that happens sometimes, and higher pressure scenarios are more likely to be targets for this. Most of my disagreement is that I think most people are trying to do the right thing; dealing with occasional bad actors who are outnumbered is easier than dealing with lots of bad actors who outnumber everyone else. A system that works for the latter I'd expect to work for the former though.
But in fact this should read more like this:
“Okay, but why should you trust me? Good question; the answer is that you definitely shouldn’t trust me—especially since complaint handling is part of my role, so I have a professional interest, which makes me exceptionally likely to be a bad actor. Do not trust! Verify! If you can’t verify, assume treachery until proven otherwise!”
Hrm. I don't agree with that much emphasis and I'm not sure how much of that is interpretation. I do feel a little bit of refreshment at encountering someone with even more CONSTANT VIGILANCE than I have, it's a nice change of pace. Do you happen to work in computer security by any chance?
More seriously, I don't think the systems around me work well if they need to verify every step, so there's some spot checking and trust extended instead. Paying more attention to this over the last couple years has made me more aware of all the gaps where someone could cause problems if they had a mind to.
My comments are aimed precisely at this goal also. You can read them as saying “your setup does not and cannot succeed at this, so long as you take the approach described in the OP”.
Hrm. I have a small objection here, which is that I don't view the main post as laying out an approach for dealing with this. I said I don't have a solution. To use a chess analogy, I'm not saying "use the Italian Game opening, then look to control the centre." I'm saying "don't move f3 as a starter, and if you feel compelled to do it anyway really don't move g4, it's an embarrassing way to lose." If someone showed up in the comments and said hey, I do think there's a solution, here it is- well, I'd read their solution carefully and be happy if it turned out to be correct.
I… didn’t claim that any police force criminalizes everyone who makes any move towards trying to change the system, so… no bet!
Ah, I may have misinterpreted you. I read
The logic in the OP is easily recognizable as the logic of every police force, every security service, and every authoritarian enforcement organization. It’s the logic that says “if you’re not one of us, then you’re either a clueless normie who will unthinkingly submit to our authority, or else you’re probably a criminal—unless you can, with great effort and in the face of considerable skepticism, prove to us (and yes, the burden of proof is entirely on you) that you’re one of the rare harmless weirdoes (emphasis on the ‘harmless’; if you give any hint that you’re challenging our authority, or make any move toward trying to change the system, then the ‘harmless’ qualifier is immediately stripped from you, and you move right back into the ‘criminal’ category)”.
and the bolded parts (bolding mine) seemed to say every police force does criminalized everyone who makes any move towards trying to change the system. I. . . do see a distinction between "moved into the criminal category" and "criminalized according to the written legal code" but that does seem a thin distinction. Still, my misinterpretation.
> Would you please clarify this? As you might imagine, if it’s the second I’m going to say that’s not what I’m trying to do.
First one.
I appreciate the clarification, including the edit!
(I am happy to assume perfectly good intentions on the part of actual-you, the actual person I am talking to, for the purposes of this and similar discussions. It’s just that we have to keep in mind the possibility of a hypothetical-you who is in the same situation and is writing the same things but who does not have good intentions.)
Yep, noted and agreed. And likewise the possibility of a hypothetical-you who is trying to make sure whatever process gets used isn't going to catch them. Neither of you might go to meetups much but LessWrong moderation decisions are probably relevant. (To be clear I don't make those, I'm not a mod here and I don't even make moderation decisions on ACX comments, I can just see the same line.)
Most of my disagreement is that I think most people are trying to do the right thing; dealing with occasional bad actors who are outnumbered is easier than dealing with lots of bad actors who outnumber everyone else.
Yes, perhaps most people are trying to do the right thing, but (a) they are mostly not trying very hard, and (b) trying to do the right thing is just not anywhere close to sufficient for actually doing the right thing.
It is extremely easy to just find yourself doing the wrong thing, if you are not systematically and effectively avoiding all the things that nudge you toward doing the wrong thing. This is why I have emphasized, throughout this discussion, that I am not accusing anyone in particular of bad faith or bad character, and that not only should you trust no one, you should not even trust yourself, because “trying to do the right thing” is not sufficient even from your own perspective.
Do you happen to work in computer security by any chance?
I do not, but I will take the question as a compliment.
More seriously, I don’t think the systems around me work well if they need to verify every step, so there’s some spot checking and trust extended instead.
Yes. But there is a difference between making a considered judgment not to verify every step of some process, or not to check every instance, etc., and simply not having thought about it. (At the very least, the former decision can be revisited, re-evaluated, updated—the latter decision cannot even be acknowledged or accounted for, because it was never made in the first place!)
And, of course—as per my other comment—it may well be that the answer to “if we had to do this the ‘proper’ way, then we couldn’t do it at all” is “then you shouldn’t do it at all”.
Re: the “every police force” commentary—the whole “make any move toward trying to change the system” was mostly intended to qualify the “harmless weirdoes” scenario, not to necessarily apply to all people of any sort. (But also, the “moved into the criminal category” distinction is pretty important. But this is a tangent at this point, so let’s table it for now…)
Yep, noted and agreed. And likewise the possibility of a hypothetical-you who is trying to make sure whatever process gets used isn’t going to catch them. Neither of you might go to meetups much but LessWrong moderation decisions are probably relevant. (To be clear I don’t make those, I’m not a mod here and I don’t even make moderation decisions on ACX comments, I can just see the same line.)
FWIW, I think that approaches to conflict resolution in in-person meetups and on online forums should differ considerably, for many reasons, but certainly in large part due to the different ways in which evaluation of bad actors / problems / etc. can/must happen in those two types of contexts. I would not give the same advice to operators of a web forum as to organizers of a meetup, and I would be suspicious of anyone who insisted on applying the same approach to both contexts.
I think the thesis should be "everyone has an opinion on how conflicts should be handled that miraculously works out so they're right in any given conflict."
I think analyzing different types of actors with different goals isn't elucidating. Bad actors are explicitly self-serving; good actors are probably still a little biased and petty. Being right shouldn't be the main thing but it probably is. It's also easier to remember everyone self-serve biases than "this is one of 5 different types of people whose interest in conflict resolution benefits 5 different goal categories they might have."
It seems to me that the most robust solution is to do it the hard way: know the people involved really well, both directly and via reputation among people you also know really well--ideally by having lived with them in a small community for a few decades.