There's a vision here of what LessWrong could/should be, and what a rationalist community could/should be more generally. I want to push back against that vision, and offer a sketch of an alternative frame.
The post summarizes the vision I want to push back against as something like this:
What I really want from LessWrong is to make my own thinking better, moment to moment. To be embedded in a context that evokes clearer thinking, the way being in a library evokes whispers. To be embedded in a context that anti-evokes all those things my brain keeps trying to do, the way being in a church anti-evokes coarse language.
Now, I do think that's a great piece to have in a vision for the LessWrong or the rationalist community. But I don't think it's the central piece, at least not in my preferred vision.
What's missing? What is the central piece?
Fundamentally, the problem with this vision is that it isn't built for a high-dimensional world. In a high-dimensional world, the hard part of reaching an optimum isn't going-uphill-rather-than-downhill; it's figuring out which direction is best, out of millions of possible directions. Half the directions are marginally-good, half are marg...
I think I agree with the models here, and also want to add a complicating factor that I think impacts the relevance of this.
I think running a site like this in a fully consequentialist way is bad. When you're public and a seed of something, you want to have an easily-understandable interface with the world; you want it to be the case that other people who reason about you (of which there will be many, and who are crucial to your plan's success!) can easily reason about you. Something more like deontology or virtue ethics ("these are the rules I will follow" or "these are the virtues we will seek to embody") makes it much easier for other agents to reason about you.
And so the more that I as a mod (or the mod team in general) rely on our individual prudence or models or so on, the more difficult it becomes for users to predict what will happen, and that has costs. (I still think that it ultimately comes down to our prudence--the virtues that we're trying to embody do in fact conflict sometimes, and it's not obvious how to resolve those conflicts--but one of the things my prudence is considering are those legibility costs.)
And when we try to figure out what virtues we should embody on...
+1 to all this, and in particular I'm very strongly on board with rationality going beyond AI safety. I'm a big fan of LessWrong's current nominal mission to "accelerate intellectual progress", and when I'm thinking about making progress in a high-dimensional world, that's usually the kind of progress I'm thinking about. (... Which, in turn, is largely because intellectual/scientific/engineering progress seem to be the "directions" which matter most for everything else.)
I think your main point here is wrong.
Your analysis rests on a lot of assumptions:
1) It's possible to choose a basis which does a good job separating the slope from the level
2) Our perturbations are all small relative to the curvature of the terrain, such that we can model things as an n-dimensional plane
3) "Known" errors can be easily avoided, even in many dimensional space, such that the main remaining question is what the right answers are
4) Maintenance of higher standards doesn't help distinguish between better and worse directions.
5) Drama pushes in random directions, rather than directions selected for being important and easy to fuck up.
1) In a high dimensional space, almost all bases have the slope distributed among many basis vectors. If you can find a basis that has a basis vector pointing right down the gradient and the rest normal to it, that's great. If your bridge has one weak strut, fix it. However, there's no reason to suspect we can always or even usually do this. If you had to describe the direction of improvement from a rotting log to a nice cable stayed bridge, there's no way you could do it simply. You could name the direction "more better", but in order to act...
This is a great comment. There are some parts which I think are outright wrong (e.g. drama selects for most-important-disagreements), but for the most part it correctly identifies a bunch of shortcomings of the linear model from my comment.
I do think these shortcomings can generally be patched; the linear model is just one way to explain the core idea, and other models lead to the same place. The main idea is something like "in a high dimensional space, choosing the right places to explore is way more important than speed of exploration", and that generalizes well beyond linearity.
I'm not going to flesh that out more right now. This all deserves a better explanation than I'm currently ready to write.
Yeah, I anticipated that the "Drama is actually kinda important" bit would be somewhat controversial. I did qualify that it was selected "(if imperfectly)" :p
Most things are like "Do we buy our scratch paper from walmart or kinkos?", and there are few messes of people so bad that it'd make me want to say "Hey, I know you think what you're fighting about is important, but it's literally less important than where we buy our scratch paper, whether we name our log files .log or .txt, and literally any other random thing you can think of".
(Actually, now that I say this, I realize that it can fairly often look that way and that's why "bikeshedding" is a term. I think those are complicated by factors like "What they appear to be fighting about isn't really what they're fighting about", "Their goals aren't aligned with the goal you're measuring them relative to", and "The relevant metric isn't how well they can select on an absolute scale or relative to your ability, but relative to their own relatively meager abilities".)
In one extreme, you say "Look, you're fighting about this for a reason, it's clearly the most important thing, or at least top five, ignore anyone arguing otherwise".
In a...
If you had to describe the direction of improvement from a rotting log to a nice cable stayed bridge, there's no way you could do it simply. You could name the direction "more better", but in order to actually point at it or build a bridge, many many design choices will have to be made. In most real world problems, you need to look in many individual directions and decide whether it's an improvement or not and how far to go. Real world value is built on many "marginal" improvements.
This was an outstandingly useful mental image for me, and one I suspect I will incorporate into a lot of thoughts and explanations. Thanks.
EDIT: finished reading the rest of this, and it's tied (with Vaniver's) for my favorite comment on this post (at least as far as the object level is concerned; there are some really good comments about the discussions themselves).
Drama
I object to describing recent community discussions as "drama". Figuring out what happened within community organizations and holding them accountable is essential for us to have a functioning community. [I leave it unargued that we should have community.]
I agree that figuring out what happened and holding people/orgs accountable is important. That doesn't make the process (at least the process as it worked this time) not drama. I certainly don't think that the massive amount of attention the recent posts achieved can be attributed to thousands of people having a deeply-held passion for building effective organizations.
(I like the above and agree with most of it and am mulling and hope to be able to reply substantively, but in the meantime I wanted to highlight one little nitpick that might be more than a nitpick.)
Maybe finding and fixing organizational problems will lead to marginally more researcher time/effort on alignment, or maybe the drama itself will lead to a net loss of researcher attention to alignment. But these are both mechanisms of going marginally faster or marginally slower along the direction we're already pointed. In a high-dimensional world, that's not the sort of thing which matters much.
I think this leaves out a thing which is an important part of most people's values (mine included), which is that there's something bad about people being hurt, and there's something good about not hurting people, and that's relevant to a lot of people (me included) separate from questions of how it impacts progress on AI alignment. Like, on the alignment forum, I get subordinating people's pain/suffering/mistreatment to questions of mission progress (maybe), but I think that's not true of a more general place like LessWrong.
Put another way, I think there might be a gap between the importance you reflectively assign to the Drama, and the importance many others reflectively assign to it. A genuine values difference.
I do think that on LessWrong, even people's pain/suffering/mistreatment shouldn't trump questions of truth and accuracy, though. Shouldn't encourage us to abandon truth and accuracy.
Addendum to the quoted claim:
Maybe finding and fixing organizational problems will lead to marginally more researcher time/effort on alignment, or maybe the drama itself will lead to a net loss of researcher attention to alignment. But these are both mechanisms of going marginally faster or marginally slower along the direction we're already pointed. In a high-dimensional world, that's not the sort of thing which matters much.
... and it's also not the sort of thing which matters much for reduction of overall pain/suffering/mistreatment, even within the community. (Though it may be the sort of thing which matters a lot for public perceptions of pain/suffering/mistreatment.) This is a basic tenet of EA: the causes which elicit great public drama are not highly correlated with the causes which have lots of low-hanging fruit for improvement. Even within the rationalist community, our hardcoded lizard-brain drama instincts remain basically similar, and so I expect the same heuristic to apply: public drama is not a good predictor of the best ways to reduce pain/suffering/mistreatment within the community.
But that's a post-hoc explanation. My actual gut-level response to this comment was ...
My guess is that it's something like: Captain America makes bold claims with sharp boundaries that contain a lot of applause-light spirit, and tend to implicitly deny nuance. They are usually in the right direction, but "sidesy" and push people more toward being in disjoint armed camps.
Yup, I'm glad someone brought this up. I think the right model here is Demons in Imperfect Search. Transposons are a particularly good analogy - they're genes whose sole function is to copy-and-paste themselves into the genome. If you don't keep the transposons under control somehow, they'll multiply and quickly overrun everything else, killing the cell.
So keeping the metaphorical transposons either contained or subcritical is crucial. I think LessWrong handled that basically-successfully with respect to recent events: keeping demon threads at least contained is exactly what the frontpage policy is supposed to do, and the demon threads indeed stayed off the frontpage. It was probably supercritical for a while, but it was localized, so it died down in the end and the rest of the site and community are still basically intact.
(Again, as I mentioned in response to Ruby's comment, none of this is to say that the recent discussions didn't serve any useful functional role. Even transposons serve a functional role in some organisms. But the discussions certainly seemed to grow in a way decoupled from any plausible estimate of their usefulness.)
Important thing to note from this model: the goal with demons is just to keep them subcritical and/or contained. Pushing them down to zero doesn't add much.
Executive summary: I have no idea what you're talking about.
Standards are not really popular. Most people don't like them. Half the people here, I think, don't even see the problem that I'm trying to point at. Or they see it, but they don't see it as a problem.
I gather that you're upset about how the Leverage conversation went, and also Cancel Culture, so I assume your chief proposition is that LessWrong is canceling Geoff Anders; but you haven't actually made that case, just vaguely gestured with words.
I think that a certain kind of person is becoming less prevalent on LessWrong, and a certain other kind of person is becoming more prevalent, and while I have nothing against the other kind, I really thought LessWrong was for the first group.
What are the two kinds of persons? Really, I honestly do not know what you are claiming here. Repeat: I don't have even a foggy guess as to what your two kinds of person are. Am I "a certain other kind of person"? How can I know?
Distinguish feeling from fact.
This post has virtually no facts. It has short, punchy sentences with italics for emphasis. It is written like a hortatory sermon. Its primary tool is rhetoric. The f...
Yikes, despite Duncan's best attempts at disclaimers and clarity and ruling out what he doesn't mean, he apparently still didn't manage to communicate the thing he was gesturing at. That's unfortunate. (And also worries me whether I have understood him correctly.)
I will try to explain some of how I understand Duncan.
I have not read the first Leverage post and so cannot comment on those examples, but I have read jessicata's MIRI post.
and this still not having incorporated the extremely relevant context provided in this, and therefore still being misleading to anyone who doesn't get around to the comments, and the lack of concrete substantiation of the most radioactive parts of this, and so on and so forth.
As I understand it: This post criticized MIRI and CFAR by drawing parallels to Zoe Curzi's experience of Leverage. Having read the former but not the latter, the former seemed... not very substantive? Making vague parallels rather than object-level arguments? Merely mirroring the structure of the other post? In any case, there's a reason why the post sits at 61 karma with 171 votes and 925 comments, and that's not because it was considered uncontroversially true. Similarly, there's...
This comment was much longer in draft, but I've deleted the remainder because I don't want to seem "impatient" or "sneering". I'm just confused: You wrote all these words intending to convince people of something, but you don't specify what it is, and you don't use the tools we typically use to convince (facts, reliable sources, syllogistic reasoning, math, game theory...). Am I just not part of the intended audience? If so, who are they?
Thanks very much for taking the time to include this paragraph; it's doing precisely the good thing. It helps my brain not e.g. slide into a useless and unnecessary defensiveness or round you off to something you're not trying to convey.
I gather that you're upset about how the Leverage conversation went, and also Cancel Culture, so I assume your chief proposition is that LessWrong is canceling Geoff Anders; but you haven't actually made that case, just vaguely gestured with words.
That's not, in fact, my chief proposition. I do claim that something-like-the-mass-of-users is doing something-resembling-canceling-Leverage (such that e.g. if I were to propose porting over some specific piece of Leverage tech to LW or an EA org's internal cul...
I spent 15 minutes re-reading the thread underneath orthonormal's comment to try to put myself in your head. I think maybe I succeeded, so here goes, but from a person whose job involves persuading people, it's Not Optimal For Your Argument that I had to do this to engage with your model here, and it's potentially wasteful if I've failed at modeling you.
I read both of the comments discussed below, at the time I was following the original post and comments, but did not vote on either.
***
orthonormal P1: Anders seemed like a cult leader/wannabe based on my first impressions, and I willingly incurred social cost to communicate this to others
orthonormal P2 [which I inferred using the Principle of Charity]: Most of the time, people who immediately come across as cult leaders are trying to start a cult
Duncan P1: It's bad when LW upvotes comments with very thin epistemic rigor
Duncan P2: This comment has very thin epistemic rigor because it's based on a few brief conversations
Gloss: I don't necessarily agree with your P2. It's not robust, but nor is it thin; if true, it's one person's statement that, based on admittedly limited evidence, they had a high degree of confidence that...
Thank you for the effort! Strong upvoted.
Quick point to get out of the way: re: the comment that you thought would likely meet my standards, yes, it does; when I hovered over it I saw that I had already (weak) upvoted it.
Here's my attempt to rewrite orthonormal's first comment; what I would have said in orthonormal's shoes, if I were trying to say what I think orthonormal is trying to say.
...All right, here comes some subjective experience. I'm offering this up because it seems relevant, and it seems like we should be in wide-net data gathering mode.
I met Geoff Anders at our 2012 CFAR workshop, and my overwhelming impression was "this person wants to be a cult leader." This was based on [specific number of minutes] of conversation.
The impression stuck with me strongly enough that I felt like mentioning it maybe as many as [specific number] of times over the years since, in various conversations. I was motivated enough on this point that it actually somewhat drove a wedge between me and two increasingly-Leverage-enmeshed friends, in the mid-2010's.
I feel like this is important and relevant because it seems like yet again we're in a situation where a bunch of peo
Thanks, supposedlyfun, for pointing me to this thread.
I think it's important to distinguish my behavior in writing the comment (which was emotive rather than optimized - it would even have been in my own case's favor to point out that the 2012 workshop was a weeklong experiment with lots of unstructured time, rather than the weekend that CFAR later settled on, or to explain that his CoZE idea was to recruit teens to meddle with the other participants' CoZE) from the behavior of people upvoting the comment.
I expect that many of the upvotes were not of the form "this is a good comment on the meta level" so much as "SOMEBODY ELSE SAW THE THING ALL ALONG, I WORRIED IT WAS JUST ME".
Thanks for asking about the ITT.
I think that if I put a more measured version of myself back into that comment, it has one key difference from your version.
"Pay attention to me and people like me" is a status claim rather than a useful model.
I'd have said "pay attention to a person who incurred social costs by loudly predicting one later-confirmed bad actor, when they incur social costs by loudly predicting another".
(My denouncing of Geoff drove a wedge between me and several friends, including my then-best friend; my denouncing of the other one drove a wedge between me and my then-wife. Obviously those rifts had much to do with how I handled those relationships, but clearly it wasn't idle talk from me.)
Otherwise, I think the content of your ITT is about right.
(The emotional tone is off, even after translating from Duncan-speak to me-speak, but that may not be worth going into.)
For the record, I personally count myself 2 for 2.5 on precision. (I got a bad vibe from a third person, but didn't go around loudly making it known; and they've proven to be not a trustworthy person but not nearly as dangerous as I view the other two. I'll accordingly not name them.)
I'm going to take a stab at cruxing here.
Whether it's better for the LW community when comments explicitly state a reasonable amount of the epistemic hedging that they're doing.
Out of all the things you would have added to orthonormal's comment, the only one that I didn't read at the time as explicit or implicit in zir comment was, "Not as anything definitive, but if I do an honest scan over the past decade, I feel like I'm batting ... 3/5, maybe, with 2 more that are undecided, and the community consensus is doing more like 1/5". I agree it would be nice if people gave more information about their own calibration where available. I don't know whether it was available to orthonormal.
As for the rest, I'm sticking that at the end of this comment as a sort of appendix.
If I'm right about the crux, that is totally not in the set of Things That I Thought You Might Have Been Saying after reading the original post. Re-reading the original post now, I don't see how I could have figured out that this is what our actual disagreement was.
I notice that I am surprised that {the norm of how explicit a comment needs to be regarding its own epistemic standard} prompted you to write the original pos...
Understood: the comment-karma-disparity issue is, for you, a glaring example of a larger constellation.
Also understood: you and I have different preferences for explicitly stating underlying claims. I don't think your position is unreasonable, just that it will lead to much-longer comments possibly at the cost of clarity and engagement. Striking that balance is Hard.
I think we've drilled as far down as is productive on my concerns with the text of your post. I would like to see your follow-up post on the entire constellation, with the rigor customary here. You could definitely persuade me. I maybe was just not part of the target audience for your post.
It's bad that comments which are good along three different axes, and bad along none as far as I can see, are ranked way below comments that are much worse along those three axes and also have other flaws
I have an alternative and almost orthogonal interpretation for why the karma scores are the way they are.
Both in your orthonormal-Matt example, and now in this meta-example, the shorter original comments require less context to understand and got more upvotes, while the long meandering detail-oriented high-context responses were hardly even read by anyone.
This makes perfect sense to me - there's a maximum comment length after which I get a strong urge to just ignore / skim a comment (which I initially did with your response here; and I never took the time to read Matt's comments, though I also didn't vote on orthonormal's comment one way or another, nor vote in the jessicata post much at all), and I would be astonished if that only happened to me.
Also think about how people see these comments in the first place. Probably a significant chunk comes from people browsing the comment feed on the LW front page, and it makes perfect sense to scroll past a long sub-sub-sub-comment that mig...
First, some off the cuff impressions of matt's post (in the interest of data gathering):
In the initial thread I believe that I read the first paragraph of matt's comment, decided I would not get much out of it, and stopped reading without voting.
Upon revisiting the thread and reading matt's comment in full, I find it difficult to understand and do not believe I would be able to summarize or remember its main points now, about 15 minutes after the fact.
This seems somewhat interesting to test, so here is my summary from memory. After this I'll reread matt's post and compare what I thought it said upon first reading with what I think it says upon a second closer reading:
[person who met geoff] is making anecdotal claims about geoff's cult-leader-ish nature based on little data. People who have much more data are making contrary claims, so it is surprising that [person]'s post has so many upvotes. [commenter to person] is using deadpan in a particular way, which could mean multiple things depending on context but I lack that context. I believe that they are using it to communicate that geoff said so in a non-joking manner, but that is also hearsay.
Commentary before re-reading: I expect ...
Strong upvote for doing this process/experiment; this is outstanding and I separately appreciate the effort required.
Do other readers find my summary, particularly my more concise summary, an accurate portrayal of matt's comment? How would they react to that much more concise comment, as compared to matt's comment?
I find your summary at least within-bounds, i.e. not fully ruled out by the words on the page. I obviously had a different impression, but I don't think that it's invalid to hold the interpretations and hypotheses that you do.
I particularly like and want to upvote the fact that you're being clear and explicit about them being your interpretations and hypotheses; this is another LW-ish norm that is half-reliable and I would like to see fully reliable. Thanks for doing it.
To add one point:
When it comes to assessing whether a long comment or post is hard to read, quality and style of writing matters, too. SSC's Nonfiction Writing Advice endlessly hammers home the point of dividing text into increasingly smaller chunks, and e.g. here's one very long post by Eliezer that elicited multiple comments of the form "this was too boring to finish" (e.g. this one), some of which got alleviated merely by adding chapter breaks.
And since LW makes it trivial to add headings even to comments (e.g. I used headings here), I guess that's one more criterion for me to judge long comments by.
(One could even imagine the LW site nudging long comments towards including stuff like headings. One could imagine a good version of a prompt like this: "This comment / post is >3k chars but consists of only 3 paragraphs and uses no headings. Consider adding some level of hierarchy, e.g. via headings.")
All right, a more detailed response.
You're fighting fire with fire.
I am not fighting fire with fire. I request that you explicitly retract the assertion, given that it is both a) objectively false, and b) part of a class of utterances that are in general false far more often than they are true, and which tend to make it harder to think and see clearly in exactly the way I'm gesturing at with the OP.
Some statements that would not have been false:
"This seems to me like it's basically fighting fire with fire."
"I believe that, in practice, this ends up being fighting fire with fire."
"I'm having a hard time summing this up as anything other than 'fighting fire with fire.'"
...and I reiterate that those subtle differences make a substantial difference in people's general ability to do the collaborative truth-seeking thing, and are in many ways precisely what I'm arguing for above.
I clearly outline what I am identifying as "fire" in the above post. I have one list which is things brains do wrong, and another list which lays out some "don'ts" that roughly correspond to those problems.
I am violating none of those don'ts, and, in my post, exhibiting none of those wrongbrains. &nbs...
I would like to register that I think this is an excellent comment, and in fact caused me to downvote the grandparent where I would otherwise have neutral or upvoted. (This is not the sort of observation I would ordinarily feel the need to point out, but in this case it seemed rather appropriate to do so, given the context.)
Huh. Interesting.
I had literally the exact same experience before I read your comment dxu.
I imagine it's likely that Duncan could sort of burn out on being able to do this [1] since it's pretty thankless difficult cognitive work. [2]
But it's really insightful to watch. I do think he could potentially tune up [3] the diplomatic savvy a bit [4] since I think while his arguments are quite sound [5] I think he probably is sometimes making people feel a little bit stupid via his tone. [6]
Nevertheless, it's really fascinating to read and observe. I feel vaguely like I'm getting smarter.
###
Rigor for the hell of it [7]:
[1] Hedged hypothesis.
[2] Two-premise assertion with a slightly subjective basis, but I think a true one.
[3] Elaborated on a slightly different but related point further in my comment below to him with an example.
[4] Vague but I think acceptably so. To elaborate, I mean making one's ideas even when in disagreement with a person palatable to the person one is disagreeing with. Note: I'm aware it doesn't acknowledge the cost of doing so and running that filter. Note also: I think, with skill and practice, this can be done without sacrificing the content of the message. It is a...
Given that there is lots of "let's comment on what things about a comment are good and which things are bad" going on in this thread, I will make more explicit a thing that I would have usually left implicit:
My current sense is that this comment maybe was better to write than no comment, given the dynamics of the situation, but I think the outcome would have been better if you had waited to write your long comment. This comment felt like it kicked up the heat a bunch, and while I think that was better than just leaving things unresponded, my sense is the discussion overall would have gone better if you had just written your longer comment.
Jumping in here in what i hope is a prosocial way. I assert as hypothesis that the two of you currently disagree about what level of meta the conversation is/should-be at, and each feels that the other has an obligation to meet them at their level, and this has turned up the heat a lot.
maybe there is a more oblique angle then this currently heated one?
I'll have more to say on this in the future, but for now I just want to ramble about something.
I've been reading through some of the early General Semantics works. Partially to see if there are any gaps in my understanding they can fill, partially as a historical curiosity (how much of rationality did they have figured out, and what could they do with it?), partially because it might be good fodder for posts on LW (write a thousand posts to Rome).
And somehow a thing that keeps coming into my mind while reading them is the pre-rigorous/rigorous/post-rigorous split Terrence Tao talks about, where mathematicians start off just doing calculation and not really understanding proofs, and then they understand proofs through careful diligence, and then they intuitively understand proofs and discard many of the formalisms in their actions and speech.
Like, the early General Semantics writers pay careful attention to many things that I feel like I have intuitively incorporated; they're trying to be rigorous about scientific thinking (in the sense that they mean) in a way that I think I can be something closer to post-rigorous. Rather than this just being "I'm sloppier than they were", I think...
For me, what this resonates most clearly with is the interaction I just had with Ben Pace.
Ben was like "X"
And I was like "mostly yes to X, but also no to the implicature that I think surrounds X which is pretty bad."
And Ben was like "oh, definitely not that! Heck no! Thanks for pointing it out, but no, and also I think I'll change nothing in response to finding out that many people might draw that from X!"
And my response was basically "yeah, I don't think that Ben Pace on the ideal LessWrong should do anything different."
Because the thing Ben said was fine, and the implicature is easy to discard/get past. Like "almost certainly Ben Pace isn't trying to imply [that crap], I don't really need to feel defensive about it, I can just offhandedly say 'by the way, not [implication], and it's fine for Ben to not have ruled that out just like it's fine for Ben to not also actively rule out 'by the way, don't murder people' in every comment."
But that's because Ben and I have a high-trust, high-bandwidth thing going.
The more that LW as a whole is clean-and-trustworthy in the way that the Ben-Duncan line segment is clean-and-trustworthy, the less that those implications are f...
And maybe I should update in that direction, and just ignore a constant background shrieking.
I'm not sure about this; there is some value in teaching undergrads rigor, and you seem more motivated to than I am. And, like, I did like Logan's comment about rumor, and I think more people observing things like that sooner is better. I think my main hope with the grandparent was to check if you're thinking the rigor is the moon or the finger, or something.
Some strings of wisdom that seem related:
"You have to know the rules before you can break them."
There has to be some sense that you're riffing deliberately and not just wrong about the defaults
The ability to depart from The Standard Forms is dependent on both the level of trust and the number of bystanders who will get the wrong idea (see my and Critch's related posts, or my essay on the social motte-and-bailey).
"Level three players can't distinguish level two players from level four players."
This suggests to me a different idea on how to improve LessWrong: make an automated "basics of disagreement" test. This involves recognizing a couple of basic concepts like cruxes and common knowledge, and involves looking at some comment threads and correctly diagnosing "what's going on" in them (e.g. where are they talking past each other) and you have to notice a bunch of useful ways to intervene.
Then if you pass, your username on comments gets a little badge next to it, and your strong vote strength gets moved up to +4 (if you're not already there).
The idea is to make it clearer who is breaking the rules that they know, versus who is breaking the rules that they don't know.
This post makes me kind of uncomfortable and I feel like the locus is in... bad boundaries maybe? Maybe an orientation towards conflict, essentializing, and incentive design?
Here's an example where it jumped out at me:
Another metaphor is that of a garden.
You know what makes a garden?
Weeding.
Gardens aren't just about the thriving of the desired plants. They're also about the non-thriving of the non-desired plants.
And weeding is hard work, and it's boring, and it's tedious, and it's unsexy.
Here's another:
But gardens aren't just about the thriving of the desired plants. They're also about the non-thriving of the non-desired plants.
There's a difference between "there are many black ravens" and "we've successfully built an environment with no white ravens." There's a difference between "this place substantially rewards black ravens" and "this place does not reward white ravens; it imposes costs upon them."
Like... this is literally black and white thinking?
And why would a good and sane person ever want to impose costs on third parties ever except like in... revenge because we live in an anarchic horror world, or (better) as punishment after a wise and just proceed...
Like... this is literally black and white thinking?
Yes, because there is in fact a difference between "stuff that promotes individuals' and groups' ability to see and think clearly" and stuff that does not, and while we have nowhere near a solid grasp on it, we do know some of it really well at this point. There are some things that just do not belong in a subculture that's trying to figure out what's true.
Some things are, in fact, better than others, especially when you have a goal/value set/utility function.
And why would a good and sane person ever want to impose costs on third parties ever except like in... revenge because we live in an anarchic horror world, or (better) as punishment after a wise and just proceeding where rehabilitation would probably fail but deterrence might work?
Note the equivocation here between "I can't think of a reason why someone would want to impose costs on third parties except X" and "therefore there probably aren't non-X reasons." This is an example of the thing; it makes it harder to see the answer to the question "why?" Harder than it has to be, by making the "why" seem rhetorical, and as-if-there-couldn't-be-a-good-answer.
I'd like to impose some costs on statements like the above, because they tend to confuse things.
On epistemic grounds: The thing you should be objecting to in my mind is not the part where I said that "because I can't think of a reason for X, that implies that there might not be a reason for X".
(This isn't great reasoning, but it is the start of something coherent. (Also, it is an invitation to defend X coherently and directly. (A way you could have engaged with is by explaining why adversarial attacks on the non-desired weeds would be a good use of resources rather than just... like... living and letting live, and trying to learn from things you initially can't appreciate?)))
On human decency and normative grounds: The thing you should be objecting to is that I directly implied that you personally were might not be "sane and good" because your advice seemed to be violating ideas about conflict and economics that seem normative to me.
This accusation could also have an epistemic component (which would be an ad hominem) if I were saying "you are saying X and are not sane and good and therefore not-X". But I'm not saying this.
I'm saying that your proposed rules are bad because they request expensive actions for unclear benefits that seem likely to lead to unproductive ...
If you think I'm irrational, please enumerate the ways. Please be nuanced and detailed and unconfused. List 100 little flaws if you like.
I'm having a hard time doing this because your two comments are both full of things that seem to me to be doing exactly the fog-inducing, confusion-increasing thing. But I'm also reasonably confident that my menu of options looks like:
...
- Don't respond, and the-audience-as-a-whole, i.e. the-culture-of-LessWrong, will largely metabolize this as tacit admission that you were right, and I was unable to muster a defense because I don't have one that's grounded in truth
- Respond in brief, and the very culture that I'm saying currently isn't trying to be careful with its thinking and reasoning will round-off and strawman and project onto whatever I say. This seems even likelier than usual here in this subthread, given that your first comment does this all over the place and is getting pretty highly upvoted at this point.
- Respond at length, here but not elsewhere, and try to put more data and models out there to bridge the inferential gaps (this feels doomy/useless, though, because this is a site already full of essays detailing all of the things wrong with your comments)
- Respond at length to all such comments, even though it's easier to produce bullshit than to refute bullshit, meaning that I'm basically committing to put forth two hours of effort for every one that other people can throw at me, which is a recipe for exhaustion and demoralization and failure, and which is precisely why the OP was written. "People not d
I'd contend that a post can be "in good faith" in the sense of being a sincere attempt to communicate your actual beliefs and your actual reasons for them, while nonetheless containing harmful patterns such as logical fallacies, misleading rhetorical tricks, excessive verbosity, and low effort to understand your conversational partner. Accusing someone of perpetuating harmful dynamics doesn't necessarily imply bad faith.
In fact, I see this distinction as being central to the OP. Duncan talks about how his brain does bad things on autopilot when his focus slips, and he wants to be called on them so that he can get better at avoiding them.
(My sense is that dxu is not referring to JenniferRM's post, so much as the broader dynamic of how disagreement and engagement unfold, and what incentives that creates.)
It's often useful to have possibly false things pointed out to keep them in mind as hypotheses or even raw material for new hypotheses. When these things are confidently asserted as obviously correct, or given irredeemably faulty justifications, that doesn't diminish their value in this respect, it just creates a separate problem.
A healthy framing for this activity is to explain theories without claiming their truth or relevance. Here, judging what's true acts as a "solution" for the problem, while understanding available theories of what might plausibly be true is the phase of discussing the problem. So when others do propose solutions, do claim what's true, a useful process is to ignore that aspect at first.
Only once there is saturation, and more claims don't help new hypotheses to become thinkable, only then this becomes counterproductive and possibly mostly manipulation of popular opinion.
I liked the effort put into this comment, and found it worth reading, but disagree with it very substantially. I also think I expect it to overall have bad consequences on the discussion, mostly via something like "illusion of transparency" and "trying to force the discussion to happen that you want to happen, and making it hard for people to come in with a different frame", but am not confident.
I think the first one is sad, and something I expect would be resolved after some more rounds of comments or conversations. I don't actually really know what to do about the second one, like, on a deeper level. I feel like "people wanting to have a different type of discussion than the OP wants to have" is a common problem on LW that causes people to have bad experiences, and I would like to fix it. I have some guesses for fixes, but none that seem super promising. I am also not totally confident it's a huge problem and worth focussing on at the margin.
In light of your recent post on trying to establish a set of norms and guidelines for LessWrong (I think you accidentally posted it before it was finished, since some chunks of it were still missing, but it seemed to elaborate on things you put forth in stag hunt), it seems worthwhile to revisit this comment you made about a month ago that I commented on. In my comment I focused on the heat of your comment, and how that heat could lead to misunderstandings. In that context, I was worried that a more incisive critique would be counterproductive. Among other things, it would be increasing the heat in a conversation that I believed to be too heated. The other worries were that I expected that you would interpret the critique as an attack that needed defending, I intuited that you were feeling bad and that taking a very critical lens to your words would worsen your mood, and that this comment is going to take me a bunch of work (Author's note: I've finished writing it. It took about 6 hours to compose, although that includes some breaks). In this comment, I'm going to provide that more incisive critique.
My goal is to engender a greater degree of empathy in you when you engage with comm...
I'm glad you took the time to respond here, and there is a lot I like about this comment. In particular, I appreciate this comment for:
There are, however, some points of disagreement I'd like to raise and some possible deleterious consequences I'd like to flag.
I share the concern raised by habryka about the illusion of transparency, which may be increasing your confidence that you are interpreting the intended meaning (and intended consequences) of Jennifer's words. I'll go into (possibly too much) detail on one very short example of what you've written and how it may involve some misreading of Jennifer's comment. You quote Jennifer:
Perhaps you could explain "epistemic hygiene" to me in mechanistic detail, and show how I'm messing it up?
and respond:
Again the trap; ...
I was also confused about what you meant by epistemic hygiene when finishing the essays. Elsewhere someone asked whether they were one of the ones doing the bad thing you we...
[Obvious disclaimer: I am not Duncan, my views are not necessarily his views, etc.]
It seems to me that your comment is [doing something like] rounding off Duncan's position to [something like] conflict theory, and contrasting it to the alternative of a mistake-oriented approach. This impression mostly comes from passages like the following:
You're sad about the world. I'm sad about it too. I think a major cause is too much poking. You're saying the cause is too little poking. So I poked you. Now what?
If we really need to start banning the weeds, for sure and for true... because no one can grow, and no one can be taught, and errors in rationality are terrible signs that a person is an intrinsically terrible defector... then I might propose that you be banned?
...And obviously this is inimical to your selfish interests. Obviously you would argue against it for this reason if you shared the core frame of "people can't grow, errors are defection, ban the defectors" because you would also think that you can't grow, and I can't grow, and if we're calling for each other's banning based on "essentializing pro-conflict social logic" because we both think the other is a "weed"... well..
Thank you for this great comment. I feel bad not engaging with Duncan directly, but maybe I can engage with your model of him? :-)
I agree that Duncan wouldn't agree with my restatement of what he might be saying.
What I attributed to him was a critical part (that I object to) of the entailment of the gestalt of his stance or frame or whatever. My hope was that his giant list of varying attributes of statements and conversational motivations could be condensed into a concept with a clean intensive definition other than a mushy conflation of "badness" and "irrational". For me these things are very very different and I'll say much more about this below.
One hope I had was that he would vigorously deny that he was advocating anything like what I mentioned by making clear that, say, he wasn't going to wander around (or have large groups of people wander around) saying "I don't like X produced by P and so let's impose costs (ie sanctions (ie punishments)) on P and on all X-like things, and if we do this search-and-punish move super hard, on literally every instance, then next time maybe we won't have to hunt rabbits, and we won't have to cringe and we won't have to feel angry at ever...
dxu:
my model of Duncan predicts that there are some people on LW whose presence here is motivated (at least significantly in part) by wanting to grow as a rationalist,
Jennifer:
I think that it is likely that neither Duncan nor I likely consider ourselves in the first category.
Duncan, in the OP, which Jennifer I guess skimmed:
What I really want from LessWrong is to make my own thinking better, moment to moment. To be embedded in a context that evokes clearer thinking, the way being in a library evokes whispers. To be embedded in a context that anti-evokes all those things my brain keeps trying to do, the way being in a church anti-evokes coarse language.
… modular sub-communities …
… a staging area and audition space for more specific and more demanding subcultures …
Here is a thng I wrote some years ago (this is a slightly cleaned up chat log, apologies for the roughness of exposition):
...There was an analogue to this in WoW as well, where, as I think I’ve mentioned, there often was such a thing as “within this raid guild, there are multiple raid groups, including some that are more ‘elite’/exclusive than the main one”; such groups usually did not use the EPGP or other allocation system of the main group, but had their own thing.
(I should note that such smaller, more elite/exclusive groups, typically skewed closer to “managed communism” than to “regulated capitalism” on the spectrum of loot systems, which I do not think is a coincidence.)
[name_redacted]: Fewer people, higher internal trust presumably.
“Higher internal trust” is true, but not where I’d locate the cause. I’d say “higher degree of sublimation of personal interest to group interest”.
[name_redacted]: Ah. … More dedicated?
Yes, and more willing to sacrifice for the good of the raid. Like, if you’re trying to maintain a raiding guild of 100 people, keep it functioning a
I quoted it rather than paraphrasing because I wanted to not put words in your mouth while you were in a potentially adversarial mood
If a person writes "I currently get A but what I really want is B"
...and then you selectively quote "I currently get A" as justification for summarizing them as being unlikely to want B...
...right after they've objected to you strawmanning and misrepresenting them left and right, and made it very clear to you that you are nowhere near passing their ITT...
...this is not "simplification."
Apologizing for "over-simplifying," under these circumstances, is a cop-out. The thing you are doing is not over-simplification. You are [not talking about simpler versions of me and my claim that abstract away some of the detail]. You are outright misrepresenting me, and in a way that's reeeaaalll hard to believe is not adversarial, at this point.
It is at best falling so far short of cooperative discourse as to not even qualify as a member of the set, and at worst deliberate disingenuousness.
If a person wholly misses you once, that's run-of-the-mill miscommunication.
If, after you point out all the ways they missed you, at length, they brush that off a...
There's a lot here, and I've put in a lot of work writing and rewriting. After failing for long enough to put things in a way that is both succinct and clear, I'm going to abandon hopes of the latter and go all in on the former. I'm going to use the minimal handles for the concepts I refer to, in a way similar to using LW jargon like "steelman" without the accompanying essays, in hopes that the terms are descriptive enough on their own. If this ends up being too opaque, I can explicate as needed later.
Here's an oversimplified model to play with:
GWS took an approach of offering proof of security and making fairly modest bids for both security and respect. As a result, the message was accepted, but it was fairly restrained in what it attempted to communicate. Fo...
And why would a good and sane person ever want to impose costs on third parties ever except like in... revenge because we live in an anarchic horror world, or (better) as punishment after a wise and just proceeding where rehabilitation would probably fail but deterrence might work?
This paragraph sounds to me like when you say "costs" you are actually thinking of "punishments", with an implication of moral wrongdoing. I'm uncertain that Duncan intended that implication (and if he did, I'd like to request that both of you use the more specific term).
If you continue to endorse the quoted paragraph as applied to "costs" that are not necessarily punishments, then I further contend that costs are useful in several scenarios where there is no implication of wrongdoing:
The SUPER obvious example is markets, where costs are used as a mechanism to control resource allocation.
Another common scenario is as a filter for seriousness. Suppose you are holding a contest where you will award a prize to the best foozle. If entry in the contest is free of charge, you might get a lot of entries from amateur foozle artists who know that they have negligible chance of winning...
Please don't make this place worse again by caring about points for reasons other than making comments occur in the right order on the page.
I wish this statement explaining what goal your advice is designed to optimize had appeared at the top of the advice, rather than the bottom.
My current world-model predicts that this is not what most people believe points are for, and that getting people to treat points in this way would require a high-effort coordinated push, probably involving radical changes to the UI to create cognitive distance from how points are used on other sites.
Specifically, I think the way most people actually use points is as a prosthetic for nonverbal politics; they are the digital equivalent of shooting someone a smile or a glower. Smiles/glowers, in turn, are a way of informing the speaker that they are gaining/losing social capital, and informing bystanders that they could potentially gain/lose social capital depending on which side of the issue they support.
My model says this is a low-level human instinctive social behavior, with the result that it is very easy to make people behave this way, but simultaneously very hard for most people to explain ...
Like... this is literally black and white thinking?
This is written in a way that seems to imply that if it is black and white thinking that would be bad. It also doesn't read as a question despite having a question mark.
People whos neurotype makes them default to black and white thinking can get really good when a concept does apply or doesn't apply. It has strengths and weaknesses. You are taking the attitude that it is widely known for its weaknesses. Demonstrating what is being glossed over or what kinds of things would be missed by it. I guess later in the post there are descipriton how other stances have it better but I think failure analysis on what is worse for the strongly dualistic view is sorely needed.
One possible source of uncomfortableness if if the typical mind assumptions doesn't hold because the interaction target mind is atypical.
Phrasings like "And why would a good and sane person ever [...]" seem to prepare to mark individuals for rejection. And again it has a question word but doesn't read like a question.
I am worried about a mechanic where people recognised as deviant are categorised as undesirable.
I do share worry about a distinction between "in" and "out" leads to a very large out group and actions that are as large as possible instead of proprotioned to neccesity or level of guarantee.
"Black and white thinking" is another name for a reasonably well defined cognitive tendency that often occurs in proximity to reasonably common mental problems.
Part of the reason "the fallacy of gray" is a thing that happens is that advice like that can be a useful and healthy thing for people who are genuinely not thinking in a great way.
Adding gray to the palette can be a helpful baby step in actual practice.
Then very very similar words to this helpful advice can also be used to "merely score debate points" on people who have a point about "X is good and Y is bad". This is where the "fallacy" occurs... but I don't think the fallacy would occur if it didn't have the "plausible cover" that arises from the helpful version.
A typical fallacy of gray says something like "everything is gray, therefore lets take no action and stop worrying about this stuff entirely".
One possible difference, that distinguishes "better gray" from "worse gray" is whether you're advocating for fewer than 2 or more than 2 categories.
Compare: "instead of two categories (black and white), how about more than two categories (black and white and gray), or maybe even five (pure black, dark gray, gray, ...
An important note, that I couldn't manage to fold into the essay proper:
I have direct knowledge of at least three people who would like to say positive things about their experience at Leverage Research, but feel they cannot. People who are fully on board with the fact that their experiences do not erase Zoe's, but whose own very different stories are relevant to understanding what actually went wrong, so that it can be fixed for those who suffered, and prevented in the future.
And at least these three people are not speaking up, because the current state of affairs is such that they feel they can't do so without committing social suicide.
This is a fact about them, not a fact about LessWrong or what would actually happen.
But it sure is damning that they feel that way, and that I can't exactly tell them that they're wrong.
Personally, I think it's clear that there was some kind of disease here, and I'd like to properly diagnose it, and that means making it possible to collect all of the relevant info, not an impoverished subset.
We were missing Zoe's subset. It's good that we have it now.
It's bad that we're still dealing with an incomplete dataset, though, and it's really bad that a lot of people seem to think that the dataset isn't meaningfully incomplete.
"three people... would like to say positive things about their experience at Leverage Research, but feel they cannot":
Oof. I appreciate you mentioning that.
(And a special note of thanks, for being willing to put down a concrete number? It helps me try to weigh it appropriately, while not compromising anonymity.)
Navigating the fact that people seem to be scared of coming forward on every side of this, is hard. I would love advice on how to shape this thread better.
If you think of something I can do to make talking about all of {the good, the bad, the neutral, the ugly, and the complicated}, easier? I can't guarantee I'll agree to it, but I really do want to hear it.
Please feel free to reach out to me on LW, anytime in the next 2 months. Not just Duncan, anyone. Offer does expire at start of January, though.
I am especially interested in concrete suggestions that improve the Pareto Frontier of reporting, here. But I'm also pretty geared up to try to steelman any private rants that get sent my way, too.
(In this context? I have already been called all of "possessed, angry, jealous, and bad with secrets." I was willing to steelman the lot, because there is a hint of truth in each, alth...
Hm... I notice I'm maybe feeling some particular pressure to personally address this one?
Because I called out the deliberate concentration of force in the other direction that happened on an earlier copy of the BayAreaHuman thread.
I am not really recanting that? I still think something "off" happened there.
But I could stand up and give a more balanced deposition.
To be clear? I do think BAH's tone was a tad aggressive. And I think there were other people in the thread, who were more aggressive than that. I think Leverage Basic Facts EA had an even more aggressive comment thread.
I also think each of the concrete factual claims BAH made, did appear to check out with at least one corner of Leverage, according to my own account-collecting (although not always at the same time).
(I also think a few of the LBFEA's wildest claims, were probably true. Exclusion of the Leverage website from Wayback Machine is definitely true*. The Slack channels characterizing each Pareto attendee as a potential recruit, seems... probably true?)
There were a lot of corners of Leverage, though. Several of them were walled off from the corners BAH talked about, or were not very near to it.
For what it's worth, ...
LW is likely currently on something like a Pareto frontier of several values, where it is difficult to promote one value better without sacrificing others. I think that this is true, and also think that this is probably what OP believes.
The above post renders one axis of that frontier particularly emotionally salient, then expresses willingness to sacrifice other axes for it.
I appreciate that the post explicitly points out that is willing to sacrifice these other axes. It nevertheless skims a little bit over what precisely might be sacrificed.
Let's name some things that might be sacrificed:
(1) LW is a place newcomers to rationality can come to ask questions, make posts, and participate in discussion, hopefully without enormous barrier to entry. Trivial inconveniences to this can have outsized effects.
(2) LW is a kind of bulletin board and coordination center for things of general interest to an actual historical communities. Trivial inconveniences to sharing such information can once again have an outsized effect.
(3) LW is a place to just generally post things of interest, including fiction, showerthoughts, and so on, to the kind of person who is interested in rationality, AI, ...
I meant a relative Pareto frontier, vis-a-vis the LW team's knowledge and resources. I think your posts on how to expand the frontier are absolutely great, and I think they (might) add to the available area within the frontier.
"If you want to suggest that OP is part of a "genre of rhetoric": make the case that it is, name it explicitly."
I mean, most of OP is about evoking emotion about community standards; deliberately evoking emotions is a standard part of rhetoric. (I don't know what genre -- ethos if you want to invoke Aristotle -- but I don't think it particularly matters.) OP explicitly says that he would like LW to be smaller -- i.e., sacrifice other values, for the value he's just evoked emotion about. I take this to just be a description of how the essay works, not a pejorative imputation of motives.
I could definitely have done better, and I too went through several drafts, and the one I posted was probably posted because I was tired of editing rather than because it was best. I have removed the sentences in the above that seem most pejorative.
Strong upvote. Thank you for writing this, it articulates the problems better than I had them in my head and enhances my focus. This deserves a longer reply, but I'm not sure if I'll get to write it today, so I'll respond with my initial thoughts.
What I really want from LessWrong is to make my own thinking better, moment to moment. To be embedded in a context that evokes clearer thinking, the way being in a library evokes whispers. To be embedded in a context that anti-evokes all those things my brain keeps trying to do, the way being in a church anti-evokes coarse language.
I want this too.
In the big, important conversations, the ones with big stakes, the ones where emotions run high—
I don’t think LessWrong, as a community, does very well in those conversations at all.
Regarding the three threads you list, I, others involved in managing of LessWrong, and leading community figures who've spoken to me are all dissatisfied with how those conversations went and believe it calls for changes in LessWrong.
Solutions I am planning or considering:
Regarding the three threads you list, I, others involved in managing of LessWrong, and leading community figures who've spoken to me are all dissatisfied with how those conversations went and believe it calls for changes in LessWrong.
I'm deeply surprised by this. If there is a consensus among the LW managers and community figures, could one of them write a post about it laying out what was dissatisfactory and what changes they feel need to be made, or at least the result they want from the changes? I know you're a highly conscientious person with too much on zir hands already, so please don't take this upon yourself.
I am also surprised by this! I think this sentence is kind of true, and am dissatisfied with the threads, but I don't feel like my take is particularly well-summarized with the above language, at least in the context of this post (like, I feel like this sentence implies a particular type of agreement with the OP that I don't think summarizes my current position very well, though I am also not totally confident I disagree with the OP).
I am in favor of experimenting more with some karma stuff, and have been encouraging people to work on that within the Lightcone team. I think there is lots of stuff we could do better, and definitely comparing us to some ideal that I have in my head, I think things definitely aren't going remotely as well as I would like them to, but I do feel like the word "dissatisfied" seems kind of wrong. I think there are individual comments that seem bad, but overall I think the conversations have been quite good, and I am mildly positively surprised by how well they have been going.
While I am not technically a "New User" in the context of the age of my account, I comment very infrequently, and I've never made a forum-level post.
I would rate my own rationality skills and knowledge at slightly above the average person but below the average active LessWrong member. While I am aware that I possess many habits and biases that reduce the quality of my written content, I have the sincere goal of becoming a better rationalist.
There are times when I am unsure whether an argument or claim that seems incorrect is flawed or if it is my reasoning that is flawed. In such cases, it seems intuitive to write a critical comment which explicitly states what I perceive to be faulty about that claim or argument and what thought processes have led to this perception. In the case that these criticisms are valid, then the discussion of the subject is improved and those who read the comment will benefit. If the criticisms are not valid, then I may be corrected by a response that points out where my reasoning went wrong, helping me avoid making such errors in the future.
Amateur rationalists like myself are probably going to make mistakes when it comes to criticism of other people's written content, even when we strive to follow community guidelines. My concern with your suggestions is that these changes may discourage users like me from creating flawed posts and comments that help us grow as rationalists.
I think there's a real danger of that, in practice.
But I've had lots of experience with "my style of moderation/my standards" being actively good for people taking their first steps toward this brand of rationalism; lots of people have explicitly reached out to me to say that e.g. my FB wall allowed them to do just those sorts of first, flawed steps.
A big part of this is "if the standards are more generally held, then there's more room for each individual bend-of-the-rules." I personally can spend more spoons responding positively and cooperatively to [a well-intentioned newcomer who's still figuring out the norms of the garden] if I'm not also feeling like it's pretty important for me to go put out fires elsewhere.
I think that smart people can hack LW norms and propagandize / pointscore / accumulate power with relative ease. I think this post is pretty much an example of that:
- a lot of time is spent gesturing / sermoning about the importance of fighting biases etc. with no particularly informative or novel content (it is after all intended to "remind people of why they care".). I personally find it difficult to engage critically with this kind of high volume and low density.
- ultimately the intent seems to be an effort to coordinate power against types of posters that Duncan doesn't like
I just don't see how most of this post is supposed to help me be more rational. The droning on makes it harder to engage as an adversary, than if the post were just "here are my terrible ideas", but it does so in an arational way.
I bring this up in part because Duncan seems to be advocating that his adherence to LW norms means he can't just propagandize etc.
If you read the OP and do not choose to let your brain project all over it, what you see is, straightforwardly, a mass of claims about how I feel, how I think, what I believe, and what I think should be the case.
...I explicitly underscore that I think
If I'm reading you correctly, it sounds like there's actually multiple disagreements you have here--a disagreement with Duncan, but also a disagreement with the current norms of LW.
My impression is primarily informed by these bits here:
I think that smart people can hack LW norms and propagandize / pointscore / accumulate power with relative ease. [...]
If people here really think you can't propagandize or bad-faith accumulate points/power while adhering to LW norms, well, I think that's bad for rationality.
Could you say more about this? In particular, assuming my reading is accurate, I'm interested in knowing (1) ways in which you think the existing norms are inadequate to the task of preventing bad-faith point-scoring, (2) whether you think it's possible to patch those issues by introducing better norms. (Incidentally, if your answer to (2) is "yes", then it's actually possible your position and Duncan's are less incompatible than it might first appear, since you might just have different solutions in mind for the same problem.)
(Of course, it's also possible you think LW's norms are irreparable, which is a fact that--if true--would still be worth drawing attention to, even if it is somewhat sad. Possibly this is all you were trying to do in the parent comment, in which case I could be reading more into what you said than is there. If that's the case, though, I'd still like to see it confirmed.)
Maybe it is good to clarify: I'm not really convinced that LW norms are particularly conducive to bad faith or psychopathic behavior. Maybe there are some patches to apply. But mostly I am concerned about naivety. LW norms aren't enough to make truth win and bullies / predators lose. If people think they are, that alone is a problem independent of possible improvements.
since you might just have different solutions in mind for the same problem.
I think that Duncan is concerned about prejudicial mobs being too effective and I am concerned about systematically preventing information about abuse from surfacing. To some extent I do just see this as a conflict based on interests -- Duncan is concerned about the threat of being mobbed and advocating tradeoffs accordingly, I'm concerned about being abused / my friends being abused and advocating tradeoffs accordingly. But to me it doesn't seem like LW is particularly afflicted by prejudicial mobs and is nonzero afflicted by abuse.
I don't think Duncan acknowledges the presence of tradeoffs here but IMO there absolutely have to be tradeoffs. To me the generally upvoted and accepted responses to jessicata's post are making a tradeo...
I like this highlighting of the tradeoffs, and have upvoted it. But:
But to me it doesn't seem like LW is particularly afflicted by prejudicial mobs and is nonzero afflicted by abuse.
... I think this is easier to say when one has never been the target of a prejudicial mob on LessWrong, and/or when one agrees with the mob and therefore doesn't think of it as prejudicial.
I've been the target of prejudicial mobbing on LessWrong. Direct experience. And yes, it impacted work and funding and life and friendships outside of the site.
propagandize / pointscore / accumulate power with relative ease
There's a way in which this is correct denotatively, even though the connotation is something I disagree with. Like, I am in fact arguing for increasing a status differential between some behaviors that I think are more appropriate for LW and others that I think are less appropriate. I'm trying at least to be up front about what those behaviors are, so that people can disagree. e.g. if you think that it's actually not a big deal to distinguish between observation and inference, because people already do a good job teasing those apart.
But yes: I wouldn't use the "power" frame, but there's a way in which, in a dance studio, there's "power" to be had in conforming to the activity of dance and dance instruction, and "less power" in the hands of people not doing those things.
an effort to coordinate power against types of posters that Duncan doesn't like
I don't think this is the case; I want to coordinate power against a class of actions. I am agnostic as to who is taking those actions, and even specifically called out that if there are people who are above the local law we should be candid about that ...
If you do happen to feel like listing a couple of underappreciated norms that you think do protect rationality, I would like that.
Brevity
Your OP is way too long (or not sufficiently indexed) for me to, without considerable strain, determine how much or how meaningfully I think this claim is true. Relatedly I don't know what you are referring to here.
All of which is to say that I spend a decent chunk of the time being the guy in the room who is most aware of the fuckery swirling around me, and therefore the guy who is most bothered by it.
I feel like this claim is being used as evidence to bolster a call to action, without being well supported. Unsupported evidence is always worse than supported, but it's especially grating in this case because providing counter-evidence can reasonably be predicted to be interpreted as a social attack, which people have inhibitions against making (and then when do they do get made, come out in really unproductive ways). I would prefer a norm where, if you're going to claim raw or relative intelligence as a reason to believe you, you need to provide at least a link to evidence.
I should note that my interest in sharing the FB posts is that they render “Duncan’s claim to rare insight” discussable. I don’t expect them to move my opinion much because we’re FB friends, and I see many of them already.
Now that you’ve acknowledged that this is fair to assess in its own right, I’d like to share my current assessment of your insight levels:
But anyway, in addition to the-corpus-of-all-of-my-essays, such as the handful I've published this month and things like In Defense of Punch Bug and It's Not What It Looks Like and Invalidating Imaginary Injury and Common Knowledge and Miasma and any number of my Admonymous answers, I also spent five minutes scrolling back through my most recent FB posts and here are the first twenty or so that seemed to be about noticing or caring about fuckery (not organized by impressiveness):
https://www.facebook.com/duncan.sabien/posts/4946318165402861
https://www.facebook.com/duncan.sabien/posts/4924756570892354
https://www.facebook.com/duncan.sabien/posts/4920978771270134
https://www.facebook.com/duncan.sabien/posts/4918223831545628
https://www.facebook.com/duncan.sabien/posts/4918136751554336
https://www.facebook.com/duncan.sabien/posts/4911478975553447
https://www.facebook.com/duncan.sabien/posts/4907166399318038
https://www.facebook.com/duncan.sabien/posts/4907029615998383
https://www.facebook.com/duncan.sabien/posts/4889568061077872
https://www.facebook.com/duncan.sabien/posts/4888310204536991?comment_id=4888448571189821
https://www.facebook.com/duncan.sabien/posts/4877212198980125?comment_id=48773...
Well, for starters, I had unfollowed him on FB by about 2016 as a result of being just continually frustrated by his relentless pessimism. So I probably missed a whole lot of what others saw as red flags.
Summary: I found this post persuasive, and only noticed after the fact that I wasn't clear on exactly what it had persuaded me of. I think it may do some of the very things it is arguing against, although my epistemic status is that I have not put in enough time to analyze it for a truly charitable reading.
Disclaimer: I don't have the time (or energy) to put in the amount of thought I'd want to put in before writing this comment. But nevertheless, my model of Duncan wants me to write this comment, so I'm posting it anyway. Feel free to ignore it if it's wrong, useless, or confusing, and I'm sorry if it's offensive or poorly thought out!
Object-level: I quite liked your two most recent posts on Concentration of Force and Stag Hunts. I liked them enough that I almost sent them to someone else saying "here's a good thing you should read!" It wasn't until I read the comment below by SupposedlyFun that I realized something slightly odd is going on and I hadn't noticed it. I really should have noticed some twinge in the back of my mind on my own, but it took someone else pointing it out for me to catch it. I think you might be guilty of the very thing you're complaining about ...
Hmm, my two-sentence summary attempt for this post would be: "In recent drama-related posts, the comment section discussion seems very soldier-mindset instead of scout-mindset, including things like up- and down-voting comments based on which "team" they support rather than soundness of reasoning, and not conceding / correcting errors when pointed out, etc. This is a failure of the LW community and we should brainstorm how to fix it."
If that's a bad summary, it might not be Duncan's fault, I kinda skimmed.
Strong upvoted for the effort it takes to write a short, concise thing. =P
I endorse this as a most-of-it summary, though I think the details matter.
I found this post persuasive, and only noticed after the fact that I wasn't clear on exactly what it had persuaded me of.
I want to affirm that this to me seems like it should be alarming to you. To me a big part of rationality is about being resilient to this phenomenon and a big part of successful rationality norms is banning the tools for producing this phenomenon.
My brain notes, in passing, that Eternal September, and the Septembers that preceded it, can be described in terms of concentration of force. If a forum has a certain culture, and a bunch of noobs come in without it (instead exhibiting some kind of "mainstream lowest-common-denominator" culture, incompatible with that of the forum)... If they come in one by one, then they'll face negative reinforcement (downvotes and/or critical comments), pushing them to either adopt the forum's culture or leave; if they arrive in a clump, then they'll be in a position to positively support each other, reducing the negative-reinforcement effect. If enough of them arrive in a group, then the forum's "immune system" may fail to stop them, and they may end up changing the forum's culture.
Today, a sudden influx of users tends to come from some big event. Like when a hugely popular site drops a link to your forum, or when there's a huge story on your forum that draws a lot of interest from outsiders. (The Leverage story is one of the latter; a story like Zoe Curzi's is fascinating to humans. I told a non-rationalist friend about it, and he said it was very juicy gossip. &n...
First of all, thank you, Duncan, for this post. I feel like it captures important perspectives that I've had, and problems that I can see and puts them together in a pretty good way. (I also share your perspective that the post Could Be Better in several ways, but I respect you not letting the perfect be the enemy of the good.)
I find myself irritated right now (bothered, not angry) that our community's primary method of highlighting quality writing is by karma-voting. It's a similar kind of feeling to living in a democracy--yes, there are lots of systems that are worse, but really? Is this really the best we can do? (No particular shade on Ruby or the Lightcone team--making things is hard and I'm certainly glad LW exists and is as good as it is.)
Like, I think I have an idea that might make things substantially better that's not terrible: make the standard signal for quality being a high price on a quality-arbitrated betting market. This is essentially applying the concept of Futarchy to internet forums (h/t ACX and Hanson). (If this is familiar to you, dear reader, feel free to skip to responses to this comment, where I talk about features of this proposal and other ideas.) Here's ...
I like this idea. It has a lot of nice attributes.
I wrote some in the past about what all the different things are that a voting/karma system on LW is trying to produce, with some thoughts on some proposals that feel a bit similar to this: https://www.lesswrong.com/posts/EQJfdqSaMcJyR5k73/habryka-s-shortform-feed?commentId=8meuqgifXhksp42sg
I guess the question is, what is the optimal amount of consensus. Where do we want to be, on the scale from Eternal September to Echo Chamber?
Seems the me that the answer depends on how much correct we are, on average. To emphasise: how much correct we actually are, not how much correct we want to be, or imagine ourselves to be.
On a website where moderators are correct about almost everything, most disagreement is a noise. (It may provide a valuable feedback on "what other people believe", but not on how things actually are.) It is okay to punish disagreement, because in the rare situations where it is correct and you notice it, you can afford the karma hit for opposing the moderators. (And hopefully the moderators are smart enough to start paying attention when a member in good standing surprisingly decides to take a karma hit.)
On a website where moderators are quite often wrong, punishing disagreement means that the community will select for people who share the same biases, or who are good at reading the room.
I believe that people are likely to overestimate how much "other reasonable people" agree with them, which is why echo chambers can happen to people who genuinely see thems...
First, I think promoting and encouraging higher standards is, if you'll pardon the idiom, doing God's work.
Thank you.
I'm so appreciative any time any member of a community looks to promote and encourage higher standards. It takes a lot of work and gets a lot of pushback and I'm always super appreciative when I see someone work at it.
Second, and on a much smaller note, if I might offer some......... stylistic feedback?
I'm only speaking here about my personal experience and heuristics. I'm not speaking for anyone else. One of my heuristics — which I darn well know isn't perfectly accurate, but it's nevertheless a heuristic I implicitly use all the time and which I know others use — is looking at language choices made when doing a quick skim of a piece as a first-pass filter of the writer's credibility.
It's often inaccurate. I know it. Still, I do it.
Your writing sometimes, when you care about an issue, seems to veer very slightly into resembling the writing of someone who is heated up about a topic in a way that leads to less productive and coherent thought.
This leads my default reaction to discounting the credibility of the message slightly.
I have to forcibly remind myse...
Some thoughts on resource bottlenecks and strategy.
There's a lot I like about the set of goals Duncan is aiming for here, and IMO the primary question is one of prioritization.
I do think some high-level things have changed since 2018-or-so. Back when I wrote Meta-tations on Moderation, the default outcome was that LW withered and died, and it was really important people move from FB to LW. Nowadays, LW seems broadly healthy, the team has more buy-in, and I think it's easier to do highly opinionated moderation more frequently for various reasons.
On the other hand, we did just recently refactor the LW team into Lightcone Infrastructure. Most of the team is now working on a broader project of "figure out the most important bottlenecks facing humanity's ability to coordinate on x-risk, and building things that fix that bottleneck" (involving lots of pivoting). Ruby is hiring more people to build more capacity on the LW team but hiring well is a slow process. And most of the plans that seem to accomplish (some version of) what Duncan is pointing to here seem really expensive.
The good news is that we're not money-constrained much these days. The biggest bottlenecked resource is team-atte...
Something that was previously seemed some-manner-of-cruxy between me and Duncan (but I'm not 100% sure about the flavor of the crux) is "LessWrong who's primary job is to be a rationality dojo" vs "LessWrong who's primary job is to output intellectual progress."
Where, certainly, there's good reason to think the Intellectual Progress machine might benefit from a rationality dojo embedded in it. But, that's just one of the ideas for how to improve rate-of-intellectual progress. And my other background models point more towards other things as being more important for that.
BUT there is a particular model-update I've had that is new, which I haven't gotten around to writing up yet. (This is less of a reply to Duncan and more to other people I've argued with over the years)
A key piece of my model is that a generative intellectual process looks very different from the finished output. It includes lots of leaps of intuition, inferential distance, etc. In order to get top-thinkers onto LW on a regular basis rather than in small private discords, it's really important for them to be able to think-out-loud without being legible at every step here. And the LW team got a lot of complaint...
Small addition: LW 1.0 made it so you had to have 10 karma before making a top-level post (maybe just on Main? I don't remember but probably you do). I think this probably matters a lot less now that new posts automatically have to be approved, and mods have to manually promote things to frontpage. But I don't know, theoretically you could gate fraught discussions like the recent ones to users above a certain karma threshold? Some of the lowest-quality comments on those posts wouldn't have happened in that case.
I think you are getting at something here, Duncan. I've become interested in the following question lately: "How should rationalists conduct themselves if their goal is to promote rationality?" Now, I understand that promoting rationality is not every rationalist's top priority, hence I stated that condition explicitly.
I've been thoroughly impressed by how Toby Ord conducts himself in his writings and interviews. He is kind, respectful, reassuring and most importantly, he doesn't engage in fear-mongering despite working on x-risks. In his EA interview, he said, "Let us not get into criticising each other for working on the second most important thing." I found this stunningly thoughtful and virtuous. I find this to be an excellent example of someone going about achieving their goals effectively.
As much as I like Dawkins and love his books, I will admit that his attitude is sometimes unhelpful towards his own goals. I recall hearing (forgot where) that before a debate on spirituality, Dawkins' interlocutor asked him to read some documents ahead of the discussion. Dawkins showed up having not read the papers and said, "I did not read your documents because I know they are wrong." [ci...
Suggestions for LW features that could shape its culture by (dis)incentivizing certain behavior (without thinking about how hard they would be to implement):
There's a certain kind of controversial post that inevitably generates meta-discussion of whether people should be allowed to post it here (most recently in this book review). Crucially, the arguments I see there are not "I don't like this" but usually "I'm afraid of what will happen when people who don't like this see it, and associate LW with it". (I found this really tedious, and would prefer a culture where we stick our own heads out and stick to "I don't like this" rather than appealing to third parties.) Also, I wish there were a way to preempt this objection, so as to not fight the same battle over and over.
Other posts are in dispute (as in, have an unusually high fraction of downvotes), like jessicata's post, but a casual reader might only see the post (and maybe its positive karma score), with all the controversy and nuance happening in the utterly impenetrable comments section.
So what might one do about that?
For what it's worth, I think some of those terrible ideas are great or close to great.
In particular:
Rob Bensinger wants me to note that he agrees.
The first one would be costly and annoying to lots of people but also time boxed and super interesting. Training ...
Nuance is the cost of precision and the bane of clarity. I think it's an error to feel positively about nuance (or something more specific like degrees of uncertainty), when it's a serious problem clogging up productive discourse, that should be burned with fire whenever it's not absolutely vital and impossible to avoid.
Uh. I want to make a nuanced response here, distinguishing the difference between "feeling positively about nuance when it's net positive and negatively when its costs exceed its benefits, and trying to distinguish between the net positive case and the net negative case, and addressing the dynamics driving each" and so forth, but your comment above makes me hesitate.
(I also think this.)
EDIT: to clarify/sort-of-summarize, for those who don't want to click through: I think there's a compelling argument to be made that much or even the majority of intellectual progress lies in the cumulative ability to make ever-finer distinctions, i.e. increasing our capacity for nuance. I think being opposed to nuance is startling, and in my current estimation it's approximately "being opposed to the project of LessWrong." Since I don't believe that Vladimir is opposed to the project of LessWrong, I declare myself confused.
The benefits of nuance are not themselves nuance. Nuance is extremely useful, but not good in itself, and the bleed-through of its usefulness into positive affect is detrimental to clarity of thought and communication.
Capacity for nuance abstracts away this problem, so might be good in itself. (It's a capacity, something instrumentally convergent. Though things useful for agents can be dangerous for humans.)
I agree with Vladimir, FWIW.
In fact, this is one of the major problems I have with—forgive me for saying so!—your own posts. They are very nuanced! But this makes them difficult, sometimes almost impossible, to understand (not to mention very long); “bane of clarity” seems exactly right to me. (Indeed, I have noticed this tendency in the writing of several members of the LW team as well, and a few others.)
You say:
I think there’s a compelling argument to be made that much or even the majority of intellectual progress lies in the cumulative ability to make ever-finer distinctions, i.e. increasing our capacity for nuance.
There is certainly something to this view. But the counterpoint is that as you make ever finer distinctions, two trends emerge:
The distinctions come to matter less and less—and yet, they impose at least constant, and often increasing, cognitive costs. But this is surely perverse! Cognitive resource expenditures should be proportional to importance/impact, otherwise you end up wasting said resources—talking, and thinking, more and more, about things that matter less and less…
The likelihood that the distinctions you are making, and the patterns you are seeing,
There is a part of Sequences which I am too lazy to find now, which goes approximately like this: "If you make five maps of the same city, and you make those maps correctly, then the maps should be the same. So if you make five maps of the same city, and you find differences between them (for example, some streets A and B intersect on one map, but run parallel on another map), it means that you made a mistake somewhere, and the maps are not as good as you wish them to be. Nonetheless, you cannot fix this mistake by merely adjusting some of the maps to fit the other ones. The sameness of the maps is a desired outcome... but it must happen naturally, as a result of all maps correctly representing the same city... not artificially, as a result of adjusting the maps to fit each other."
I get a similar feeling from some of your posts (including this one, also the punch bug). It seems to me that you care a lot about being right; and that is a good thing, and it's kinda what this community is trying to be about. And you seem strongly frustrated by people coming to conclusions dramatically different from yours; which indeed means that someone is wrong about something. And I agree that this ...
My sense is that I am disagreeing with (a set of) specific things.
The bulk update that I'm pushing for is not "switch my opinion to everything Duncan says," but "start looking for ways to make the smaller, each-nameable-in-its-own-right slips in rationality happen less often."
I don't think I'm making a meta-argument about disagreement being wrong, except insofar as I'm asserting a belief that LessWrong ought to be for a specific thing, and that, in the case where there is consensus about that thing, other things should be deprioritized. I'm not even claiming that I'm definitely right about the thing LW ought to be for! But if it's about that thing, or chooses to become so, then it needs to be less about the other thing.
I'm specifically boosting the prescriptivist point about not using the word "rational" in an inflationary way that doesn't make literal sense. Comments can be valid, explicit on their own epistemic status, true, relevant to their intended context, not making well-known mistakes, and so on and so forth, but they can't be rational, for the reason I gave, in the sense of "rational" as a property of cognitive algorithms.
I think this is a mistake
Incidentally, I like the distinction between error and mistake from linguistics, where an error is systematic or deliberatively endorsed behavior, while a mistake is intermittent behavior that's not deliberatively endorsed. That would have my comment make an error, not a mistake.
I agree that it is important to defend and concentrate the force of rationality.
However, I think the difficulty in the original context that this post is about might be that community drama is hard to address in a good way. One major constraint in understanding any topic is having a high flow of representative or unusually useful information from the topic. But for community drama, this is really hard, because it interferes in people's privacy, it involves events that happened in the past, it involves adding up many small interactions, it may be dependent on social relationships, etc..
As such, I think it's going to be really hard for anyone to prove or disprove the validity of complaints about community dynamics in online writing. The best approach I can think of right now is to just allow people to speak up about their impressions without too much burden of proof, to test if others have the same impressions. Maybe there are other, better approaches, but I don't think standards of argumentation can solve them without doing something about the information bottleneck.
Reading this post I kinda feel like you are failing to take your own advice about gardening in a certain way.
Like it feels good to call people out on their BS and get a conversation going, but also I think there's some alternative version of these two posts that's not about building up a model and trying to convince people that something is wrong that must be done about it, and instead a version that just fights the fight at the object level against particular posts, comments, etc. as part of a long slog to change the culture through direct action that others will see and emulate through a shift in the culture.
My belief here is that you can't really cause much effective, lasting change by just telling people they're doing something that sucks and is causing a problem. Rarely will they get excited about it and take up the fight. Instead you just have to fight it out, one weed at a time, until some corner of the garden is plucked and you have a small team of folks helping you with the gardening, then expanding from there.
If LW is not that place and folks don't seem to be doing the work, then maybe LW is simply not the sort of thing that can be what you'd like it to be. Heck, maybe th...
I'd like to throw out some more bad ideas, with fewer disclaimers about how terrible they are because I have less reputation to hedge against.
Inline Commenting
I very strongly endorse the point that it seems bad that someone can make bad claims in a post, which are then refuted in comments which only get read by people who get all the way to the bottom and read comments. To me the obvious (wrong) solution is to let people make inline comments. If nothing else, having a good way within comments to point to what part of the post you want to address feels like a strict win, and given that we already have pingbacks I think letting sufficiently good comments exist alongside the post would also be good. This could also be the kind of thing that a poster can enable or disable, and that a reader can toggle visibility on.
Personal Reputation
I don't have great models for how reputation should or does work on LessWrong. The second of these is testable though - I'd be curious to see what happened if prominent accounts, before commenting, flipped a coin, and in half of all cases posted through a random alt. Of course it may not be a bad thing if respected community figures get more consideration,...
On existing LW systems, and on how they shape and incentivize discussions here:
A few related thoughts: if we do push farther upon the epistemic hygiene axis, it may be worth writing a Sequence on "Living Epistemically Clean" or something along those lines, so that the standards for discussion are clear and we have a guide to upholding those standards. Specification and implementation, if you will. Such a sequence could potentially also cover, say, implementing TAPs for epistemic hygiene in the outside world. It would probably be useful to have more info on how to address your list of "things your brain tries to do" as a community.
I notice that I value some way of allowing new users to grow accustomed to writing on LW. I imagine that my first post may not be epistemically clean enough to pass the standards you're pointing towards (which isn't a point against your standards) and that others new to the site may be discouraged by, say, a ban. The help reviewing drafts that LW currently offers helps, and that's probably a useful place to focus efforts towards helping people uphold the standards you want to put up. Potentially have multiple levels of users--specific karma amounts or moderator approval would be required to comment in some areas?
Slightly meta: I noti...
Misc. thoughts on the object level:
Recap: Concentration of force is
At each relevant moment, you want to project locally superior or overwhelming force. having the most [resources] actually present, or perhaps this just means having the right [resources] pointed in the right directions.
Taking this analogy to forum moderation, this seems to mean the ability
Policing in the civilian sense of e.g., the original Metropolitan Police)
Additionally, information is needed: The ability to detect posts requiring policing, but I understand that the LW moderation team already has this ability.
One thing we are working on in the Guild of the ROSE is a sort of accreditation or ranking system, which we informally call the "belt system" because it has many but not all of the right connotations. It is possible to have expertise in how to think better and it's desirable to have a way of recognizing people who demonstrate their expertise, for a variety of reasons. Currently the ranking system is planned to be a partly based on performance within the courses we are providing, and party based in objective tests of skill ("belt tests"). But we are still experimenting with various ideas and haven't rolled it out.
I'm confused.
In the counterfactual where lesswrong had the epistemic and moderation standards you desire, what would have been the result of the three posts in question, say three days after they were first posted? Can you explain why, using the standards you elucidated here?
(If you've answered this elsewhere, I apologize).
Full disclosure: I read all three of those posts, and downvoted the third post (and only that one), influenced in part by some of the comments to that post.
The three posts would all exist.
The first one would be near zero, karmawise, and possibly slightly negative. It would include a substantial disclaimer, up front, noting and likely apologizing for the ways in which the first draft was misleading and underjustified. This would be a result of the first ten comments containing at least three highly upvoted ones pointing that out, and calling for it.
The second post would be highly upvoted; Zoe's actual writing was well in line with what I think a LWer should upvote. The comments would contain much less piling on and being real confident and rampant extrapolation; they would be focused mainly on "okay, how do we integrate this new data (which we largely take at face value and assume to be true)? What are the multiple worlds with which it is compatible? Which worlds that we previously thought possible have been ruled out by this new information?"
They would be doing a lot of split and commit, in other words. Most people in the comments there seemed to have at max a single hypothesis, and to be collecting confirmation rather than seeking falsification.
(The support for Zoe would be unchanged; that part is ...
I guess long-time lurkers/new posters like me are part of the problem(though obviously I assume most online only LW members didn't engage with a California drama post). I still think LW is a great place for discussion and just being exposed to new ideas and good feedback, but I'm probably dragging down the sanity level.
Re fear: I think the SSC situation made it clear that LW and rationalist adjacent spaces are more public than users might think, maybe people are hesitant because they don't want to get twitter blasted or show up as a snap in an NYTimes arti...
It's a Trojan horse. It's just such good-thoughts-wrapped-in-bad-forms that people give a pass to, which has the net effect of normalizing bad forms.
Should this be "bad thoughts wrapped in good forms ... normalizing bad thoughts"?
No. The content of the comment is good. The bad is that it was made in response to a comment that was not requesting a response or further elaboration or discussion (or at least not doing so explicitly; the quoted comment does not explicitly point at any part of the comment it's replying to as being such a request). My read of the situation is that person A shared their experience in a long comment, and person B attempted to shut them down / socially-punish them / defend against the comment by replying with a good statement about unhealthy dynamics, implying that person A was playing into that dynamic, without specifying how person A played into that dynamic, when it seems to me that in fact person A was not part of that dynamic and person B was defending themselves without actually saying what they're protecting nor how it's being threatened. This occurs to me as bad form, and I believe it's what Duncan is pointing at.
I commend your vision of LessWrong.
I expect that if something like it is someday achieved, it'll mostly be done the hard way through moderation, example-setting and simply trying as hard as possible to do the right thing until most people do the right thing most of the time.
But I also expect that the design of LessWrong on a software level will go a long way towards enabling, enforcing and encouraging the kinds of cultural norms you describe. There are plenty of examples of a website's culture being heavily influenced by its design choices - Twitter's 280-...
But it’s not actually doing the thing, and as far as I can tell it’s not really trying to do the thing, either—not in the way that blue-tribe Americans are actually trying to do something about racism, from pushing for institutional change all the way down to individuals taking personal responsibility for aiding strangers-in-need.
Is this really what you want? The way that blue-tribe Americans are “actually trying to do something about racism” seems to have almost no chance of actually doing anything about racism, nor even to be designed or intended to h...
Personally, I found your highlighting of that paragraph in particular useful, and believe the piece is stronger now that I've removed it, and would not mind if you went into more detail. This is a tangent/digression, but it's a relevant one, so as long as it's headlined as such I don't think it's distracting.
(This is actually one of my private "rules of engagement" and something I've got an essay in the works about: changes-of-topic are exhausting when they're not self-aware about being changes-of-topic, but if they're clearly identified then that cost goes way down.)
Meta comments on this post:
I appreciate your commitment to precision, specificity, and disclaimers, to rule out what you don't mean and to pinpoint exactly what you do mean. I aspire to clear communication, too, and it's obvious to me that you've put orders of magnitude more effort and skillpoints into it than I could ever imagine doing in my own attempts to communicate. (The best I can personally do is write in a longwinded manner, since my attempts at concise language are too easy to misunderstand.)
That said, this has downsides, too. For instance, I'm no...
point at small things as if they are important
Taking unimportant things seriously is important. It's often unknown that something is important, or known that it isn't, and that doesn't matter for the way in which it's appropriate to work on details of what's going on with it. General principles of reasoning should work well for all examples, important or not. Ignoring details is a matter of curiosity, allocating attention, it shouldn't impact how the attention that happens to fall on a topic treats it.
...general enthusiasm for even rather dull and tediou
Where bad commentary is not highly upvoted just because our monkey brains are cheering, and good commentary is not downvoted or ignored just because our monkey brains boo or are bored.
Suggestion: give our monkey brains a thing to do that lets them follow incentives while supporting (or at least not interfering with) the goal. Some ideas:
Strong upvote. Also, I first noticed these types of dynamics at a large scale in the comments on Duncan's Dragon Army proposal (I'm linking to the Medium version since the LW version seems to be gone).
That thread (the subset of it that was happening on LW 1.0) was one of the things that convinced me to build LW 2.0 (I was already working on it, but wasn't sure how much I would commit to it). Because that thread was really quite bad, and a lot of it had to do with deep site architecture things that were hard to change.
The post has a lot of things going on. I am chipping on a it somewhat one spoonful at a time.
I didn't engage with the main drama directly as I am not around there.
It feels like a story about the a stag hunting party that encountered a bear. A bit meatier but a different beast and dangerous. Then some hunters go "nope, I didn't sign up for this" or just flee in abject horror. Those that stay get mauled because a small group can't deal with a full bear.
With a regular stag hunt it elicts frustration. Repeated failures might make starvation creep closer but us...
...I spend a lot of time around people who are not as smart as me, and I also spend a lot of time around people who are as smart as me (or smarter), but who are not as conscientious, and I also spend a lot of time around people who are as smart or smarter and as conscientious or conscientiouser, but who do not have my particular pseudo-autistic special interest and have therefore not spent the better part of the past two decades enthusiastically gathering observations and spinning up models of what happens...
...
All of which is to say that I spend a decent chu
Make wildly overconfident assertions that it doesn't even believe (that it will e.g. abandon immediately if forced to make a bet).
This seems like an important failure mode that I think a ton of people, including myself, fall victim to. I can't actually think of a reference post for it though. Does it exist? Really, I think it deserves it's own tag. I recall hearing it discussed in the rationality community, but moreso in passing, not as a focal point of something.
Hire a team of well-paid moderators for a three-month high-effort experiment of responding to every bad comment with a fixed version of what a good comment making the same point would have looked like. Flood the site with training data.
What's so terrible about this idea? I imagine the main way it could go wrong is not being able to find enough people willing to do it / accidentally having too low a bar and being overwhelmed by moderators who don't know what they are doing and promote the wrong norms. But I feel like there are probably enough people o...
...Standards are not really popular. Most people don’t like them. Half the people here, I think, don’t even see the problem that I’m trying to point at. Or they see it, but they don’t see it as a problem.
I think a good chunk of LW’s current membership would leave or go quiet if we actually succeeded at ratcheting the standards up.
I don’t think that’s a bad thing. I’d like to be surrounded by people who are actually trying. And if LW isn’t going to be that place, and it knows that it isn’t, I’d like to know that, so I can go off and found it (or just give up)
On the topic of standards
On the current margin, I am interested in users taking more risks with individual comments and posts, not less. People take more risk when the successes are rewarded more, and when the failures are punished less. I generally encourage very low standards for individual comments, similar to how I have very low standards for individual word-choice or sentence structure. I want to reward or punish users for their body of contributions rather than pick each one apart and make sure it's "up to standard". (As an example, see the how this moderation notice is framed as a holistic evaluation of a user's contributions, not about a single comment.)
So yes, Said, I am broadly opposed to substantially increasing the standards applied to each individual comment or paragraph. I am much more in favor of raising the amount of reward you can get for putting in remarkable amounts of effort and contributing great insights and knowledge. I think your support of Eliezer by making readthesequences.com and your support of Gwern with the site re-design are examples of the kind of things I think will make people like Eliezer and Gwern feel like their best writing is rewarded, rather ...
Re: standards:
What you say makes sense if, and only if, the presence of “bad” content is costless.
And that condition has (at least) these prerequisites:
Everyone (or near enough) clearly sees which content is bad; everyone agrees that the content is bad, and also on what makes it bad; and thus…
… the bad content is clearly and publicly judged as such, and firmly discarded, so that…
… nobody adopts or integrates the bad ideas from the bad content, and nobody’s reasoning, models, practices, behavior, etc. is affected (negatively) by the bad content; and relatedly…
… the bad content does not “crowd out” the good content, bad ideas from it do not outcompete opposing good ideas on corresponding topics, the bad ideas in the bad content never become the consensus views on any relevant subjects, and the bad reasoning in the bad content never affects the norms for discussion (of good content, or of anything) on the site (e.g., is never viewed by newcomers, taken to be representative, and understood to be acceptable).
If, indeed, these conditions obtain, then your perspective is eminently reasonable, and your chosen policy almost certainly the right one.
But it seems very clear to m...
What you say makes sense if, and only if, the presence of “bad” content is costless.
"Iff" is far too strong. I agree that the "if" claim holds. However, I think that what Ben says also makes sense if the bad/high-variance content has costs which are less than its benefits. Demanding costlessness imposes an unnecessarily high standard on positions disagreeing with your own, I think.
Contrasting your position with Ben's, I sense a potential false dichotomy. Must it be true that either we open the floodgates and allow who-knows-what on the site in order to encourage higher-variance moves, or we sternly allow only the most well-supported reasoning? I think not. What other solutions might be available?
The first—but surely not best—to come to mind is the curation < LW review < ??? pipeline, where posts are subjected to increasing levels of scrutiny and rewarded with increasing levels of visibility. Perhaps there might be some way for people to modulate "how much they update on a post" by "the amount of scrutiny the post has received." I don't think this quite fights the corrosion you point at. But it seems like something is possible here, and in any case it seems to me too early to conclude there is only one axis of variation in responses to the situation (free-wheeling vs strict).
It sounds to me like posting on your High-Standards-Frontpage is a very high effort endeavor, an amount of effort that currently only around 3-30 posts each year have put into them. I've thought of this idea before with the name "LW Journal" or "LW Peer Review", which also had a part where it wasn't only commenters critiquing your post, but we paid a few people full-time for reviewing of the posts in this pipeline, and there was also a clear pass/failure with each submission. (Scott Garrabrant has also suggested this idea to me in the past, as a publishing place for his papers.)
I think the main requirement I see is a correspondingly larger incentive to write something that passes this bar. Else I mostly expect the same fate to befall us as with LW 1.0, where Main became increasingly effortful and unpleasant for authors to post to, such that writers like Scott Alexander moved away to writing on their personal blogs.
(I'm generally interested to hear ideas for what would be a big reward for writers to do this sort of thing. The first ones that come to my mind are "money" and "being published in physical books".)
I do think that something like this would really help the site in certain ...
Strong-upvote. I want to register that I have had disagreements with Said in the past about this, and while I am still not completely sure whether I agree with his frame, recent developments have in fact caused me to update significantly towards his view.
I suspect this is true of others as well, such that I think Said's view (as well as associated views that may differ in specifics but agree in thrust) can no longer be treated as the minority viewpoint. (They may still be the minority view, but if so I don't expect it to be a small minority anymore, where "small" might be operationalized as "less than 1 in 5 people on this site".)
There are, at the very least, three prominent examples that spring to mind of people advocating something like "higher epistemic standards on LW": Duncan, Said, and (if I might be so bold) myself. There are, moreover, a smattering of comments from less prolific commenters, most of whom seem to express agreement with Duncan's OP. I do not think this is something that should be ignored, and I think the site may benefit from some kind of poll of its userbase, just to see exactly how much consensus there is on this.
(I recognize that the LW/Lightcone team may n...
As a note on balance, I think you addressed the (predicted) costs of raising standards but not the costs of the existing standards.
I know of at least three people that I believe Ben Pace would consider high-quality potential contributors who are not writing and commenting much on LessWrong because the comments are exhausting and costly in approximately the ways I'm gesturing at.
(We have some disagreement but more overlap than disagreement.)
And I myself am quite likely to keep adding essays but am extremely unlikely to be anything other than a thought-dropper, at the current level of you-can-get-away-with-strawmanning-and-projecting-and-bullying-and-various-other-violations-of-existing-nominal-norms-in-the-comments-and-it'll-be-highly-upvoted.
I think that should be part of the weighing, too. Like, the cost of people feeling cringier is real, but so too is the cost of people who don't even bother to show up, because they don't feel safe doing so.
I also note that I strongly claim that a generally cleaner epistemic pool straightforwardly allows for more experimentation and exploration. If there's more good faith in the atmosphere (because there's less gunk causing people to...
because the comments are exhausting and costly in approximately the ways I'm gesturing at.
(We have some disagreement but more overlap than disagreement.)
As I understand Ben Pace, he's saying something like "I want people to take more risks so that we find more gold", and you're replying with something like "I think people will take more risks if we make the space more safe, by policing things like strawmanning."
It seems central to me to somehow get precise and connected to reality, like what specific rules you're suggesting policing (strawmanning? projecting? Everything in the Sequences?), and maybe look at some historic posts and comments and figure out which bits you would police and which you wouldn't. (I'm really not sure if this is in the 'overlap' space or the 'disagreement' space.)
So yes, Said, I am broadly opposed to substantially increasing the standards applied to each individual comment or paragraph. I am much more in favor of raising the amount of reward you can get for putting in remarkable amounts of effort and contributing great insights and knowledge.
After finishing writing, I did have a further note to add on where I actually think I am more open to raising standards.
As well as rewarding people more for their entire body of contributions to LessWrong, I am also more open to negatively judging people more for their entire body of contributions to LessWrong. Compare two users: one who writes a couple of 100+ karma posts per year but who also has occasional very snarky and rude comments, versus one who never writes snarky comments but always kind of doesn't understand the dialogue and muddies the waters, and produces 100 comments each year. I think the latter has far more potential to be costly for the site, and the former has the potential to be far more valuable for the site, even though the worst comments of the former are much worse than the worst comments of the latter.
I assumed this post was mostly aimed at the LW team (maybe with some opportunity for other people to weigh in). I think periodically posting posts dedicated to arguing the moderation policy should change is fine and good.
Worth noting that I've had different disagreements you with and Duncan. In both cases I think the discussion is much subtler than "increase standards: yay/nay?". It matters a lot which standards, and what they're supposed to be doing, and how different things trade off against each other.
Would the StackExchange model work? Granting ranks on the basis of productive contribution, along with privileges, ultimately recruiting moderators from the highest ranks.
Hire a team of well-paid moderators for a three-month high-effort experiment of responding to every bad comment with a fixed version of what a good comment making the same point would have looked like. Flood the site with training data.
Maybe we can start with a smaller experiment, like a group of (paid or volunteer) moderators do this for just one post? I sometimes wish that someone would point out all the flaws in my comments so I can tell what I can improve on, but I'm not sure if that won't be so unpleasant that I'd stop wanting to participate (or th...
It is not clear to me what point you're making with your examples. Have you written an object-level analysis of a failed LW conversation? I realize that doing that in the straightforward way would antagonize a lot of people, and I recognize that might not be worth it, but maybe there's some clever workaround? Perhaps you could create a role account for your dark side, post the sort of things you think are welcomed here but shouldn't be, confirm empirically that they are, then write a condemnation of those?
...All of which is to say that I spend a decent chunk of the time being the guy in the room who is most aware of the fuckery swirling around me, and therefore the guy who is most bothered by it. It's like being a native French speaker and dropping in on a high school French class in a South Carolina public school, or like being someone who just learned how to tell good kerning from bad keming. I spend a lot of time wincing, and I spend a lot of time not being able to fix The Thing That's Happening because the inferential gaps are so large that I'd have
Duncan hosted 2+ long FB threads at the time where a lot of people shared their experiences with Brent, and I think were some of the main ways that Berkeley rationalists oriented to the situation and shared information, and overall I think it was far better than the counterfactual of Duncan not having hosted those threads. I recall, but cannot easily find, Kelsey Piper also saying she was surprised how well the threads went, and I think Duncan's contributions there and framing and moderation were a substantial part of what went well.
It is public record he was among the last defenders of said abuser. Duncan will dispute this characterization but I will include the full text he posted on facebook long after everyone else figured Brent out.
Just to share my impression, I think it's false to say "long after everyone else figured Brent out". I think it's more accurate to say that it was "shortly after it became very socially risky to defend Brent", but I think a lot of people in the threads confessed to still being quite disoriented, and I think providing a defense of Brent was a positive move to happen in the dialogue, even while I think it was not the true position. I don't think Duncan punished anyone for disagreeing with him which is especially important, I think he did a pretty credible job of bringing it up as a perspective to engage with while not escalating in a fight.
For a bit more primary source, here's some Duncan quotes from his post atop the second FB thread:
I focused yesterday's conversation on what I suspected would be a minority viewpoint which might be lost in a rush to judgment. I wanted to preserve doubt, where doubt is often underpreserved, and remind people to gather as much information as they could before setting their personal opinions. I was afraid people would think that helping those who were suffering necessarily meant hurting Brent—that it was inextricably a zero-sum game.
and
I want to reorient now to something that I underweighted yesterday, and which is itself desperately important, and which I hope people will participate in with as much energy:
Are there concrete things I can do to help Persephone?
Are there concrete things I can do to help T?
Are there concrete things I can do to help other people in a similar boat?
Are there concrete things I can do to stop others from ending up in that same boat?
I also had nothing to do with the Burning Man group (have never been to Burning Man, came out to a beach in San Francisco once along with [a dozen other people also otherwise not involved with that group] to see a geodesic dome get partially assembled?) and the confidence with which that user asserts this falsehood seems relevant.
Some facts relevant to the question of whether we were close friends:
You are knowingly rude about calling people stupid.
You might be less knowingly rude when you use "autistic" as a synonym for stupid in "do not have my particular pseudo-autistic special interest" and somewhat half of connotation of "in a way that could be made clear to an autistic ten-year-old."
This is not cool as it was not cool at this occasion
I do hope that you do not think that arguing for higher standard makes for an acceptable break in basic hospitality norms and I don't think you are claiming to be a cool person above the law.
I was not using autistic as a synonym for stupid, and literally never have. I do not think that autistic is a synonym for stupid.
I (extremely) agree with you that doing so is and would be rude, bad, unwelcoming, and a violation of basic hospitality norms.
Your reply indicates that I should do a little more work to show that what I was gesturing at with those comments was that "understandable to an autistic ten-year-old" seems, to me, to be a positive standard. Your reply is evidence that I'm failing to rule out the interpretation "stupid."
I have been a teacher for many decades, and have taught about half a dozen literal actual autistic ten-year-olds. My fiancé is also autistic, and writes a blog for autists trying to navigate modern society. I wrote a 650,000 word fanfic prominently featuring a twelve-year-old autistic OC as one of the protagonists. I believe that I am on the spectrum, and use the term pseudo-autistic special interest literally and denotatively.
In many ways, "autistic ten-year-old" is my central archetype model of what it is to be human. I believe that a culture which presents a literal actual autistic ten-year-old with all sorts ...
I am not the author of the original post, and as such I am rather freer in my ability to express pushback to criticism of said post, without invoking social friction-costs like "But you might only be saying that to defend yourself", etc.
So, to wit: I think you are mostly-to-entirely mistaken in your construal of the sentences in question. I do not believe the sentences in question carry even the slightest implication that the word "autistic" is synonymous with, evocative of, or otherwise associated with the word "stupid". Moreover, since I am not the post's author, I don't have to hedge my judgement for fear of double-counting evidence; as such I will state outright that I consider your interpretation of the quoted sentences, not just mistaken, but unreasonable.
Let's take a look at the sentences in question. The first:
...I spend a lot of time around people who are not as smart as me, and I also spend a lot of time around people who are as smart as me (or smarter), but who are not as conscientious, and I also spend a lot of time around people who are as smart or smarter and as conscientious or conscientiouser, but who do not have my particular pseudo-autistic special interest and hav
Thank you for explaining your vote, I have upvoted it.
Your analysis that trying to reword "pseudo-autistic" to stupid is indeed correct that it can't really be done.
I have a different bone to pick about it and that I need to separately tell what the bone is speaks of my communication being inadequate. Fine expressions to me would be "autistic special interest" or "intense special interest". The danger there is not to dilute the meaning anything near a "hobby", special interests are a separate thing and the latter is what is went after.
The bone is more with the use of "pseudo". If you are a bit homosexual you are not "pseudohomosexual" or "pseudoheterosexual". You can be bisexual or homocurious, those are nearer to being actual things. But "pseudohomosexuality" is not a thing but a weird adhoc construction attempt. You don't do weird "its not gay if the balls are not touching" games is gay is something non-negative. You can be neurotypical, you can be autistic, but rather than being "pseudo-autistic" you have aspergers or you are a high functioning autist or you are a person on the autism spectrum.
Anyway on the other thread it has surfaced that the main motivation was to avoid self-...
This is an essay about the current state of the LessWrong community, and the broader EA/rationalist/longtermist communities that it overlaps and bridges, inspired mostly by the dynamics around these three posts. The concepts and claims laid out in Concentration of Force, which was originally written as part one of this essay, are important context for the thoughts below.
Summary/thesis, mostly cribbed from user anon03's comment below: In many high-importance and high-emotion discussions on LessWrong, the comments and vote distribution seem very soldier-mindset instead of scout-mindset, and the overall soundness and carefulness of reasoning and discourse seems to me to be much lower than baseline, which already felt a smidge too low. This seems to indicate a failure of the LW community further up the chain (i.e. is a result of a problem, not the problem itself) and I think we should put forth real effort to fix it, and I think the most likely target is something like a more-consistent embrace and enforcement of some very basic rationality discourse norms.
(And somewhere in the back of his mind was a small, small note of confusion, a sense of something wrong about that story; and it should have been a part of Harry's art to notice that tiny note, but he was distracted. For it is a sad rule that whenever you are most in need of your art as a rationalist, that is when you are most likely to forget it.)
I claim that something has gone a little bit wrong.
And as readers of many of my/other/essays/know, I claim that things going a little bit wrong is often actually quite a big problem.
I am not alone in thinking that the small scale matters. Tiny mental flinches, itty bitty little incentives, things thrown ever so slightly off course (and then never brought back). That small things often have outsized or cumulative effects is a popular view, either explicitly stated or discernible as an underlying assumption in the writings of Eliezer Yudkowsky, Nate Soares, Logan Brienne Strohl, Scott Alexander, Anna Salamon, and Andrew Critch, just to name a few.
Yet I nevertheless feel that I encounter resistance of various forms when attempting to point at small things as if they are important. Resistance rather than cooperative disagreement—impatience, dismissal, often condescension or sneering, sometimes projection and strawmanning.
This is absolutely at least in part due to my own clumsiness and confusion. A better version of me, more skilled at communication and empathy and bridging inferential gaps, would undoubtedly run into these problems less. Would better be able to recruit people's general enthusiasm for even rather dull and tedious and unsexy work, on that split-second level.
But it seems to me that I can't locate the problem entirely within myself. That there's something out there that's Actually Broken, and that it fights back, at least a little bit, when I try to point at it and fix it.
Here's to taking another shot at it.
Below is a non-exhaustive list of things which my brain will tend to do, if I don't put forth strategic effort to stop it:
I actually tend not to do these things. I do them fairly rarely, and ever more rarely as time goes on and I improve my cognitive software and add to my catalogue of mental checks and balances and discover more of my brain's loopholes and close them up one by one.
But it's work. It's hard work. It takes up a nontrivial portion of my available spoons every single day.
And more—it requires me to leave value on the table. To not-win fights that I could have won, had I been more willing to kick below the belt. There's a reason human brains slide toward those shortcuts, and it's because those shortcuts tend to work.
But the cost—
My brain does not understand the costs, most of which are distant and abstract. My brain was not evolved to understand, on an intuitive, reflexive level, things like:
All it sees is a chance to win.
"Harry," whispered Dumbledore, "phoenixes do not understand how winning a battle can lose a war." Tears were streaming down the old wizard's cheeks, dripping into his silver beard. "The battle is all they know. They are good, but not wise. That is why they choose wizards to be their masters."
My brain is a phoenix. It sees ways to win the immediate, local confrontation, and does not understand what it would be sacrificing to secure that victory.
I spend a lot of time around people who are not as smart as me.
(This is a rude statement, but it's one I'm willing to spend the points to make. It's right that we generally dock people points for rudeness; rudeness tracks a real and important set of things and our heuristics for dealing with it are actually pretty decent. I hereby acknowledge and accept the regrettable cost of my action.)
I spend a lot of time around people who are not as smart as me, and I also spend a lot of time around people who are as smart as me (or smarter), but who are not as conscientious, and I also spend a lot of time around people who are as smart or smarter and as conscientious or conscientiouser, but who do not have my particular pseudo-autistic special interest and have therefore not spent the better part of the past two decades enthusiastically gathering observations and spinning up models of what happens when you collide a bunch of monkey brains under various conditions.
(Repetition is a hell of a drug.)
All of which is to say that I spend a decent chunk of the time being the guy in the room who is most aware of the fuckery swirling around me, and therefore the guy who is most bothered by it. It's like being a native French speaker and dropping in on a high school French class in a South Carolina public school, or like being someone who just learned how to tell good kerning from bad keming. I spend a lot of time wincing, and I spend a lot of time not being able to fix The Thing That's Happening because the inferential gaps are so large that I'd have to lay down an hour's worth of context just to give the other people the capacity to notice that something is going sideways.
(Note: often, what it feels like from the inside when you are incapable of parsing some particular distinction is that the other person has a baffling and nonsensical preference between two things that are essentially indistinguishable. To someone with colorblindness, there's just no difference between those two shades. Sometimes, when you think someone is making a mountain out of a molehill, they are in fact making a mountain out of a molehill. But sometimes there's a mountain there, and it's kind of wild that you can't see it. It's wise to keep this possibility in mind.)
I don't like the fact that my brain undermines my ability to see and think clearly, if I lose focus for a minute.
I don't like the fact that my brain undermines other people's ability to see and think clearly, if I lose focus for a minute.
I don't like the fact that, much of the time, I'm all on my own to maintain focus, and keep my eye on these problems, and notice them and nip them in the bud.
I'd really like it if I were embedded in a supportive ecosystem. If there were clear, immediate, and reliable incentives for doing it right, and clear, immediate, and reliable disincentives for doing it wrong. If there were actual norms (as opposed to nominal ones, norms-in-name-only) that gave me hints and guidance and encouragement. If there were dozens or even hundreds of people around, such that I could be confident that, when I lose focus for a minute, someone else will catch me.
Catch me, and set me straight.
Because I want to be set straight.
Because I actually care about what's real, and what's true, and what's justified, and what's rational, even though my brain is only kinda-sorta halfway on board, and keeps thinking that the right thing to do is Win.
Sometimes, when people catch me, I wince, and sometimes, I get grumpy, because I'm working with a pretty crappy OS, here. But I try to get past the wince as quickly as possible, and I try to say "thank you," and I try to make it clear that I mean it, because honestly, the people that catch me are on my side. They are helping me live up to a value that I hold in my own heart, even though I don't always succeed in embodying it.
I like it when people save me from the mistakes I listed above. I genuinely like it, even if sometimes it takes my brain a moment to catch up.
I've got a handful of metaphors that are trying to triangulate something important.
One of them is "herd immunity." In particular, those nifty side-by-side time lapses that show the progression of virulent illness in populations with different rates of vaccination or immunity. The way that the badness will spread and spread and spread when only half the population is inoculated, but fizzle almost instantly when 90+% is.
If it's safe to assume that most people's brains are throwing up the bad stuff at least as often as mine does, then it seems to matter a lot how infect-able the people around you are. How quickly their immune systems kick in, before the falsehoods take root and replicate and spread.
And speaking of immune systems, another metaphor is "epistemic hygiene." There's a reason that phrase exists. It exists because washing your hands and wearing a mask and using disinfectant and coughing into your elbow makes a difference. Cleaner people get sick less, and propagate sickness less, and cleanliness is made up of a bunch of tiny, pre-emptive actions.
I have no doubt that you would be bored senseless by therapy, the same way I'm bored when I brush my teeth and wipe my ass, because the thing about repairing, maintaining, and cleaning is: it's not an adventure. There's no way to do it so wrong you might die. It's just work, and the bottom line is, some people are okay going to work, and some people—
Well, some people would rather die. Each of us gets to choose.
(There was a decent chance that there was going to be someone in the comments using the fact that this essay contains a Rick & Morty quote to delegitimize me and the point that I'm making, but then I wrote this sentence and that became a harder trick to pull off. Not impossible, though.)
Another metaphor is that of a garden.
You know what makes a garden?
Weeding.
Gardens aren't just about the thriving of the desired plants. They're also about the non-thriving of the non-desired plants.
And weeding is hard work, and it's boring, and it's tedious, and it's unsexy.
What I'm getting out of LessWrong these days is readership. It's a great place to come and share my thoughts, and have them be seen by people—smart and perceptive people, for the most part, who will take those thoughts seriously, and supply me with new thoughts in return, many of which I honestly wouldn't have ever come to on my own.
That's valuable.
But it's not what I really want from LessWrong.
What I really want from LessWrong is to make my own thinking better, moment to moment. To be embedded in a context that evokes clearer thinking, the way being in a library evokes whispers. To be embedded in a context that anti-evokes all those things my brain keeps trying to do, the way being in a church anti-evokes coarse language.
I'd like an environment that takes seriously the fact that the little things matter, and that understands that standards and principles that are only enforced 90% of the time aren't actually enforced.
I think LessWrong actually does a pretty good job of toeing the rationality line, and following its own advice, if you take the sum total of all of its conversations.
But if you look at the conversations that matter—the times when a dose of discipline is most sorely needed, and when its absence will do the most damage—
In the big, important conversations, the ones with big stakes, the ones where emotions run high—
I don't think LessWrong, as a community, does very well in those conversations at all. When the going gets tough, the number of people who are steadfastly unwilling to let their brains do the things, and steadfastly insistent that others not get away with it either feels like it dwindles to almost nothing, and as a result, the entirely predictable thing happens: people start using symmetric weapons, and they work.
(I set aside a few minutes to go grab some examples—not an exhaustive search, just a quick skim. There's the total vote count on this comment compared to these two, and the fact that it took nearly three weeks for a comment like this one to appear, and the fact that this is in negative territory, and this comment chain which I discussed in detail in another recent post, and this and its child being positive while this and this hover around zero, and this still not having incorporated the extremely relevant context provided in this, and therefore still being misleading to anyone who doesn't get around to the comments, and the lack of concrete substantiation of the most radioactive parts of this, and so on and so forth.)
To be clear: there are also many examples of the thing going well. If you count up from nothing, and just note all the places where LessWrong handled these conversations better than genpop, there are many! More, even, than what I'm highlighting as the bad stuff.
But gardens aren't just about the thriving of the desired plants. They're also about the non-thriving of the non-desired plants.
There's a difference between "there are many black ravens" and "we've successfully built an environment with no white ravens." There's a difference between "this place substantially rewards black ravens" and "this place does not reward white ravens; it imposes costs upon them." It should be possible—no, it should be easy to have a conversation about whether the incidence of white ravens has been sufficiently reduced, separate from the question of the total incidence of black ravens, and to debate what the ratio of white ravens to black ravens needs to be, and how long a white raven should hang out before being chased away, and what it would cost to do things differently, and whether that's worth it, and I notice that this very sentence is becoming pretty defensive, and is emerging in response to past experiences, and a strong expectation that my attempt at nuance and specificity is likely to fail, because the culture does not sufficiently disincentivize projection and strawmanning and misrepresentation, and so attempts-to-be-clear cannot simply be offhand but must be preemptively fortified and made proof against adversarial interpretation and geez, this kind of sucks, no?
In Concentration of Force, which was originally part one of this essay, I mention the process of evaporative cooling, and I want to ask: who is being evaporatively cooled out of LessWrong these days, and is that the feedback loop we want to set up?
I think it isn't. I think that a certain kind of person—
(one who buys that it's important to stick to the rationality 101 basics even when it's inconvenient, and that even a small percentage of slips along this axis is a pretty big deal)
—is becoming less prevalent on LessWrong, and a certain other kind of person—
(one who doesn't buy the claim that consistency-on-the-small-stuff matters a lot, and/or thinks that there are other higher goals that supersede approximately-never-letting-the-standards-slip)
—is becoming more prevalent, and while I have nothing against the latter in general, I really thought LessWrong was for the former.
Here's my vision of LessWrong:
LessWrong should be a place where rationality has reliable concentration of force.
Where rhetorical trickery does not work. Where supposition does not get mistaken for fact. Where people's words are treated as if they mean what they say, and if there seems to be another layer of implication or inference, that is immediately surfaced and made explicit so the hypothesis can be checked, rather than the assumption run with. Where we are both capable of distinguishing, and careful to distinguish, our interpretations from our observations, and our plausible hypotheses from our justified conclusions. Where we hold each other to that standard, and receive others holding us to that standard as prosocial and cooperative, because we want help holding the line. Where bad commentary is not highly upvoted just because our monkey brains are cheering, and good commentary is not downvoted or ignored just because our monkey brains boo or are bored.
Perhaps most importantly, where none of the above is left on the level of "c'mon, we all know." Where bad stuff doesn't go unmentioned because it's just assumed that everyone knows it's bad. That just results in newcomers not knowing the deal, and ultimately means the standards erode over time.
(A standard people are hesitant or embarrassed or tentative about supporting, or that isn't seen as cool or sophisticated to underline, is not one that endures for very long.)
"Professor Quirrell," said Harry gravely, "all the Muggle-raised students in Hogwarts need a safety lecture in which they are told the things so ridiculously obvious that no wizardborn would ever think to mention them. Don't cast curses if you don't know what they do, if you discover something dangerous don't tell the world about it, don't brew high-level potions without supervision in a bathroom, the reason why there are underage magic laws, all the basics."
I spend a decent chunk of my time doing stuff like upvoting comments that are mostly good, but noting in reply to them specific places in which I think they were bad or confused or norm-violating. I do this so that I don't accidentally create a social motte-and-bailey, and erode the median user's ability to tell good from bad.
This is effortful work. I wish more people pitched in, more of the time, the way this user did here and here and here and here.
In my opinion, the archetype of the Most Dangerous Comment is something like this one:
One of the things that can feel like gaslighting in a community that attracts highly scrupulous people is when posting about your interpretation of your experience is treated as a contractual obligation to defend the claims and discuss any possible misinterpretations or consequences of what is a challenging thing to write in the first place.
This is a bad comment (in context, given what it's replying to). It's the kind of thing my brain produces, when I lose focus for a minute.
But it sounds good. It makes you Feel Like You're On The Right Team as you read it, so long as you're willing to overlook the textbook strawmanning it does, of the comment it's replying to.
It's a Trojan horse. It's just such good-thoughts-wrapped-in-bad-forms that people give a pass to, which has the net effect of normalizing bad forms.
It's when the people we agree with are doing it wrong that we are most in need of standards, firmly held.
(I have a few theories about why people are abandoning or dismissing or undermining the standards, in each of a few categories. Some people, I think, believe that it's okay to take up banned weapons as long as the person you're striking at is in the outgroup. Some people seem to think that suffering provides a justification for otherwise unacceptable behavior. Some people seem to think that you can skip steps as long as you're obviously a good guy, and others seem to think that nuance and detail are themselves signs of some kind of anti-epistemic persuadery. These hypotheses do not exhaust the space of possibility.)
It is an almost trivial claim that there are not enough reasonable people in the world. There literally never will be, from the position of a group that's pushing for sanity—if the quality of thought and discourse in the general population suddenly rose to match the best of LessWrong, the best of the LessWrongers would immediately set their sights on the next high-water mark, because this sure ain't enough.
What that means is that, out there in the broader society, rationality will approximately always lose the local confrontations. Battles must be chosen with great care, and the forces of reason meticulously prepared—there will be occasional moments of serendipity when things go well, and the rare hero that successfully speaks a word of sanity and escapes unscathed, but for the most part those victories won't come by accident.
Here, though—here, within the walls of the garden—
A part of me wants to ask "what's the garden for, if not that? What precisely are the walls trying to keep out?"
In my post on moderating LessWrong, I set forth the following principle:
In no small part, the duty of the moderation team is to ensure that no LessWronger who’s trying to adhere to the site’s principles is ever alone, when standing their ground against another user (or a mob of users) who isn’t
I no longer think that's sufficient. There aren't enough moderators for reliable concentration of force. I think it's the case that LessWrongers trying to adhere to the site's principles are often alone—and furthermore, they have no real reason, given the current state of affairs, to expect not to be alone.
Sometimes, people show up. Often, if you look on a timescale of days or weeks. But not always. Not quickly. Not reliably.
(And damage done in the meantime is rarely fully repaired. If someone has broadcast falsehoods for a week and they've been strongly upvoted, it's not enough to just say "Oops, sorry, I was wrong." That comes nowhere close to fixing what they broke.)
Looking at the commentary in the threads of the last month—looking at the upvotes and the downvotes—looking at what was said, and by whom, and when—
It's not promising.
It's not promising in the sense that the people in the parking lot consider it their responsibility to stand in defense of the parent who left their kids in the car.
That they do so reliably, and enthusiastically. That they show up in force. That they show up in force because they expect to be backed up, the way that people in a city expect to be backed up if they push back against someone shouting racist slurs. That they consider themselves obligated to push back, rather than considering it not-their-problem.
It is a defining characteristic of stag hunts that when everybody actually buys in, the payoff is pretty huge.
It is also a defining characteristic of stag hunts that when critical mass fails to cohere, those who chose stag get burned, and feel cheated, and lose big.
This post is nowhere close to being a sufficient coordination mechanism to cohere a stag hunt. No one should change their behavior in response to this alone.
But it's a call for a stag hunt.
The elephant in the room, which deserves its own full section but I wasn't able to pull it together:
Standards are not really popular. Most people don't like them. Or rather, most people like them in the abstract, but chafe when they get in the way, and it's pretty rare for someone to not think that their personal exception to the standard is more justified than others' violations. Half the people here, I think, don't even see the problem that I'm trying to point at. Or they see it, but they don't see it as a problem.
I think a good chunk of LW's current membership would leave or go quiet if we actually succeeded at ratcheting the standards up.
I don't think that's a bad thing. I'd like to be surrounded by people who are actually trying. And if LW isn't going to be that place, and it knows that it isn't, I'd like to know that, so I can go off and found it (or just give up).
... because I don't have better ones, yet.
The target is "rationality has reliable concentration of force."
The current assessment is "rationality does not have reliable concentration of force."
The vector, then, is things which either increase the number of people showing up in true rationalist style, relative to those who are not, or things which increase the power of people adhering to rationalist norms, relative to those who are not.
More of the good thing, and/or less of the bad thing.
Here are some terrible ideas for moving in that direction—for either increasing or empowering the people who are interested in the idea of a rationality subculture, and decreasing or depowering those who are just here to cargo cult a little.
These are all terrible ideas.
These are all
terrible
ideas.
I'm going to say it a third time, because LessWrong is not yet a place where I can rely on my reputation for saying what I actually mean and then expect to be treated as if I meant the thing that I actually said: I recognize that these are terrible ideas.
But you have to start somewhere, if you're going to get anywhere, and I would like LessWrong to get somewhere other than where it was over the past month. To be the sort of place where doing the depressingly usual human thing doesn't pay off. Where it's more costly to do it wrong than to do it right.
Clearly, it's not going to "just happen." Clearly, we need something to riff off of.
The guiding light in front of all of those terrible ideas—the thing that each of them is a clumsy and doomed attempt to reach for—is making the thing that makes LessWrong different be "LessWrong is a place where rationality has reliable concentration of force."
Where rationality is the-thing-that-has-local-superiority-in-most-conflicts. Where the people wielding good discourse norms and good reasoning norms always outnumber the people who aren't—or, if they can't outnumber them, we at least equip them well enough that they always outgun them.
Not some crazy high-tower thing. Just the basics, consistently done.
Distinguish inference from observation.
Distinguish feeling from fact.
Expose cruxes, or acknowledge up front that you haven't found them yet, and that this is kind of a shame.
Don't weaponize motte-and-bailey equivocation.
Start from a position of charity and good faith, or explain why you can't in concrete and legible detail. Cooperate past the first apparent "defect" from your interlocutor, because people have bad days and the typical mind fallacy is a hell of a drug, as is the double illusion of transparency.
Don't respond to someone's assertion of [A] with "But [B] is abhorrent!" Don't gloss over the part where your argument depends on the assumption that [A→B].
And most importantly of all: don't actively resist the attempts of others to do these things, or to remind others to do them. Don't sneer, don't belittle, don't dismiss, don't take-it-as-an-attack. Act in the fashion of someone who wants to be reminded of such things, even when it's inconvenient or triggers a negative emotional reaction.
Until users doing the above (and similar) consistently win against users who aren't, LessWrong is going to miss out on a thing that clearly a lot of us kind of want, and kind of think might actually matter to some other pretty important goals.
Maybe that's fine. Maybe all we really need is the low-effort rough draft. Maybe the 80/20 is actually the right balance. In fact, we're honestly well past the 80/20—LessWrong is at least an 85/37 by this point.
But it's not actually doing the thing, and as far as I can tell it's not really trying to do the thing, either—not on the level of "approximately every individual feels called to put forth a little extra effort, and approximately every individual feels some personal stake when they see the standards being degraded."
Instead, as a collective, we've got one foot on the gas and the other on the brake, and that probably isn't the best strategy for any worthwhile goal. One way or another, I think we should actually make up our minds, here, and either go out and hunt stag, or split up and catch rabbits.
Author's note: this essay is not as good as I wished it would be. In particular, it's falling somewhat short of the very standard it's pulling for, in a way that I silver-line as "reaching downward across the inferential gap" but which is actually just the result of me not having the spoons to do this or this kind of analysis on each of a dozen different examples. Having spent the past six days improving it, "as good as I wish it would be" is starting to look like an asymptote, so I chose now over never.