This is an essay about the current state of the LessWrong community, and the broader EA/rationalist/longtermist communities that it overlaps and bridges, inspired mostly by the dynamics around these three posts.  The concepts and claims laid out in Concentration of Force, which was originally written as part one of this essay, are important context for the thoughts below.

Summary/thesis, mostly cribbed from user anon03's comment below: In many high-importance and high-emotion discussions on LessWrong, the comments and vote distribution seem very soldier-mindset instead of scout-mindset, and the overall soundness and carefulness of reasoning and discourse seems to me to be much lower than baseline, which already felt a smidge too low.  This seems to indicate a failure of the LW community further up the chain (i.e. is a result of a problem, not the problem itself) and I think we should put forth real effort to fix it, and I think the most likely target is something like a more-consistent embrace and enforcement of some very basic rationality discourse norms.

(And somewhere in the back of his mind was a small, small note of confusion, a sense of something wrong about that story; and it should have been a part of Harry's art to notice that tiny note, but he was distracted. For it is a sad rule that whenever you are most in need of your art as a rationalist, that is when you are most likely to forget it.)

I claim that something has gone a little bit wrong.

And as readers of many of my/other/essays/know, I claim that things going a little bit wrong is often actually quite a big problem.

I am not alone in thinking that the small scale matters.  Tiny mental flinches, itty bitty little incentives, things thrown ever so slightly off course (and then never brought back). That small things often have outsized or cumulative effects is a popular view, either explicitly stated or discernible as an underlying assumption in the writings of Eliezer Yudkowsky, Nate Soares, Logan Brienne Strohl, Scott Alexander, Anna Salamon, and Andrew Critch, just to name a few.

Yet I nevertheless feel that I encounter resistance of various forms when attempting to point at small things as if they are important.  Resistance rather than cooperative disagreement—impatience, dismissal, often condescension or sneering, sometimes projection and strawmanning.

This is absolutely at least in part due to my own clumsiness and confusion.  A better version of me, more skilled at communication and empathy and bridging inferential gaps, would undoubtedly run into these problems less.  Would better be able to recruit people's general enthusiasm for even rather dull and tedious and unsexy work, on that split-second level.

But it seems to me that I can't locate the problem entirely within myself.  That there's something out there that's Actually Broken, and that it fights back, at least a little bit, when I try to point at it and fix it.

Here's to taking another shot at it.

Below is a non-exhaustive list of things which my brain will tend to do, if I don't put forth strategic effort to stop it:

  • Make no attempt to distinguish between what it feels is true and what is reasonable to believe.
  • Make no attempt to distinguish between what it feels is good and what is actually good.
  • Make wildly overconfident assertions that it doesn't even believe (that it will e.g. abandon immediately if forced to make a bet).
  • Weaponize equivocation and maximize plausible deniability à la motte-and-bailey, squeezing the maximum amount of wiggle room out of words and phrases.  Say things that it knows will be interpreted a certain way, while knowing that they can be defended as if they meant something more innocent.
  • Neglect the difference between what things look like and what they actually are; fail to retain any skepticism on behalf of the possibility that I might be deceived by surface resemblance.
  • Treat a 70% probability of innocence and a 30% probability of guilt as a 100% chance that the person is 30% guilty (i.e. kinda guilty).
  • Wantonly project or otherwise read into people's actions and statements; evaluate those actions and statements by asking "what would have to be true inside my head, for me to output this behavior?" and then just assume that that's what's going on for them.
  • Pretend that it is speaking directly to a specific person while secretly spending the majority of its attention and optimization power on playing to some imagined larger audience.
  • Generate interventions that will make me feel better, regardless of whether or not they'll solve the problem (and regardless of whether or not there even is a real problem to be solved, versus an ungrounded anxiety/imaginary injury).

I actually tend not to do these things.  I do them fairly rarely, and ever more rarely as time goes on and I improve my cognitive software and add to my catalogue of mental checks and balances and discover more of my brain's loopholes and close them up one by one.

But it's work.  It's hard work.  It takes up a nontrivial portion of my available spoons every single day.

And more—it requires me to leave value on the table.  To not-win fights that I could have won, had I been more willing to kick below the belt.  There's a reason human brains slide toward those shortcuts, and it's because those shortcuts tend to work.

But the cost—

My brain does not understand the costs, most of which are distant and abstract.  My brain was not evolved to understand, on an intuitive, reflexive level, things like:

  • The staggeringly high potential value of the gradual accumulation of truth
  • The staggeringly high potential value of expanding my circle of cooperation
  • The staggeringly high potential value of a world in which people are not constantly subject to having their thoughts and emotions yanked around and manipulated against their will and without their knowledge or consent

All it sees is a chance to win.

"Harry," whispered Dumbledore, "phoenixes do not understand how winning a battle can lose a war." Tears were streaming down the old wizard's cheeks, dripping into his silver beard. "The battle is all they know. They are good, but not wise. That is why they choose wizards to be their masters."

My brain is a phoenix.  It sees ways to win the immediate, local confrontation, and does not understand what it would be sacrificing to secure that victory.

I spend a lot of time around people who are not as smart as me.

(This is a rude statement, but it's one I'm willing to spend the points to make.  It's right that we generally dock people points for rudeness; rudeness tracks a real and important set of things and our heuristics for dealing with it are actually pretty decent.  I hereby acknowledge and accept the regrettable cost of my action.)

I spend a lot of time around people who are not as smart as me, and I also spend a lot of time around people who are as smart as me (or smarter), but who are not as conscientious, and I also spend a lot of time around people who are as smart or smarter and as conscientious or conscientiouser, but who do not have my particular pseudo-autistic special interest and have therefore not spent the better part of the past two decades enthusiastically gathering observations and spinning up models of what happens when you collide a bunch of monkey brains under various conditions.

(Repetition is a hell of a drug.)

All of which is to say that I spend a decent chunk of the time being the guy in the room who is most aware of the fuckery swirling around me, and therefore the guy who is most bothered by it. It's like being a native French speaker and dropping in on a high school French class in a South Carolina public school, or like being someone who just learned how to tell good kerning from bad keming.  I spend a lot of time wincing, and I spend a lot of time not being able to fix The Thing That's Happening because the inferential gaps are so large that I'd have to lay down an hour's worth of context just to give the other people the capacity to notice that something is going sideways.

(Note: often, what it feels like from the inside when you are incapable of parsing some particular distinction is that the other person has a baffling and nonsensical preference between two things that are essentially indistinguishable. To someone with colorblindness, there's just no difference between those two shades.  Sometimes, when you think someone is making a mountain out of a molehill, they are in fact making a mountain out of a molehill.  But sometimes there's a mountain there, and it's kind of wild that you can't see it.  It's wise to keep this possibility in mind.)

I don't like the fact that my brain undermines my ability to see and think clearly, if I lose focus for a minute.

I don't like the fact that my brain undermines other people's ability to see and think clearly, if I lose focus for a minute.

I don't like the fact that, much of the time, I'm all on my own to maintain focus, and keep my eye on these problems, and notice them and nip them in the bud.

I'd really like it if I were embedded in a supportive ecosystem.  If there were clear, immediate, and reliable incentives for doing it right, and clear, immediate, and reliable disincentives for doing it wrong.  If there were actual norms (as opposed to nominal ones, norms-in-name-only) that gave me hints and guidance and encouragement.  If there were dozens or even hundreds of people around, such that I could be confident that, when I lose focus for a minute, someone else will catch me.

Catch me, and set me straight.

Because I want to be set straight.

Because I actually care about what's real, and what's true, and what's justified, and what's rational, even though my brain is only kinda-sorta halfway on board, and keeps thinking that the right thing to do is Win.

Sometimes, when people catch me, I wince, and sometimes, I get grumpy, because I'm working with a pretty crappy OS, here.  But I try to get past the wince as quickly as possible, and I try to say "thank you," and I try to make it clear that I mean it, because honestly, the people that catch me are on my side.  They are helping me live up to a value that I hold in my own heart, even though I don't always succeed in embodying it.

I like it when people save me from the mistakes I listed above.  I genuinely like it, even if sometimes it takes my brain a moment to catch up.

I've got a handful of metaphors that are trying to triangulate something important.

One of them is "herd immunity."  In particular, those nifty side-by-side time lapses that show the progression of virulent illness in populations with different rates of vaccination or immunity.  The way that the badness will spread and spread and spread when only half the population is inoculated, but fizzle almost instantly when 90+% is.

If it's safe to assume that most people's brains are throwing up the bad stuff at least as often as mine does, then it seems to matter a lot how infect-able the people around you are.  How quickly their immune systems kick in, before the falsehoods take root and replicate and spread.

And speaking of immune systems, another metaphor is "epistemic hygiene."  There's a reason that phrase exists.  It exists because washing your hands and wearing a mask and using disinfectant and coughing into your elbow makes a difference.  Cleaner people get sick less, and propagate sickness less, and cleanliness is made up of a bunch of tiny, pre-emptive actions.  

I have no doubt that you would be bored senseless by therapy, the same way I'm bored when I brush my teeth and wipe my ass, because the thing about repairing, maintaining, and cleaning is: it's not an adventure. There's no way to do it so wrong you might die. It's just work, and the bottom line is, some people are okay going to work, and some people—

Well, some people would rather die. Each of us gets to choose.

(There was a decent chance that there was going to be someone in the comments using the fact that this essay contains a Rick & Morty quote to delegitimize me and the point that I'm making, but then I wrote this sentence and that became a harder trick to pull off. Not impossible, though.)

Another metaphor is that of a garden.

You know what makes a garden?


Gardens aren't just about the thriving of the desired plants.  They're also about the non-thriving of the non-desired plants.

And weeding is hard work, and it's boring, and it's tedious, and it's unsexy.

What I'm getting out of LessWrong these days is readership.  It's a great place to come and share my thoughts, and have them be seen by people—smart and perceptive people, for the most part, who will take those thoughts seriously, and supply me with new thoughts in return, many of which I honestly wouldn't have ever come to on my own.

That's valuable.

But it's not what I really want from LessWrong.

What I really want from LessWrong is to make my own thinking better, moment to moment.  To be embedded in a context that evokes clearer thinking, the way being in a library evokes whispers.  To be embedded in a context that anti-evokes all those things my brain keeps trying to do, the way being in a church anti-evokes coarse language.

I'd like an environment that takes seriously the fact that the little things matter, and that understands that standards and principles that are only enforced 90% of the time aren't actually enforced.

I think LessWrong actually does a pretty good job of toeing the rationality line, and following its own advice, if you take the sum total of all of its conversations.

But if you look at the conversations that matter—the times when a dose of discipline is most sorely needed, and when its absence will do the most damage—

In the big, important conversations, the ones with big stakes, the ones where emotions run high—

I don't think LessWrong, as a community, does very well in those conversations at all. When the going gets tough, the number of people who are steadfastly unwilling to let their brains do the things, and steadfastly insistent that others not get away with it either feels like it dwindles to almost nothing, and as a result, the entirely predictable thing happens: people start using symmetric weapons, and they work.

(I set aside a few minutes to go grab some examples—not an exhaustive search, just a quick skim.  There's the total vote count on this comment compared to these two, and the fact that it took nearly three weeks for a comment like this one to appear, and the fact that this is in negative territory, and this comment chain which I discussed in detail in another recent post, and this and its child being positive while this and this hover around zero, and this still not having incorporated the extremely relevant context provided in this, and therefore still being misleading to anyone who doesn't get around to the comments, and the lack of concrete substantiation of the most radioactive parts of this, and so on and so forth.)

To be clear: there are also many examples of the thing going well.  If you count up from nothing, and just note all the places where LessWrong handled these conversations better than genpop, there are many!  More, even, than what I'm highlighting as the bad stuff.

But gardens aren't just about the thriving of the desired plants.  They're also about the non-thriving of the non-desired plants.

There's a difference between "there are many black ravens" and "we've successfully built an environment with no white ravens."  There's a difference between "this place substantially rewards black ravens" and "this place does not reward white ravens; it imposes costs upon them."  It should be possible—no, it should be easy to have a conversation about whether the incidence of white ravens has been sufficiently reduced, separate from the question of the total incidence of black ravens, and to debate what the ratio of white ravens to black ravens needs to be, and how long a white raven should hang out before being chased away, and what it would cost to do things differently, and whether that's worth it, and I notice that this very sentence is becoming pretty defensive, and is emerging in response to past experiences, and a strong expectation that my attempt at nuance and specificity is likely to fail, because the culture does not sufficiently disincentivize projection and strawmanning and misrepresentation, and so attempts-to-be-clear cannot simply be offhand but must be preemptively fortified and made proof against adversarial interpretation and geez, this kind of sucks, no?

In Concentration of Force, which was originally part one of this essay, I mention the process of evaporative cooling, and I want to ask: who is being evaporatively cooled out of LessWrong these days, and is that the feedback loop we want to set up?

I think it isn't.  I think that a certain kind of person—

(one who buys that it's important to stick to the rationality 101 basics even when it's inconvenient, and that even a small percentage of slips along this axis is a pretty big deal)

—is becoming less prevalent on LessWrong, and a certain other kind of person—

(one who doesn't buy the claim that consistency-on-the-small-stuff matters a lot, and/or thinks that there are other higher goals that supersede approximately-never-letting-the-standards-slip)

—is becoming more prevalent, and while I have nothing against the latter in general, I really thought LessWrong was for the former.

Here's my vision of LessWrong:

LessWrong should be a place where rationality has reliable concentration of force.

Where rhetorical trickery does not work.  Where supposition does not get mistaken for fact.  Where people's words are treated as if they mean what they say, and if there seems to be another layer of implication or inference, that is immediately surfaced and made explicit so the hypothesis can be checked, rather than the assumption run with.  Where we are both capable of distinguishing, and careful to distinguish, our interpretations from our observations, and our plausible hypotheses from our justified conclusions.  Where we hold each other to that standard, and receive others holding us to that standard as prosocial and cooperative, because we want help holding the line. Where bad commentary is not highly upvoted just because our monkey brains are cheering, and good commentary is not downvoted or ignored just because our monkey brains boo or are bored.

Perhaps most importantly, where none of the above is left on the level of "c'mon, we all know."  Where bad stuff doesn't go unmentioned because it's just assumed that everyone knows it's bad.  That just results in newcomers not knowing the deal, and ultimately means the standards erode over time.

(A standard people are hesitant or embarrassed or tentative about supporting, or that isn't seen as cool or sophisticated to underline, is not one that endures for very long.)

"Professor Quirrell," said Harry gravely, "all the Muggle-raised students in Hogwarts need a safety lecture in which they are told the things so ridiculously obvious that no wizardborn would ever think to mention them. Don't cast curses if you don't know what they do, if you discover something dangerous don't tell the world about it, don't brew high-level potions without supervision in a bathroom, the reason why there are underage magic laws, all the basics."

I spend a decent chunk of my time doing stuff like upvoting comments that are mostly good, but noting in reply to them specific places in which I think they were bad or confused or norm-violating.  I do this so that I don't accidentally create a social motte-and-bailey, and erode the median user's ability to tell good from bad.

This is effortful work.  I wish more people pitched in, more of the time, the way this user did here and here and here and here.

In my opinion, the archetype of the Most Dangerous Comment is something like this one:

One of the things that can feel like gaslighting in a community that attracts highly scrupulous people is when posting about your interpretation of your experience is treated as a contractual obligation to defend the claims and discuss any possible misinterpretations or consequences of what is a challenging thing to write in the first place.

This is a bad comment (in context, given what it's replying to).  It's the kind of thing my brain produces, when I lose focus for a minute.

But it sounds good.  It makes you Feel Like You're On The Right Team as you read it, so long as you're willing to overlook the textbook strawmanning it does, of the comment it's replying to.

It's a Trojan horse.  It's just such good-thoughts-wrapped-in-bad-forms that people give a pass to, which has the net effect of normalizing bad forms.

It's when the people we agree with are doing it wrong that we are most in need of standards, firmly held. 

(I have a few theories about why people are abandoning or dismissing or undermining the standards, in each of a few categories.  Some people, I think, believe that it's okay to take up banned weapons as long as the person you're striking at is in the outgroup. Some people seem to think that suffering provides a justification for otherwise unacceptable behavior.  Some people seem to think that you can skip steps as long as you're obviously a good guy, and others seem to think that nuance and detail are themselves signs of some kind of anti-epistemic persuadery.  These hypotheses do not exhaust the space of possibility.)

It is an almost trivial claim that there are not enough reasonable people in the world. There literally never will be, from the position of a group that's pushing for sanity—if the quality of thought and discourse in the general population suddenly rose to match the best of LessWrong, the best of the LessWrongers would immediately set their sights on the next high-water mark, because this sure ain't enough.

What that means is that, out there in the broader society, rationality will approximately always lose the local confrontations.  Battles must be chosen with great care, and the forces of reason meticulously prepared—there will be occasional moments of serendipity when things go well, and the rare hero that successfully speaks a word of sanity and escapes unscathed, but for the most part those victories won't come by accident.

Here, though—here, within the walls of the garden—

A part of me wants to ask "what's the garden for, if not that?  What precisely are the walls trying to keep out?"

In my post on moderating LessWrong, I set forth the following principle:

In no small part, the duty of the moderation team is to ensure that no LessWronger who’s trying to adhere to the site’s principles is ever alone, when standing their ground against another user (or a mob of users) who isn’t

I no longer think that's sufficient.  There aren't enough moderators for reliable concentration of force.  I think it's the case that LessWrongers trying to adhere to the site's principles are often alone—and furthermore, they have no real reason, given the current state of affairs, to expect not to be alone.

Sometimes, people show up.  Often, if you look on a timescale of days or weeks.  But not always.  Not quickly.  Not reliably.

(And damage done in the meantime is rarely fully repaired.  If someone has broadcast falsehoods for a week and they've been strongly upvoted, it's not enough to just say "Oops, sorry, I was wrong."  That comes nowhere close to fixing what they broke.)

Looking at the commentary in the threads of the last month—looking at the upvotes and the downvotes—looking at what was said, and by whom, and when—

It's not promising.

It's not promising in the sense that the people in the parking lot consider it their responsibility to stand in defense of the parent who left their kids in the car.

That they do so reliably, and enthusiastically.  That they show up in force.  That they show up in force because they expect to be backed up, the way that people in a city expect to be backed up if they push back against someone shouting racist slurs.  That they consider themselves obligated to push back, rather than considering it not-their-problem.

It is a defining characteristic of stag hunts that when everybody actually buys in, the payoff is pretty huge.

It is also a defining characteristic of stag hunts that when critical mass fails to cohere, those who chose stag get burned, and feel cheated, and lose big.

This post is nowhere close to being a sufficient coordination mechanism to cohere a stag hunt.  No one should change their behavior in response to this alone.

But it's a call for a stag hunt.

The elephant in the room, which deserves its own full section but I wasn't able to pull it together:

Standards are not really popular.  Most people don't like them.  Or rather, most people like them in the abstract, but chafe when they get in the way, and it's pretty rare for someone to not think that their personal exception to the standard is more justified than others' violations.  Half the people here, I think, don't even see the problem that I'm trying to point at.  Or they see it, but they don't see it as a problem.

I think a good chunk of LW's current membership would leave or go quiet if we actually succeeded at ratcheting the standards up.

I don't think that's a bad thing.  I'd like to be surrounded by people who are actually trying.  And if LW isn't going to be that place, and it knows that it isn't, I'd like to know that, so I can go off and found it (or just give up).

Terrible Ideas

... because I don't have better ones, yet. 

The target is "rationality has reliable concentration of force."

The current assessment is "rationality does not have reliable concentration of force."

The vector, then, is things which either increase the number of people showing up in true rationalist style, relative to those who are not, or things which increase the power of people adhering to rationalist norms, relative to those who are not.

More of the good thing, and/or less of the bad thing.

Here are some terrible ideas for moving in that direction—for either increasing or empowering the people who are interested in the idea of a rationality subculture, and decreasing or depowering those who are just here to cargo cult a little.

  • Publish a set of absolute user guidelines (not suggestions) and enforce them without exception instead of enforcing them like speed limits.  e.g. any violation from a new user, or any violation from an established user not retracted immediately upon pushback = automatic three-day ban.  If there are Special Cool People™ who are above the law, be explicit about that fact in a way that could be made clear to an autistic ten-year-old.
  • Create a pledge similar to (but better and more comprehensive than) this pledge, and require all users to sign in order to be able to post, comment, and vote. Alternately, make such a pledge optional, but allow users who have pledged and are living up to it some kind of greater power—larger vote strength, or the ability to flag comments as yellow or orange (or blue or green!) regardless of their popularity.
  • Hire a team of well-paid moderators for a three-month high-effort experiment of responding to every bad comment with a fixed version of what a good comment making the same point would have looked like.  Flood the site with training data.
  • Make a fork of LessWrong run by me, or some other hopeless idealist that still thinks that there might be something actually good that we can get if we actually do the thing (but not if we don't).
  • Create an anonymous account with special powers called TheCultureCurators or something, and secretly give the login credentials to a small cadre of 3-12 people with good judgment and mutual faith in one another's good judgment.  Give TheCultureCurators the ability to make upvotes and downvotes of arbitrary strength, or to add notes to any comment or post à la Google Docs, or to put a number on any comment or post that indicates what karma TheCultureCurators believe that post should have.
  • Give up, and admit that we're kinda sorta nominally about clear thinking and good discourse, but not actually/only to the extent that it's convenient and easy, because either "the community" or "some model of effectiveness" takes priority, and put that admission somewhere that an autistic ten-year-old would see it before getting the wrong idea.

These are all terrible ideas.

These are all



I'm going to say it a third time, because LessWrong is not yet a place where I can rely on my reputation for saying what I actually mean and then expect to be treated as if I meant the thing that I actually said: I recognize that these are terrible ideas.

But you have to start somewhere, if you're going to get anywhere, and I would like LessWrong to get somewhere other than where it was over the past month.  To be the sort of place where doing the depressingly usual human thing doesn't pay off.  Where it's more costly to do it wrong than to do it right.

Clearly, it's not going to "just happen."  Clearly, we need something to riff off of.

The guiding light in front of all of those terrible ideas—the thing that each of them is a clumsy and doomed attempt to reach for—is making the thing that makes LessWrong different be "LessWrong is a place where rationality has reliable concentration of force."

Where rationality is the-thing-that-has-local-superiority-in-most-conflicts.  Where the people wielding good discourse norms and good reasoning norms always outnumber the people who aren't—or, if they can't outnumber them, we at least equip them well enough that they always outgun them.

Not some crazy high-tower thing.  Just the basics, consistently done.

Distinguish inference from observation.

Distinguish feeling from fact.

Expose cruxes, or acknowledge up front that you haven't found them yet, and that this is kind of a shame.

Don't weaponize motte-and-bailey equivocation.

Start from a position of charity and good faith, or explain why you can't in concrete and legible detail.  Cooperate past the first apparent "defect" from your interlocutor, because people have bad days and the typical mind fallacy is a hell of a drug, as is the double illusion of transparency.

Don't respond to someone's assertion of [A] with "But [B] is abhorrent!"  Don't gloss over the part where your argument depends on the assumption that [A→B].

And most importantly of all: don't actively resist the attempts of others to do these things, or to remind others to do them.  Don't sneer, don't belittle, don't dismiss, don't take-it-as-an-attack.  Act in the fashion of someone who wants to be reminded of such things, even when it's inconvenient or triggers a negative emotional reaction.

Until users doing the above (and similar) consistently win against users who aren't, LessWrong is going to miss out on a thing that clearly a lot of us kind of want, and kind of think might actually matter to some other pretty important goals.

Maybe that's fine.  Maybe all we really need is the low-effort rough draft.  Maybe the 80/20 is actually the right balance.  In fact, we're honestly well past the 80/20—LessWrong is at least an 85/37 by this point.

But it's not actually doing the thing, and as far as I can tell it's not really trying to do the thing, either—not on the level of "approximately every individual feels called to put forth a little extra effort, and approximately every individual feels some personal stake when they see the standards being degraded."

Instead, as a collective, we've got one foot on the gas and the other on the brake, and that probably isn't the best strategy for any worthwhile goal.  One way or another, I think we should actually make up our minds, here, and either go out and hunt stag, or split up and catch rabbits.

Author's note: this essay is not as good as I wished it would be.  In particular, it's falling somewhat short of the very standard it's pulling for, in a way that I silver-line as "reaching downward across the inferential gap" but which is actually just the result of me not having the spoons to do this or this kind of analysis on each of a dozen different examples. Having spent the past six days improving it, "as good as I wish it would be" is starting to look like an asymptote, so I chose now over never.

New Comment
375 comments, sorted by Click to highlight new comments since: Today at 10:32 PM
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

There's a vision here of what LessWrong could/should be, and what a rationalist community could/should be more generally. I want to push back against that vision, and offer a sketch of an alternative frame.

The post summarizes the vision I want to push back against as something like this:

What I really want from LessWrong is to make my own thinking better, moment to moment.  To be embedded in a context that evokes clearer thinking, the way being in a library evokes whispers.  To be embedded in a context that anti-evokes all those things my brain keeps trying to do, the way being in a church anti-evokes coarse language.

Now, I do think that's a great piece to have in a vision for the LessWrong or the rationalist community. But I don't think it's the central piece, at least not in my preferred vision.

What's missing? What is the central piece?

Fundamentally, the problem with this vision is that it isn't built for a high-dimensional world. In a high-dimensional world, the hard part of reaching an optimum isn't going-uphill-rather-than-downhill; it's figuring out which direction is best, out of millions of possible directions. Half the directions are marginally-good, half are marg... (read more)

I think I agree with the models here, and also want to add a complicating factor that I think impacts the relevance of this.

I think running a site like this in a fully consequentialist way is bad. When you're public and a seed of something, you want to have an easily-understandable interface with the world; you want it to be the case that other people who reason about you (of which there will be many, and who are crucial to your plan's success!) can easily reason about you. Something more like deontology or virtue ethics ("these are the rules I will follow" or "these are the virtues we will seek to embody") makes it much easier for other agents to reason about you.

And so the more that I as a mod (or the mod team in general) rely on our individual prudence or models or so on, the more difficult it becomes for users to predict what will happen, and that has costs. (I still think that it ultimately comes down to our prudence--the virtues that we're trying to embody do in fact conflict sometimes, and it's not obvious how to resolve those conflicts--but one of the things my prudence is considering are those legibility costs.)

And when we try to figure out what virtues we should embody on... (read more)

+1 to all this, and in particular I'm very strongly on board with rationality going beyond AI safety. I'm a big fan of LessWrong's current nominal mission to "accelerate intellectual progress", and when I'm thinking about making progress in a high-dimensional world, that's usually the kind of progress I'm thinking about. (... Which, in turn, is largely because intellectual/scientific/engineering progress seem to be the "directions" which matter most for everything else.)

I think your main point here is wrong.

Your analysis rests on a lot of assumptions:

1) It's possible to choose a basis which does a good job separating the slope from the level

2) Our perturbations are all small relative to the curvature of the terrain, such that we can model things as an n-dimensional plane

3) "Known" errors can be easily avoided, even in many dimensional space, such that the main remaining question is what the right answers are

4) Maintenance of higher standards doesn't help distinguish between better and worse directions.

5) Drama pushes in random directions, rather than directions selected for being important and easy to fuck up.

1) In a high dimensional space, almost all bases have the slope distributed among many basis vectors. If you can find a basis that has a basis vector pointing right down the gradient and the rest normal to it, that's great. If your bridge has one weak strut, fix it. However, there's no reason to suspect we can always or even usually do this. If you had to describe the direction of improvement from a rotting log to a nice cable stayed bridge, there's no way you could do it simply. You could name the direction "more better", but in order to act... (read more)

This is a great comment. There are some parts which I think are outright wrong (e.g. drama selects for most-important-disagreements), but for the most part it correctly identifies a bunch of shortcomings of the linear model from my comment.

I do think these shortcomings can generally be patched; the linear model is just one way to explain the core idea, and other models lead to the same place. The main idea is something like "in a high dimensional space, choosing the right places to explore is way more important than speed of exploration", and that generalizes well beyond linearity.

I'm not going to flesh that out more right now. This all deserves a better explanation than I'm currently ready to write.

Yeah, I anticipated that the "Drama is actually kinda important" bit would be somewhat controversial. I did qualify that it was selected "(if imperfectly)" :p

Most things are like "Do we buy our scratch paper from walmart or kinkos?", and there are few messes of people so bad that it'd make me want to say "Hey, I know you think what you're fighting about is important, but it's literally less important than where we buy our scratch paper, whether we name our log files .log or .txt, and literally any other random thing you can think of".

(Actually, now that I say this, I realize that it can fairly often look that way and that's why "bikeshedding" is a term. I think those are complicated by factors like "What they appear to be fighting about isn't really what they're fighting about", "Their goals aren't aligned with the goal you're measuring them relative to", and "The relevant metric isn't how well they can select on an absolute scale or relative to your ability, but relative to their own relatively meager abilities".)

In one extreme, you say "Look, you're fighting about this for a reason, it's clearly the most important thing, or at least top five, ignore anyone arguing otherwise".

In a... (read more)

9[DEACTIVATED] Duncan Sabien2y
This was an outstandingly useful mental image for me, and one I suspect I will incorporate into a lot of thoughts and explanations.  Thanks. EDIT: finished reading the rest of this, and it's tied (with Vaniver's) for my favorite comment on this post (at least as far as the object level is concerned; there are some really good comments about the discussions themselves).
4Said Achmiz2y
Outstanding comment. (Easily the best I’ve read on Less Wrong in the last month, top five in the last year.)


I object to describing recent community discussions as "drama". Figuring out what happened within community organizations and holding them accountable is essential for us to have a functioning community. [I leave it unargued that we should have community.]

I agree that figuring out what happened and holding people/orgs accountable is important. That doesn't make the process (at least the process as it worked this time) not drama. I certainly don't think that the massive amount of attention the recent posts achieved can be attributed to thousands of people having a deeply-held passion for building effective organizations.

Not sure if this is what you're getting at. My estimate is that only a few dozen people participated and that I would ascribe to most of them either a desire for good organizations, a desire to protect people or a desire for truth and good process to be followed. I'd put entertainment seeking as a non-trivial motivation for many, and to be responsible for certain parts of the conversation, but not the overall driver.
9[DEACTIVATED] Duncan Sabien2y
For me personally, they're multiplied terms in the Fermi.  Like, engagement = [desire for good]*["entertainment"]*[several other things]. I wouldn't have been there at all just for the drama.  But also if there was zero something-like-pull, zero something-like-excitement, I probably wouldn't have been there either.   I don't feel great about this.
This sounds right, I think it generalizes to a lot of other people too.
To expand on this (though I only participated in the sense of reading the posts and a large portion of the comments), my reflective preference was to read through enough to have a satisfactorily-reliable view of the evidence presented and how it related to the reliability of data and analyses from the communities in question. And I succeeded in doing so (according to my model of my current self’s upper limitations regarding understanding of a complex sociological situation without any personally-observed data). But I could feel that the above preference was being enforced by willpower which had to compete against a constantly (though slowly) growing/reinforced sense of boredom from the monotony of staying on the same topic(s) in the same community with the same broad strokes of argument far beyond what is required to understand simpler subjects. If there had been less drama, I would have read far less into the comments, and misses a few informative discussions regarding the two situations in question (CFAR/MIRI and Leverage 1.0). So basically, the “misaligned subagent-like-mental-structures” manifestation of akrasia is messing things up again.

(I like the above and agree with most of it and am mulling and hope to be able to reply substantively, but in the meantime I wanted to highlight one little nitpick that might be more than a nitpick.)

Maybe finding and fixing organizational problems will lead to marginally more researcher time/effort on alignment, or maybe the drama itself will lead to a net loss of researcher attention to alignment. But these are both mechanisms of going marginally faster or marginally slower along the direction we're already pointed. In a high-dimensional world, that's not the sort of thing which matters much.

I think this leaves out a thing which is an important part of most people's values (mine included), which is that there's something bad about people being hurt, and there's something good about not hurting people, and that's relevant to a lot of people (me included) separate from questions of how it impacts progress on AI alignment.  Like, on the alignment forum, I get subordinating people's pain/suffering/mistreatment to questions of mission progress (maybe), but I think that's not true of a more general place like LessWrong.

Put another way, I think there might be a gap between the importance you reflectively assign to the Drama, and the importance many others reflectively assign to it.  A genuine values difference.

I do think that on LessWrong, even people's pain/suffering/mistreatment shouldn't trump questions of truth and accuracy, though.  Shouldn't encourage us to abandon truth and accuracy.

Addendum to the quoted claim:

Maybe finding and fixing organizational problems will lead to marginally more researcher time/effort on alignment, or maybe the drama itself will lead to a net loss of researcher attention to alignment. But these are both mechanisms of going marginally faster or marginally slower along the direction we're already pointed. In a high-dimensional world, that's not the sort of thing which matters much.

... and it's also not the sort of thing which matters much for reduction of overall pain/suffering/mistreatment, even within the community. (Though it may be the sort of thing which matters a lot for public perceptions of pain/suffering/mistreatment.) This is a basic tenet of EA: the causes which elicit great public drama are not highly correlated with the causes which have lots of low-hanging fruit for improvement. Even within the rationalist community, our hardcoded lizard-brain drama instincts remain basically similar, and so I expect the same heuristic to apply: public drama is not a good predictor of the best ways to reduce pain/suffering/mistreatment within the community.

But that's a post-hoc explanation. My actual gut-level response to this comment was ... (read more)

6[DEACTIVATED] Duncan Sabien2y
FWIW, I agree with this (to the extent that I've actually understood you).  Like, I think this is compatible with the OP, and do not necessarily disagree with a heuristic of flagging Captain America statements.  If 80% of them are bad, then the 20% that are good should indeed have to undergo scrutiny.
3Said Achmiz2y
What is this “Captain America” business (in this context)? Would you mind explaining, for those of us who aren’t hip with the teen culture or what have you?
6[DEACTIVATED] Duncan Sabien2y
My guess is that it's something like: Captain America makes bold claims with sharp boundaries that contain a lot of applause-light spirit, and tend to implicitly deny nuance.  They are usually in the right direction, but "sidesy" and push people more toward being in disjoint armed camps.
6Said Achmiz2y
Any chance of getting an example of such bold claims? (And, ideally, confirmation from johnswentworth that this is what’s meant?) (I ask only because I really have no knowledge of the relevant comic books on which to base any kind of interpretation of this part of the discussion…)
I explain a bit more of what I mean here: (Disclaimer: that's an old essay which isn't great by my current standards, and certainly doesn't make much attempt to justify the core model. I think it's pointing to the right thing, though.)
5[DEACTIVATED] Duncan Sabien2y 
5Said Achmiz2y
Hmm, I see. But I am fairly sure that I endorse this sentiment. Or do you think there is a non-obvious interpretation where he’s wrong?
9[DEACTIVATED] Duncan Sabien2y
I endorse this one myself (have used it in an essay before).  But it's definitely ... er, well, it emboldens people who are wrong (but unaware of it) just as much as it emboldens people who are right? I dunno.  I can't pass John's ITT here; just trying to help.  =)
It also enourages nitpicking about details where people disagree, which means that if you have several people like this on the same team, the arguing probably never stops.
John’s linked article went into it in detail:
I also think another problem here is that you are doing a Taylor expansion of the value of the community with respect to the various parameters it could have. This only really works if the proposed change is small or if the value is relatively globally linear. However, there can be many necessary-but-not-sufficient parameters, in which case the function isn't linear globally, but instead has a small peak surrounded by many directions of flatness. It seems to me that rationality with regards to local deductions, ingroup/outgroup effects, etc. could be necessary-but-not-sufficient. Without these, it's much easier to get thrown off course to some entirely misguided direction - but as you point out, having it does not necessarily provide the right guidance to make progress.
I think this depends on the nature of the bad direction. A usual bad direction might just up some smallish chunk of a single person's time, which on its own isn't that big of a problem (but it does add up, leading to the importance of the things you mentioned). However, one problem with certain topics like drama (sorry Ruby, I don't have a better word even though I realize it's problematic) is that it is highly motivating. This means that it easily attracts attention from many more people, that these people will spend much more time proportionally engaging with it, and that it has much more lasting consequences on the community. Thus getting it right seems to matter more than the typical topic.
Yup, I'm glad someone brought this up. I think the right model here is Demons in Imperfect Search. Transposons are a particularly good analogy - they're genes whose sole function is to copy-and-paste themselves into the genome. If you don't keep the transposons under control somehow, they'll multiply and quickly overrun everything else, killing the cell. So keeping the metaphorical transposons either contained or subcritical is crucial. I think LessWrong handled that basically-successfully with respect to recent events: keeping demon threads at least contained is exactly what the frontpage policy is supposed to do, and the demon threads indeed stayed off the frontpage. It was probably supercritical for a while, but it was localized, so it died down in the end and the rest of the site and community are still basically intact. (Again, as I mentioned in response to Ruby's comment, none of this is to say that the recent discussions didn't serve any useful functional role. Even transposons serve a functional role in some organisms. But the discussions certainly seemed to grow in a way decoupled from any plausible estimate of their usefulness.) Important thing to note from this model: the goal with demons is just to keep them subcritical and/or contained. Pushing them down to zero doesn't add much.
9[DEACTIVATED] Duncan Sabien2y
Agreement on the distinction between subcritical/contained and zero, and that there's usually not value in going all the way to zero.

Executive summary: I have no idea what you're talking about.

Standards are not really popular.  Most people don't like them.  Half the people here, I think, don't even see the problem that I'm trying to point at.  Or they see it, but they don't see it as a problem.

I gather that you're upset about how the Leverage conversation went, and also Cancel Culture, so I assume your chief proposition is that LessWrong is canceling Geoff Anders; but you haven't actually made that case, just vaguely gestured with words. 

I think that a certain kind of person is becoming less prevalent on LessWrong, and a certain other kind of person is becoming more prevalent, and while I have nothing against the other kind, I really thought LessWrong was for the first group.

What are the two kinds of persons? Really, I honestly do not know what you are claiming here. Repeat: I don't have even a foggy guess as to what your two kinds of person are. Am I "a certain other kind of person"? How can I know?

Distinguish feeling from fact.

This post has virtually no facts. It has short, punchy sentences with italics for emphasis. It is written like a hortatory sermon. Its primary tool is rhetoric. The f... (read more)

Yikes, despite Duncan's best attempts at disclaimers and clarity and ruling out what he doesn't mean, he apparently still didn't manage to communicate the thing he was gesturing at. That's unfortunate. (And also worries me whether I have understood him correctly.)

I will try to explain some of how I understand Duncan.

I have not read the first Leverage post and so cannot comment on those examples, but I have read jessicata's MIRI post.

and this still not having incorporated the extremely relevant context provided in this, and therefore still being misleading to anyone who doesn't get around to the comments, and the lack of concrete substantiation of the most radioactive parts of this, and so on and so forth.

As I understand it: This post criticized MIRI and CFAR by drawing parallels to Zoe Curzi's experience of Leverage. Having read the former but not the latter, the former seemed... not very substantive? Making vague parallels rather than object-level arguments? Merely mirroring the structure of the other post? In any case, there's a reason why the post sits at 61 karma with 171 votes and 925 comments, and that's not because it was considered uncontroversially true. Similarly, there's... (read more)

This comment was much longer in draft, but I've deleted the remainder because I don't want to seem "impatient" or "sneering". I'm just confused: You wrote all these words intending to convince people of something, but you don't specify what it is, and you don't use the tools we typically use to convince (facts, reliable sources, syllogistic reasoning, math, game theory...). Am I just not part of the intended audience? If so, who are they?

Thanks very much for taking the time to include this paragraph; it's doing precisely the good thing.  It helps my brain not e.g. slide into a useless and unnecessary defensiveness or round you off to something you're not trying to convey.

I gather that you're upset about how the Leverage conversation went, and also Cancel Culture, so I assume your chief proposition is that LessWrong is canceling Geoff Anders; but you haven't actually made that case, just vaguely gestured with words.

That's not, in fact, my chief proposition.  I do claim that something-like-the-mass-of-users is doing something-resembling-canceling-Leverage (such that e.g. if I were to propose porting over some specific piece of Leverage tech to LW or an EA org's internal cul... (read more)

I spent 15 minutes re-reading the thread underneath orthonormal's comment to try to put myself in your head. I think maybe I succeeded, so here goes, but from a person whose job involves persuading people, it's Not Optimal For Your Argument that I had to do this to engage with your model here, and it's potentially wasteful if I've failed at modeling you. 

I read both of the comments discussed below, at the time I was following the original post and comments, but did not vote on either.


orthonormal P1: Anders seemed like a cult leader/wannabe based on my first impressions, and I willingly incurred social cost to communicate this to others

orthonormal P2 [which I inferred using the Principle of Charity]: Most of the time, people who immediately come across as cult leaders are trying to start a cult 

Duncan P1: It's bad when LW upvotes comments with very thin epistemic rigor

Duncan P2: This comment has very thin epistemic rigor because it's based on a few brief conversations

Gloss: I don't necessarily agree with your P2. It's not robust, but nor is it thin; if true, it's one person's statement that, based on admittedly limited evidence, they had a high degree of confidence that... (read more)

Thank you for the effort!  Strong upvoted.

Quick point to get out of the way: re: the comment that you thought would likely meet my standards, yes, it does; when I hovered over it I saw that I had already (weak) upvoted it.

Here's my attempt to rewrite orthonormal's first comment; what I would have said in orthonormal's shoes, if I were trying to say what I think orthonormal is trying to say.

All right, here comes some subjective experience.  I'm offering this up because it seems relevant, and it seems like we should be in wide-net data gathering mode.

I met Geoff Anders at our 2012 CFAR workshop, and my overwhelming impression was "this person wants to be a cult leader."  This was based on [specific number of minutes] of conversation.

The impression stuck with me strongly enough that I felt like mentioning it maybe as many as [specific number] of times over the years since, in various conversations.  I was motivated enough on this point that it actually somewhat drove a wedge between me and two increasingly-Leverage-enmeshed friends, in the mid-2010's.

I feel like this is important and relevant because it seems like yet again we're in a situation where a bunch of peo

... (read more)

Thanks, supposedlyfun, for pointing me to this thread.

I think it's important to distinguish my behavior in writing the comment (which was emotive rather than optimized - it would even have been in my own case's favor to point out that the 2012 workshop was a weeklong experiment with lots of unstructured time, rather than the weekend that CFAR later settled on, or to explain that his CoZE idea was to recruit teens to meddle with the other participants' CoZE) from the behavior of people upvoting the comment.

I expect that many of the upvotes were not of the form "this is a good comment on the meta level" so much as "SOMEBODY ELSE SAW THE THING ALL ALONG, I WORRIED IT WAS JUST ME".

5[DEACTIVATED] Duncan Sabien2y
This seems true to me.  I'm also feeling a little bit insecure or something and wanting to reiterate that I think that particular comment was a net-positive addition and in my vision of LessWrong would have been positively upvoted. Just as it's important to separate the author of a comment from the votes that comment gets (which they have no control over), I want to separate a claim like "this being in positive territory is bad" (which I do not believe) from "the contrast between the total popularity of this and that is bad." I'm curious whether I actually passed your ITT with the rewrite attempt.

Thanks for asking about the ITT. 

I think that if I put a more measured version of myself back into that comment, it has one key difference from your version.

"Pay attention to me and people like me" is a status claim rather than a useful model.

I'd have said "pay attention to a person who incurred social costs by loudly predicting one later-confirmed bad actor, when they incur social costs by loudly predicting another". 

(My denouncing of Geoff drove a wedge between me and several friends, including my then-best friend; my denouncing of the other one drove a wedge between me and my then-wife. Obviously those rifts had much to do with how I handled those relationships, but clearly it wasn't idle talk from me.)

Otherwise, I think the content of your ITT is about right. 

(The emotional tone is off, even after translating from Duncan-speak to me-speak, but that may not be worth going into.)

For the record, I personally count myself 2 for 2.5 on precision. (I got a bad vibe from a third person, but didn't go around loudly making it known; and they've proven to be not a trustworthy person but not nearly as dangerous as I view the other two. I'll accordingly not name them.)

I'm going to take a stab at cruxing here.

Whether it's better for the LW community when comments explicitly state a reasonable amount of the epistemic hedging that they're doing.

Out of all the things you would have added to orthonormal's comment, the only one that I didn't read at the time as explicit or implicit in zir comment was, "Not as anything definitive, but if I do an honest scan over the past decade, I feel like I'm batting ... 3/5, maybe, with 2 more that are undecided, and the community consensus is doing more like 1/5". I agree it would be nice if people gave more information about their own calibration where available. I don't know whether it was available to orthonormal.

As for the rest, I'm sticking that at the end of this comment as a sort of appendix.

If I'm right about the crux, that is totally not in the set of Things That I Thought You Might Have Been Saying after reading the original post. Re-reading the original post now, I don't see how I could have figured out that this is what our actual disagreement was.

I notice that I am surprised that {the norm of how explicit a comment needs to be regarding its own epistemic standard} prompted you to write the original pos... (read more)

9[DEACTIVATED] Duncan Sabien2y
Hmmm, something has gone wrong.  This is not the case, and I'm not sure what caused you to think it was the case. "How explicit comments need to be regarding their own epistemic status" is a single star in the constellation of considerations that caused me to write the post.  It's one of the many ways in which I see people doing things that slightly decrease our collective ability to see what's true, in a way that compounds negatively, where people might instead do things that slightly increase our collective ability, in a way that compounds positively. But it's in no way the central casus belli of the OP.  The constellation is.  So my answer to "Why do you think the intensity scalars are so different between us?" is "maybe they aren't?  I didn't mean the thing you were surprised by." Here, I was pulling for the virtue of numeric specificity, which I think is generally understood on LW. I'm reminded of the time that some researchers investigated what various people meant by the phrase "a very real chance," and found that at least one of them meant 20% and at least one of them meant 80% (which are opposites).   It's true that numbers aren't super reliable, but even estimated/ballpark numbers (you'll note I wrote the phrase "as many as" and imagined ortho stating a ceiling) are much better for collective truth-tracking than wide-open vague phrases that allow people with very different interpretations to be equally confident in those interpretations.  The goal, after all, at least in my view, is to help us narrow down the set of possible worlds consistent with observation.  To provide data that distinguishes between possibilities. True.  (I reiterate, feeling a smidge defensive, that I've said more than once that the comment was net-positive as written, and so don't wish to have to defend a claim like "it absolutely should have been different in this way!"  That's not a claim I'm making.  I'm making the much weaker claim that my rewrite was better.  Not that the o

Understood: the comment-karma-disparity issue is, for you, a glaring example of a larger constellation. 

Also understood: you and I have different preferences for explicitly stating underlying claims. I don't think your position is unreasonable, just that it will lead to much-longer comments possibly at the cost of clarity and engagement.  Striking that balance is Hard.

I think we've drilled as far down as is productive on my concerns with the text of your post. I would like to see your follow-up post on the entire constellation, with the rigor customary here. You could definitely persuade me. I maybe was just not part of the target audience for your post.

3[DEACTIVATED] Duncan Sabien2y
(Something genuinely amusing, given the context, about the above being at 3 points out of 2 votes after four hours, compared to its parent being at 30 points out of 7 votes after five.)

It's bad that comments which are good along three different axes, and bad along none as far as I can see, are ranked way below comments that are much worse along those three axes and also have other flaws

I have an alternative and almost orthogonal interpretation for why the karma scores are the way they are.

Both in your orthonormal-Matt example, and now in this meta-example, the shorter original comments require less context to understand and got more upvotes, while the long meandering detail-oriented high-context responses were hardly even read by anyone.

This makes perfect sense to me - there's a maximum comment length after which I get a strong urge to just ignore / skim a comment (which I initially did with your response here; and I never took the time to read Matt's comments, though I also didn't vote on orthonormal's comment one way or another, nor vote in the jessicata post much at all), and I would be astonished if that only happened to me.

Also think about how people see these comments in the first place. Probably a significant chunk comes from people browsing the comment feed on the LW front page, and it makes perfect sense to scroll past a long sub-sub-sub-comment that mig... (read more)

First, some off the cuff impressions of matt's post (in the interest of data gathering):

In the initial thread I believe that I read the first paragraph of matt's comment, decided I would not get much out of it, and stopped reading without voting.

Upon revisiting the thread and reading matt's comment in full, I find it difficult to understand and do not believe I would be able to summarize or remember its main points now, about 15 minutes after the fact.

This seems somewhat interesting to test, so here is my summary from memory. After this I'll reread matt's post and compare what I thought it said upon first reading with what I think it says upon a second closer reading:

[person who met geoff] is making anecdotal claims about geoff's cult-leader-ish nature based on little data. People who have much more data are making contrary claims, so it is surprising that [person]'s post has so many upvotes. [commenter to person] is using deadpan in a particular way, which could mean multiple things depending on context but I lack that context. I believe that they are using it to communicate that geoff said so in a non-joking manner, but that is also hearsay.

Commentary before re-reading: I expect ... (read more)

9[DEACTIVATED] Duncan Sabien2y
Strong upvote for doing this process/experiment; this is outstanding and I separately appreciate the effort required. I find your summary at least within-bounds, i.e. not fully ruled out by the words on the page.  I obviously had a different impression, but I don't think that it's invalid to hold the interpretations and hypotheses that you do. I particularly like and want to upvote the fact that you're being clear and explicit about them being your interpretations and hypotheses; this is another LW-ish norm that is half-reliable and I would like to see fully reliable.  Thanks for doing it.

To add one point:

When it comes to assessing whether a long comment or post is hard to read, quality and style of writing matters, too. SSC's Nonfiction Writing Advice endlessly hammers home the point of dividing text into increasingly smaller chunks, and e.g. here's one very long post by Eliezer that elicited multiple comments of the form "this was too boring to finish" (e.g. this one), some of which got alleviated merely by adding chapter breaks.

And since LW makes it trivial to add headings even to comments (e.g. I used headings here), I guess that's one more criterion for me to judge long comments by.

(One could even imagine the LW site nudging long comments towards including stuff like headings. One could imagine a good version of a prompt like this: "This comment / post is >3k chars but consists of only 3 paragraphs and uses no headings. Consider adding some level of hierarchy, e.g. via headings.")

You're fighting fire with fire. It's hard for me to imagine a single standard that would permit this post as acceptably LessWrongian and also deem the posts you linked to as unacceptable. Here's an outline of the tactic that I see as common to both. 1. You have a goal X. 2. To achieve X, you need to coordinate people to do Y. 3. The easiest way to coordinate people to do Y is to use exhortatory rhetoric and pull social strings, while complaining when your opponent does the same thing. 4. You can justify (3) by appealing to a combination of the importance of X and of your lack of energy or desire not to be perfectionistic, while insisting that your opponents rise to a higher standard, and denying that you're doing any of this - or introspecting for a while and then shrugging and doing it anyway. 5. If you can convince others to agree with you on the overriding importance of X (using rhetoric and social strings), then suddenly the possibly offensive moral odor associated with the tactic disappears. After all, everybody (who counts) agrees with you, and it's not manipulative to just say what everybody (who counts) was thinking anyway, right? "Trying to remind people why they should care" is an example of step (3). This isn't straightforwaredly wrong. It's just a way to coordinate people, one with certain advantages and disadvantages relative to other coordination mechanisms, and one that is especially tractable for certain goals in certain contexts. In this case, it seems like one of your goals is to effect a site culture in which this tactic self-destructs. The site's culture is just so stinkin' rational that step (3) gets nipped in the bud, every time. This is the tension I feel in reading your post. On the one hand, I recognize that it's allowing itself an exception to the ban it advocates on this 5-step tactic in the service of expunging the 5-step tactic from LessWrong. On the other hand, it's not clear to me whether, if I agreed with you, I would criti

All right, a more detailed response.

You're fighting fire with fire.

I am not fighting fire with fire.  I request that you explicitly retract the assertion, given that it is both a) objectively false, and b) part of a class of utterances that are in general false far more often than they are true, and which tend to make it harder to think and see clearly in exactly the way I'm gesturing at with the OP.

Some statements that would not have been false:

"This seems to me like it's basically fighting fire with fire."

"I believe that, in practice, this ends up being fighting fire with fire."

"I'm having a hard time summing this up as anything other than 'fighting fire with fire.'"

...and I reiterate that those subtle differences make a substantial difference in people's general ability to do the collaborative truth-seeking thing, and are in many ways precisely what I'm arguing for above.

I clearly outline what I am identifying as "fire" in the above post.  I have one list which is things brains do wrong, and another list which lays out some "don'ts" that roughly correspond to those problems.

I am violating none of those don'ts, and, in my post, exhibiting none of those wrongbrains. &nbs... (read more)

I would like to register that I think this is an excellent comment, and in fact caused me to downvote the grandparent where I would otherwise have neutral or upvoted. (This is not the sort of observation I would ordinarily feel the need to point out, but in this case it seemed rather appropriate to do so, given the context.)

Huh. Interesting.

I had literally the exact same experience before I read your comment dxu.

I imagine it's likely that Duncan could sort of burn out on being able to do this [1] since it's pretty thankless difficult cognitive work. [2]

But it's really insightful to watch. I do think he could potentially tune up [3] the diplomatic savvy a bit [4] since I think while his arguments are quite sound [5] I think he probably is sometimes making people feel a little bit stupid via his tone. [6]

Nevertheless, it's really fascinating to read and observe. I feel vaguely like I'm getting smarter.


Rigor for the hell of it [7]:

[1] Hedged hypothesis.

[2] Two-premise assertion with a slightly subjective basis, but I think a true one.

[3] Elaborated on a slightly different but related point further in my comment below to him with an example.

[4] Vague but I think acceptably so. To elaborate, I mean making one's ideas even when in disagreement with a person palatable to the person one is disagreeing with. Note: I'm aware it doesn't acknowledge the cost of doing so and running that filter. Note also: I think, with skill and practice, this can be done without sacrificing the content of the message. It is a... (read more)

Upvoted for the market analogy.

7[DEACTIVATED] Duncan Sabien2y
(Thanks for being specific; this is a micro-norm I want to applaud.)
4[DEACTIVATED] Duncan Sabien2y
Nope.  False, and furthermore Kafkaesque; there is no defensible reading of either the post or my subsequent commentary that justifies this line, and that alone being up-front and framing the rest of what you have to say is extremely bad, and a straightforward example of the problem. It is a nuance-destroying move, a rounding-off move, a making-it-harder-for-people-to-see-and-think-clearly move, an implanting-falsehoods move.  Strong downvote as I compose a response to the rest.

Given that there is lots of "let's comment on what things about a comment are good and which things are bad" going on in this thread, I will make more explicit a thing that I would have usually left implicit: 

My current sense is that this comment maybe was better to write than no comment, given the dynamics of the situation, but I think the outcome would have been better if you had waited to write your long comment. This comment felt like it kicked up the heat a bunch, and while I think that was better than just leaving things unresponded, my sense is the discussion overall would have gone better if you had just written your longer comment.

5[DEACTIVATED] Duncan Sabien2y
In response to this, I'll bow out (from this subthread) for a minimum period of 3 days.  (This is in accordance with a generally wise policy I'm trying to adopt.) EDIT: I thought Oli was responding to a different thing (I replied to this from the sidebar).  I was already planning not to add anything substantive here for a few days.  I do note, though, that even if two people both unproductively turn up the heat, one after the other, in my culture it still makes a difference which one broke peace first.
4lionhearted (Sebastian Marshall)2y
Have you read "Metaphors We Live By" by Lakoff? The first 20 pages or so are almost a must-read in my opinion. Highly recommended, for you in particular. A Google search with filetype:pdf will find you a copy. You can skim it fast — not needed to close read it — and you'll get the gems. Edit for exhortation: I think you'll get a whole lot out of it such that I'd stake some "Sebastian has good judgment" points on it that you can subtract from my good judgment rep if I'm wrong. Seriously please check it out. It's fast and worth it.
This response I would characterize as steps (3) and (4) of the 5-step tactic I described. You are using more firey rhetoric ("Kafkaesque," "extremely bad," "implanting falsehoods,"), while denying that this is what you are doing. I am not going to up-vote or down-vote you. I will read and consider your next response here, but only that response, and only once. I will read no other comments on this post, and will not re-read the post itself unless it becomes necessary. I infer from your response that from your perspective, my comment here, and me by extension, are in the bin of content and participants you'd like to see less or none of on this website. I want to assure you that your response here in no way will affect my participation on the rest of this website. Your strategy of concentration of force only works if other people are impacted by that force. As far as your critical comment here, as the Black Knight said, I've known worse. If you should continue this project and attack me outside of this post, I am precommitting now to simply ignoring you, while also not engaging in any sort of comment or attack on your character to others. I will evaluate your non-activist posts the same way I evaluate anything else on this website. So just be aware that from now on, any comment of yours that strikes me as having a tone similar to this one of yours will meet with stony silence from me. I will take steps to mitigate any effect it might have on my participation via its emotional effect. Once I notice that it has a similar rhetorical character, I will stop reading it. I am specifically neutralizing the effect of this particular activist campaign of yours on my thoughts and behavior.

Jumping in here in what i hope is a prosocial way. I assert as hypothesis that the two of you currently disagree about what level of meta the conversation is/should-be at, and each feels that the other has an obligation to meet them at their level, and this has turned up the heat a lot. 

maybe there is a more oblique angle then this currently heated one?

8[DEACTIVATED] Duncan Sabien2y
It's prosocial.  For starters, AllAmericanBreakfast's "let's not engage," though itself stated in a kind of hot way, is good advice for me, too.  I'm going to step aside from this thread for at least three days, and if there's something good to come back to, I will try to do so.
9[DEACTIVATED] Duncan Sabien2y
BTW this inspired an edit in the "two kinds of persons" spot specifically, and I think the essay is much stronger for it, and I strongly appreciate you for highlighting your confusion there. EDIT: and also in the author's note at the bottom.

I'll have more to say on this in the future, but for now I just want to ramble about something.

I've been reading through some of the early General Semantics works. Partially to see if there are any gaps in my understanding they can fill, partially as a historical curiosity (how much of rationality did they have figured out, and what could they do with it?), partially because it might be good fodder for posts on LW (write a thousand posts to Rome).

And somehow a thing that keeps coming into my mind while reading them is the pre-rigorous/rigorous/post-rigorous split Terrence Tao talks about, where mathematicians start off just doing calculation and not really understanding proofs, and then they understand proofs through careful diligence, and then they intuitively understand proofs and discard many of the formalisms in their actions and speech.

Like, the early General Semantics writers pay careful attention to many things that I feel like I have intuitively incorporated; they're trying to be rigorous about scientific thinking (in the sense that they mean) in a way that I think I can be something closer to post-rigorous. Rather than this just being "I'm sloppier than they were", I think... (read more)

For me, what this resonates most clearly with is the interaction I just had with Ben Pace.

Ben was like "X"

And I was like "mostly yes to X, but also no to the implicature that I think surrounds X which is pretty bad."

And Ben was like "oh, definitely not that!  Heck no!  Thanks for pointing it out, but no, and also I think I'll change nothing in response to finding out that many people might draw that from X!"

And my response was basically "yeah, I don't think that Ben Pace on the ideal LessWrong should do anything different."

Because the thing Ben said was fine, and the implicature is easy to discard/get past.  Like "almost certainly Ben Pace isn't trying to imply [that crap], I don't really need to feel defensive about it, I can just offhandedly say 'by the way, not [implication], and it's fine for Ben to not have ruled that out just like it's fine for Ben to not also actively rule out 'by the way, don't murder people' in every comment."

But that's because Ben and I have a high-trust, high-bandwidth thing going.

The more that LW as a whole is clean-and-trustworthy in the way that the Ben-Duncan line segment is clean-and-trustworthy, the less that those implications are f... (read more)

And maybe I should update in that direction, and just ignore a constant background shrieking.

I'm not sure about this; there is some value in teaching undergrads rigor, and you seem more motivated to than I am. And, like, I did like Logan's comment about rumor, and I think more people observing things like that sooner is better. I think my main hope with the grandparent was to check if you're thinking the rigor is the moon or the finger, or something.

9[DEACTIVATED] Duncan Sabien2y
My views here aren't fully clarified, but I'm more saying "the pendulum needs to swing this way for LessWrong to be good" than saying "LessWrong being good is the pendulum being all the way over there." Or, to the extent that I understood you and am accurately representing Ben Pace, I agree with you both.

Some strings of wisdom that seem related:

"You have to know the rules before you can break them."

There has to be some sense that you're riffing deliberately and not just wrong about the defaults

The ability to depart from The Standard Forms is dependent on both the level of trust and the number of bystanders who will get the wrong idea (see my and Critch's related posts, or my essay on the social motte-and-bailey).

"Level three players can't distinguish level two players from level four players."

This suggests to me a different idea on how to improve LessWrong: make an automated "basics of disagreement" test. This involves recognizing a couple of basic concepts like cruxes and common knowledge, and involves looking at some comment threads and correctly diagnosing "what's going on" in them (e.g. where are they talking past each other) and you have to notice a bunch of useful ways to intervene.

Then if you pass, your username on comments gets a little badge next to it, and your strong vote strength gets moved up to +4 (if you're not already there).

The idea is to make it clearer who is breaking the rules that they know, versus who is breaking the rules that they don't know.

3[DEACTIVATED] Duncan Sabien2y
Interestingly, my next planned essay is an exploration of a single basic of disagreement.
Seen, also, which are you thinking of? I might have had nothing to say, or I might have just been busy when I saw the response and I wasn't tracking that I should respond to it.
2[DEACTIVATED] Duncan Sabien2y
6Ben Pace2y
This comment is surprising to me in how important I think this point is.
4[DEACTIVATED] Duncan Sabien2y
Not surprising to me given my recent interactions with you and Romeo, but I agree it's quite important and I wouldn't mind a world where it became the main frontier of this discussion.
4Said Achmiz2y
This is an instance of three levels of mastery.

This post makes me kind of uncomfortable and I feel like the locus is in... bad boundaries maybe? Maybe an orientation towards conflict, essentializing, and incentive design?

Here's an example where it jumped out at me:

Another metaphor is that of a garden.

You know what makes a garden?


Gardens aren't just about the thriving of the desired plants.  They're also about the non-thriving of the non-desired plants.

And weeding is hard work, and it's boring, and it's tedious, and it's unsexy.

Here's another:

But gardens aren't just about the thriving of the desired plants.  They're also about the non-thriving of the non-desired plants.

There's a difference between "there are many black ravens" and "we've successfully built an environment with no white ravens."  There's a difference between "this place substantially rewards black ravens" and "this place does not reward white ravens; it imposes costs upon them."

Like... this is literally black and white thinking? 

And why would a good and sane person ever want to impose costs on third parties ever except like in... revenge because we live in an anarchic horror world, or (better) as punishment after a wise and just proceed... (read more)

Like... this is literally black and white thinking?

Yes, because there is in fact a difference between "stuff that promotes individuals' and groups' ability to see and think clearly" and stuff that does not, and while we have nowhere near a solid grasp on it, we do know some of it really well at this point.  There are some things that just do not belong in a subculture that's trying to figure out what's true.

Some things are, in fact, better than others, especially when you have a goal/value set/utility function. 

And why would a good and sane person ever want to impose costs on third parties ever except like in... revenge because we live in an anarchic horror world, or (better) as punishment after a wise and just proceeding where rehabilitation would probably fail but deterrence might work?

Note the equivocation here between "I can't think of a reason why someone would want to impose costs on third parties except X" and "therefore there probably aren't non-X reasons."  This is an example of the thing; it makes it harder to see the answer to the question "why?"  Harder than it has to be, by making the "why" seem rhetorical, and as-if-there-couldn't-be-a-good-answer.

I'd like to impose some costs on statements like the above, because they tend to confuse things.

On epistemic grounds: The thing you should be objecting to in my mind is not the part where I said that "because I can't think of a reason for X, that implies that there might not be a reason for X". 

(This isn't great reasoning, but it is the start of something coherent. (Also, it is an invitation to defend X coherently and directly. (A way you could have engaged with is by explaining why adversarial attacks on the non-desired weeds would be a good use of resources rather than just... like... living and letting live, and trying to learn from things you initially can't appreciate?)))

On human decency and normative grounds: The thing you should be objecting to is that I directly implied that you personally were might not be "sane and good" because your advice seemed to be violating ideas about conflict and economics that seem normative to me.

This accusation could also have an epistemic component (which would be an ad hominem) if I were saying "you are saying X and are not sane and good and therefore not-X".  But I'm not saying this.

I'm saying that your proposed rules are bad because they request expensive actions for unclear benefits that seem likely to lead to unproductive ... (read more)

If you think I'm irrational, please enumerate the ways. Please be nuanced and detailed and unconfused. List 100 little flaws if you like.

I'm having a hard time doing this because your two comments are both full of things that seem to me to be doing exactly the fog-inducing, confusion-increasing thing.  But I'm also reasonably confident that my menu of options looks like:

  • Don't respond, and the-audience-as-a-whole, i.e. the-culture-of-LessWrong, will largely metabolize this as tacit admission that you were right, and I was unable to muster a defense because I don't have one that's grounded in truth
  • Respond in brief, and the very culture that I'm saying currently isn't trying to be careful with its thinking and reasoning will round-off and strawman and project onto whatever I say.  This seems even likelier than usual here in this subthread, given that your first comment does this all over the place and is getting pretty highly upvoted at this point.
  • Respond at length, here but not elsewhere, and try to put more data and models out there to bridge the inferential gaps (this feels doomy/useless, though, because this is a site already full of essays detailing all of the things wr
... (read more)
  • Don't respond, and the-audience-as-a-whole, i.e. the-culture-of-LessWrong, will largely metabolize this as tacit admission that you were right, and I was unable to muster a defense because I don't have one that's grounded in truth
  • Respond in brief, and the very culture that I'm saying currently isn't trying to be careful with its thinking and reasoning will round-off and strawman and project onto whatever I say. This seems even likelier than usual here in this subthread, given that your first comment does this all over the place and is getting pretty highly upvoted at this point.
  • Respond at length, here but not elsewhere, and try to put more data and models out there to bridge the inferential gaps (this feels doomy/useless, though, because this is a site already full of essays detailing all of the things wrong with your comments)
  • Respond at length to all such comments, even though it's easier to produce bullshit than to refute bullshit, meaning that I'm basically committing to put forth two hours of effort for every one that other people can throw at me, which is a recipe for exhaustion and demoralization and failure, and which is precisely why the OP was written. "People not d
... (read more)

I'd contend that a post can be "in good faith" in the sense of being a sincere attempt to communicate your actual beliefs and your actual reasons for them, while nonetheless containing harmful patterns such as logical fallacies, misleading rhetorical tricks, excessive verbosity, and low effort to understand your conversational partner.  Accusing someone of perpetuating harmful dynamics doesn't necessarily imply bad faith.

In fact, I see this distinction as being central to the OP.  Duncan talks about how his brain does bad things on autopilot when his focus slips, and he wants to be called on them so that he can get better at avoiding them.

I want to reinforce the norm of pointing out fucky dynamics when they occur...

Calling this subthread part of a fucky dynamic is begging the question a bit, I think.

If I post something that's wrong, I'll get a lot of replies pushing back.  It'll be hard for me to write persuasive responses, since I'll have to work around the holes in my post and won't be able to engage the strongest counterarguments directly.  I'll face the exact quadrilemma you quoted, and if I don't admit my mistake, it'll be unpleasant for me!  But, there's nothing fucky happening: that's just how it goes when you're wrong in a place where lots of bored people can see.

When the replies are arrant, bad faith nonsense, it becomes fucky.  But the structure is the same either way: if you were reading a thread you knew nothing about on an object level, you wouldn't be able to tell whether you were looking at a good dynamic or a bad one.

So, calling this "fucky" is calling JenniferRM's post "bullshit".  Maybe that's your model of JenniferRM's post, in which case I guess I just wasted your time, sorry about that.  If not, I hope this was a helpful refinement.

(My sense is that dxu is not referring to JenniferRM's post, so much as the broader dynamic of how disagreement and engagement unfold, and what incentives that creates.)

Fair enough!  My claim is that you zoomed out too far: the quadrilemma you quoted is neither good nor evil, and it occurs in both healthy threads and unhealthy ones.   (Which means that, if you want to have a norm about calling out fucky dynamics, you also need a norm in which people can call each others' posts "bullshit" without getting too worked up or disrupting the overall social order.  I've been in communities that worked that way but it seemed to just be a founder effect, I'm not sure how you'd create that norm in a group with a strong existing culture).

It's often useful to have possibly false things pointed out to keep them in mind as hypotheses or even raw material for new hypotheses. When these things are confidently asserted as obviously correct, or given irredeemably faulty justifications, that doesn't diminish their value in this respect, it just creates a separate problem.

A healthy framing for this activity is to explain theories without claiming their truth or relevance. Here, judging what's true acts as a "solution" for the problem, while understanding available theories of what might plausibly be true is the phase of discussing the problem. So when others do propose solutions, do claim what's true, a useful process is to ignore that aspect at first.

Only once there is saturation, and more claims don't help new hypotheses to become thinkable, only then this becomes counterproductive and possibly mostly manipulation of popular opinion.

This word "fucky" is not native to my idiolect, but I've heard it from Berkeley folks in the last year or two. Some of the "fuckiness" of the dynamic might be reduced if tapping out as a respectable move in a conversation. I'm trying not to tap out of this conversation, but I have limited minutes and so my responses are likely to be delayed by hours or days.  I see Duncan as suffering, and confused, and I fear that in his confusion (to try to reduce his suffering), he might damage virtues of lesswrong that I appreciate, but he might not.  If I get voted down, or not upvoted, I don't care. My goal is to somehow help Duncan and maybe be less confused and not suffer, and also not be interested in "damaging lesswrong". I think Duncan is strongly attached to his attempt to normatively move LW, and I admire the energy he is willing to bring to these efforts. He cares, and he gives because he cares, I think? Probably? Maybe he's trying to respond to every response as a potential "cost of doing the great work" which he is willing to shoulder?  But... I would expect him to get a sore shoulder though, eventually :-( If "the general audience" is the causal locus through which a person's speech act might accomplish something (rather than really actually wanting primarily to change your direct interlocutor's mind (who you are speaking to "in front of the audience")) then tapping out of a conversation might "make the original thesis seem to the audience to have less justification" and then, if the audience's brains were the thing truly of value to you, you might refuse to tap out? This is a real stress. It can take lots and lots of minutes to respond to everything. Sometimes problems are so constrained that the solution set is empty, and in this case it might be that "the minutes being too few" is the ultimate constraint? This is one of the reasons that I like high bandwidth stuff, like "being in the same room with a whiteboard nearby". It is hard for me to math very well
1[DEACTIVATED] Duncan Sabien2y
If your goal is to somehow help Duncan, you could start by ceasing to relentlessly and overconfidently proceed with wrong models of me.

I liked the effort put into this comment, and found it worth reading, but disagree with it very substantially. I also think I expect it to overall have bad consequences on the discussion, mostly via something like "illusion of transparency" and "trying to force the discussion to happen that you want to happen, and making it hard for people to come in with a different frame", but am not confident. 

I think the first one is sad, and something I expect would be resolved after some more rounds of comments or conversations. I don't actually really know what to do about the second one, like, on a deeper level. I feel like "people wanting to have a different type of discussion than the OP wants to have" is a common problem on LW that causes people to have bad experiences, and I would like to fix it. I have some guesses for fixes, but none that seem super promising. I am also not totally confident it's a huge problem and worth focussing on at the margin.

In light of your recent post on trying to establish a set of norms and guidelines for LessWrong (I think you accidentally posted it before it was finished, since some chunks of it were still missing, but it seemed to elaborate on things you put forth in stag hunt), it seems worthwhile to revisit this comment you made about a month ago that I commented on. In my comment I focused on the heat of your comment, and how that heat could lead to misunderstandings. In that context, I was worried that a more incisive critique would be counterproductive. Among other things, it would be increasing the heat in a conversation that I believed to be too heated. The other worries were that I expected that you would interpret the critique as an attack that needed defending, I intuited that you were feeling bad and that taking a very critical lens to your words would worsen your mood, and that this comment is going to take me a bunch of work (Author's note: I've finished writing it. It took about 6 hours to compose, although that includes some breaks). In this comment, I'm going to provide that more incisive critique.

My goal is to engender a greater degree of empathy in you when you engage with comm... (read more)

I'm glad you took the time to respond here, and there is a lot I like about this comment. In particular, I appreciate this comment for:

  • Being specific without losing sight of the general message of the parent comment.
  • Sharing how you see your situation at the outset, which puts the tone of the comment in context.
  • Identifying clear points of disagreement where possible.

There are, however, some points of disagreement I'd like to raise and some possible deleterious consequences I'd like to flag.

I share the concern raised by habryka about the illusion of transparency, which may be increasing your confidence that you are interpreting the intended meaning (and intended consequences) of Jennifer's words. I'll go into (possibly too much) detail on one very short example of what you've written and how it may involve some misreading of Jennifer's comment. You quote Jennifer:

Perhaps you could explain "epistemic hygiene" to me in mechanistic detail, and show how I'm messing it up?

and respond:

Again the trap; ...

I was also confused about what you meant by epistemic hygiene when finishing the essays. Elsewhere someone asked whether they were one of the ones doing the bad thing you we... (read more)

3[DEACTIVATED] Duncan Sabien2y
In part, this is because a major claim of the OP is "LessWrong has a canon; there's an essay for each of the core things (like strawmanning, or double cruxing, or stag hunts)."  I didn't set out to describe and define epistemic hygiene within the essay, because one of my foundational assumptions is "this work has already been done; we're just not holding each other to the available existing standards found in all the highly upvoted common memes." This is evidence I wasn't sufficiently clear.  The "trap" I was referring to was the bulleted dynamic, whereby I either cede the argument or have to put forth infinite effort.  I agree that it wasn't at all likely deliberately set by Jennifer, but also there are ways to avoid accidentally setting such traps, such as not strawmanning your conversational partner. (Strawmanning being, basically, redefining what they're saying in the eyes of the audience.  Which they then either tacitly accept or have to actively overturn.) I think that, in the context of an essay specifically highlighting "people on this site often behave in ways that make it harder to think," doing a bunch of the stuff Jennifer did is reasonably less forgivable than usual.  It's one thing to, I dunno, use coarse and foul language; it's another thing to use it in response to somebody who's just asked that we maybe swear a little less.  Especially if the locale for the discussion is named LessSwearing (i.e. the person isn't randomly bidding for the adoption of some out-of-the-blue standard). Yes.  I do not think it was a genuine attempt to engage or converge with me (the way that Said, Elizabeth, johnswentsworth, supposedlyfun, and even agrippa were clearly doing or willing to do), so much as an attempt to condescend, lecture, and belittle, and the crowd of upvotes seemed to indicate either general endorsement of those actions, or a belief that it's fine/doesn't matter/isn't a dealbreaker.  This impression has not shifted much on rereads, and is reminiscent
This is an example of the illusion of transparency issue. Many salient interpretations of what this means (informed by the popular posts on the topic, that are actually not explicitly on this topic) motivate actions that I consider deleterious overall, like punishing half-baked/wild/probably-wrong hypotheses or things that are not obsequiously disclaimed as such, in a way that's insensitive to the actual level of danger of being misleading. A more salient cost is nonsense hogging attention, but that doesn't distinguish it from well-reasoned clear points that don't add insight hogging attention. The actually serious problem is when this is a symptom of not distinguishing epistemic status of ideas on part of the author, but then it's not at all clear that punishing publication of such thoughts helps the author fix the problem. The personal skill of tagging epistemic status of ideas in one's own mind correctly is what I think of as epistemic hygiene, but I don't expect this to be canon, and I'm not sure that there is no serious disagreement on this point with people who also thought about this. For one, the interpretation I have doesn't specify community norms, and I don't know what epistemic-hygiene-the-norm should be.
2[comment deleted]2y

[Obvious disclaimer: I am not Duncan, my views are not necessarily his views, etc.]

It seems to me that your comment is [doing something like] rounding off Duncan's position to [something like] conflict theory, and contrasting it to the alternative of a mistake-oriented approach. This impression mostly comes from passages like the following:

You're sad about the world. I'm sad about it too. I think a major cause is too much poking. You're saying the cause is too little poking. So I poked you. Now what?

If we really need to start banning the weeds, for sure and for true... because no one can grow, and no one can be taught, and errors in rationality are terrible signs that a person is an intrinsically terrible defector... then I might propose that you be banned?

And obviously this is inimical to your selfish interests. Obviously you would argue against it for this reason if you shared the core frame of "people can't grow, errors are defection, ban the defectors" because you would also think that you can't grow, and I can't grow, and if we're calling for each other's banning based on "essentializing pro-conflict social logic" because we both think the other is a "weed"... well..

... (read more)

Thank you for this great comment. I feel bad not engaging with Duncan directly, but maybe I can engage with your model of him? :-)

I agree that Duncan wouldn't agree with my restatement of what he might be saying. 

What I attributed to him was a critical part (that I object to) of the entailment of the gestalt of his stance or frame or whatever. My hope was that his giant list of varying attributes of statements and conversational motivations could be condensed into a concept with a clean intensive definition other than a mushy conflation of "badness" and "irrational". For me these things are very very different and I'll say much more about this below.

One hope I had was that he would vigorously deny that he was advocating anything like what I mentioned by making clear that, say, he wasn't going to wander around (or have large groups of people wander around) saying "I don't like X produced by P and so let's impose costs (ie sanctions (ie punishments)) on P and on all X-like things, and if we do this search-and-punish move super hard, on literally every instance, then next time maybe we won't have to hunt rabbits, and we won't have to cringe and we won't have to feel angry at ever... (read more)


my model of Duncan predicts that there are some people on LW whose presence here is motivated (at least significantly in part) by wanting to grow as a rationalist,


I think that it is likely that neither Duncan nor I likely consider ourselves in the first category.

Duncan, in the OP, which Jennifer I guess skimmed:

What I really want from LessWrong is to make my own thinking better, moment to moment. To be embedded in a context that evokes clearer thinking, the way being in a library evokes whispers. To be embedded in a context that anti-evokes all those things my brain keeps trying to do, the way being in a church anti-evokes coarse language.

I see that you have, in fact, caught me in a simplification that is not consistent with literally everything you said.  I apologize for over-simplifying, maybe I should have added "primarily" and/or "currently" to make it more literally true. In my defense, and to potentially advance the conversation, you also did say this, and I quoted it rather than paraphrasing because I wanted to not put words in your mouth while you were in a potentially adversarial mood... maybe looking to score points for unfairness? My model here is that this is your self-identified "revealed preference" for actually being here right now. Also, in my experience, revealed preferences are very very very important signals about the reality of situations and the reality of people. This plausible self-described revealed preference of yours suggests to me that you see yourself as more of a teacher than a student. More of a producer than a consumer. (This would be OK in my book. I explicitly acknowledge that I see my self as more of a teacher than a student round these parts. I'm not accusing you of something bad here, in my own normative frame, though perhaps you feel it as an attack because you have difference values and norms than I do?) It is fully possible, I guess, (and you would be able to say this much better than I) that you would actually rather be a student than a teacher? And it might be that that you see this as being impossible until or unless LW moves from a rabbit equilibrium to a stag equilibrium? ... There's an interesting possible equivocation here. (1) "Duncan growing as a rationalist as much and fast as he (can/should/does?) (really?) want does in fact require a rabbit-to-stag nash equilibrium shift among all of lesswrong". (2) "Duncan growing as a rationalist as much as and fast as he wants does seems to him to require a rabbit-to-stag nash equilibrium shift among all of lesswrong... which might then logically universally require removing literally every rabbit play

… modular sub-communities …

… a staging area and audition space for more specific and more demanding subcultures …

Here is a thng I wrote some years ago (this is a slightly cleaned up chat log, apologies for the roughness of exposition):

There was an analogue to this in WoW as well, where, as I think I’ve mentioned, there often was such a thing as “within this raid guild, there are multiple raid groups, including some that are more ‘elite’/exclusive than the main one”; such groups usually did not use the EPGP or other allocation system of the main group, but had their own thing.

(I should note that such smaller, more elite/exclusive groups, typically skewed closer to “managed communism” than to “regulated capitalism” on the spectrum of loot systems, which I do not think is a coincidence.)

[name_redacted]: Fewer people, higher internal trust presumably.

“Higher internal trust” is true, but not where I’d locate the cause. I’d say “higher degree of sublimation of personal interest to group interest”.

[name_redacted]: Ah. … More dedicated?

Yes, and more willing to sacrifice for the good of the raid. Like, if you’re trying to maintain a raiding guild of 100 people, keep it functioning a

... (read more)
Yeah! This is great. This is the kind of detailed grounded cooperative reality that really happens sometimes :-)

I quoted it rather than paraphrasing because I wanted to not put words in your mouth while you were in a potentially adversarial mood

If a person writes "I currently get A but what I really want is B"

...and then you selectively quote "I currently get A" as justification for summarizing them as being unlikely to want B...

...right after they've objected to you strawmanning and misrepresenting them left and right, and made it very clear to you that you are nowhere near passing their ITT...

...this is not "simplification."

Apologizing for "over-simplifying," under these circumstances, is a cop-out.  The thing you are doing is not over-simplification.  You are [not talking about simpler versions of me and my claim that abstract away some of the detail].  You are outright misrepresenting me, and in a way that's reeeaaalll hard to believe is not adversarial, at this point.

It is at best falling so far short of cooperative discourse as to not even qualify as a member of the set, and at worst deliberate disingenuousness.

If a person wholly misses you once, that's run-of-the-mill miscommunication.

If, after you point out all the ways they missed you, at length, they brush that off a... (read more)

I'm torn about getting into this one, since on one hand it doesn't seem like you're really enjoying this conversation or would be excited to continue it, and I don't like the idea of starting conversations that feel like a drain before they even get started. In addition, other than liking my other comment on this post, you don't really know me and therefore I don't really have the respect/trust resources I'd normally lean on for difficult conversations like this (both in the "likely emotionally significant" and also "just large inferential distances with few words" senses). On the other hand I think there's something very important here, both on the object level and on a meta level about how this conversation is going so far. And if it does turn out to be a conversation you're interested in having (either now, or in a month, or whenever), I do expect it to be actually quite productive. If you're interested, here's where I'm starting: Jennifer has explicitly stated that at this point her goal is to help you. This doesn't seem to have happened. While it's important to track possibilities like "Actually, it's been more helpful than it looks", it looks more like her attempt(s) so far have failed, and this implies that she's missing something. Do you have a model that gives any specific predictions about what it might be? Regardless of whether it's worth the effort or whether doing so would lead to bad consequences in other ways, do you have a model that gives specific predictions of what it would take to convey to her the thing(s) she's missing such that the conversation with her would go much more like you think it should, should you decide it to be worthwhile? Would you be interested in hearing the predictions my models give?
3[DEACTIVATED] Duncan Sabien2y
I don't have a gearsy model, no.  All I've got is the observations that: * Duncan's post objects to a cluster of things X, Y, and Z * Jennifer's response seems to me to state that X, Y, and Z are either not worth objecting to or possibly are actually good * Jennifer's response exhibits X, Y, and Z in substantial quantity (which, to be fair, is consistent with principled disagreement, i.e. is not a sign of hypocrisy or lack-of-skill or whatever) * Duncan's objections to X, Y, and Z within Jennifer's pushback are basically falling on deaf ears, resulting in Jennifer adding more X, Y, and Z in subsequent responses * As is to be expected, given that the whole motivation for the OP was "LessWrong keeps indulging in and upvoting X, Y, and Z," Jennifer's being upvoted. I'm interested in hearing both your model and your predictions.  Perhaps a timescale of days-weeks is better than a timescale of hours-days.

There's a lot here, and I've put in a lot of work writing and rewriting. After failing for long enough to put things in a way that is both succinct and clear, I'm going to abandon hopes of the latter and go all in on the former. I'm going to use the minimal handles for the concepts I refer to, in a way similar to using LW jargon like "steelman" without the accompanying essays, in hopes that the terms are descriptive enough on their own. If this ends up being too opaque, I can explicate as needed later.

Here's an oversimplified model to play with:

  • Changing minds requires attention, and bigger changes require more attentions.
  • Bidding for bigger attention requires bigger respect, or else no reason to follow.
  • Bidding for bigger respect requires bigger security, or else not safe enough to risk following. 
  •  Bidding for that sense of security requires proof of actual security, or else people react defensively, cooperation isn't attended to, and good things don't happen

GWS took an approach of offering proof of security and making fairly modest bids for both security and respect. As a result, the message was accepted, but it was fairly restrained in what it attempted to communicate. Fo... (read more)

8Ben Pace2y
Just checking, what are X, Y and Z?  (I'm interested in a concrete answer but would be happy with a brief vague answer too!) (Added: Please don't feel obliged to write a long explanation here just because I asked, I really just wanted to ask a small question.)
6[DEACTIVATED] Duncan Sabien2y
The same stuff that's outlined in the post, both up at the top where I list things my brain tries to do, and down at the bottom where I say "just the basics, consistently done." Regenerating the list again: Engaging in, and tolerating/applauding those who engage in: * Strawmanning (misrepresenting others' points as weaker or more extreme than they are) * Projection (speaking as if you know what's going on inside other people's heads) * Putting little to no effort into distinguishing your observations from your inferences/speaking as if things definitely are what they seem to you to be * Only having or tracking a single hypothesis/giving no signal that there is more than one explanation possible for what you've observed * Overstating the strength of your claims * Being much quieter in one's updates and oopses than one was in one's bold wrongness * Weaponizing equivocation/doing motte-and-bailey * Generally, doing things which make it harder rather than easier for people to see clearly and think clearly and engage with your argument and move toward the truth This is not an exhaustive list.
Mechanistically... since stag hunt is in the title of the post... it seems like you're saying that any one person committing "enough of these epistemic sins to count as playing stag" would mean that all of lesswrong fails at the stag hunt, right? And it might be the case that a single person playing stag could be made up of them failing at even just a single one of these sins? (This is the weakest point in my mechanistic model, perhaps?) Also, what you're calling "projection" there is not the standard model of projection I think? And my understanding is that the standard model of projection is sort of explicitly something people can't choose not to do, by default. In the standard model of projection it takes a lot of emotional and intellectual work for a person to realize that they are blaming others for problems that are really inside themselves :-( (For myself, I try not to assume I even know what's happening in my own head, because experimentally, it seems like humans in general lack high quality introspective access to their own behavior and cognition.) The practical upshot here, to me, is that if the models you're advocating here are true, then it seems to me like lesswrong will inevitably fail at "hunting stags". ... And yet it also seems like you're exhorting people to stop committing these sins and exhorting them moreover to punitively downvote people according to these standards because if LW voters become extremely judgemental like this then... maybe we will eventually all play stag and thus eventually, as a group, catch a stag? So under the models that you seem to me to have offered, the (numerous individual) costs won't buy any (group) benefits? I think?  There will always inevitably be a fly in the ointment... a grain of sand in the chip fab... a student among the masters... and so the stag hunt will always fail unless it occurs in extreme isolation with a very small number of moving parts of very high quality? And yet lesswrong will hopefully
7Alex Vermillion2y
I am confused by a theme in your comments. You have repeatedly chosen to express that the failure of a single person completely destroys all the value of the website, even going so far as to quote ridiculous numbers (at the order of E-18 [1]) in support of this. The only model I have for your behavior that explains why you would do this, instead of assuming something like Duncan believing something like "The value of C cooperators and D defectors is min(0,C−D2)" is that you are trying to make the argument look weak. If there is another reason to do this, I'd appreciate an explanation, because this tactic alone is enough to make me view the argument as likely adversarial.
5[DEACTIVATED] Duncan Sabien2y
No, and if you had stopped there and let me answer rather than going on to write hundreds of words based on your misconception, I would have found it more credible that you actually wanted to engage with me and converge on something, rather than that you just really wanted to keep spamming misrepresentations of my point in the form of questions.
Epistemic status: socially brusque wild speculation. If they're in the area and it wouldn't be high effort, I'd like JenniferRM's feedback on how close I am. My model of JenniferRM isn't of someone who wants to spam misrepresentations in the form of questions. In response to Dweomite's comment below, they say: My model of the model which which outputs words like these is that they're very confident in their own understanding--viewing themself as a "teacher" rather than a student--and are trying to lead someone who they think doesn't understand by the nose through a conversation which has been plotted out in advance.
4[DEACTIVATED] Duncan Sabien2y
Plausible to me.  (Thanks.)

And why would a good and sane person ever want to impose costs on third parties ever except like in... revenge because we live in an anarchic horror world, or (better) as punishment after a wise and just proceeding where rehabilitation would probably fail but deterrence might work? 

This paragraph sounds to me like when you say "costs" you are actually thinking of "punishments", with an implication of moral wrongdoing.  I'm uncertain that Duncan intended that implication (and if he did, I'd like to request that both of you use the more specific term).


If you continue to endorse the quoted paragraph as applied to "costs" that are not necessarily punishments, then I further contend that costs are useful in several scenarios where there is no implication of wrongdoing:

The SUPER obvious example is markets, where costs are used as a mechanism to control resource allocation.

Another common scenario is as a filter for seriousness.  Suppose you are holding a contest where you will award a prize to the best foozle.  If entry in the contest is free of charge, you might get a lot of entries from amateur foozle artists who know that they have negligible chance of winning... (read more)

Please don't make this place worse again by caring about points for reasons other than making comments occur in the right order on the page.

I wish this statement explaining what goal your advice is designed to optimize had appeared at the top of the advice, rather than the bottom.


My current world-model predicts that this is not what most people believe points are for, and that getting people to treat points in this way would require a high-effort coordinated push, probably involving radical changes to the UI to create cognitive distance from how points are used on other sites.

Specifically, I think the way most people actually use points is as a prosthetic for nonverbal politics; they are the digital equivalent of shooting someone a smile or a glower.  Smiles/glowers, in turn, are a way of informing the speaker that they are gaining/losing social capital, and informing bystanders that they could potentially gain/lose social capital depending on which side of the issue they support.

My model says this is a low-level human instinctive social behavior, with the result that it is very easy to make people behave this way, but simultaneously very hard for most people to explain ... (read more)

Like... this is literally black and white thinking? 

This is written in a way that seems to imply that if it is black and white thinking that would be bad. It also doesn't read as a question despite having a question mark.

People whos neurotype makes them default to black and white thinking can get really good when a concept does apply or doesn't apply. It has strengths and weaknesses. You are taking the attitude that it is widely known for its weaknesses. Demonstrating what is being glossed over or what kinds of things would be missed by it. I guess later in the post there are descipriton how other stances have it better but I think failure analysis on what is worse for the strongly dualistic view is sorely needed. 

One possible source of uncomfortableness if if the typical mind assumptions doesn't hold because the interaction target mind is atypical.

Phrasings like "And why would a good and sane person ever [...]" seem to prepare to mark individuals for rejection. And again it has a question word but doesn't read like a question.

I am worried about a mechanic where people recognised as deviant are categorised as undesirable. 

I do share worry about a distinction between "in" and "out" leads to a very large out group and actions that are as large as possible instead of proprotioned to neccesity or level of guarantee.

"Black and white thinking" is another name for a reasonably well defined cognitive tendency that often occurs in proximity to reasonably common mental problems.

Part of the reason "the fallacy of gray" is a thing that happens is that advice like that can be a useful and healthy thing for people who are genuinely not thinking in a great way. 

Adding gray to the palette can be a helpful baby step in actual practice.

Then very very similar words to this helpful advice can also be used to "merely score debate points" on people who have a point about "X is good and Y is bad". This is where the "fallacy" occurs... but I don't think the fallacy would occur if it didn't have the "plausible cover" that arises from the helpful version. 

A typical fallacy of gray says something like "everything is gray, therefore lets take no action and stop worrying about this stuff entirely".

One possible difference, that distinguishes "better gray" from "worse gray" is whether you're advocating for fewer than 2 or more than 2 categories.

Compare: "instead of two categories (black and white), how about more than two categories (black and white and gray), or maybe even five (pure black, dark gray, gray, ... (read more)

There is another phenomenon that also gets referred to as "black and white thinking" that has more to do with rigidity of thought. The mechanisms of that are different. I am bit unsure whether it has a more standard name and wanted to find fact information but only found an opinon piece where at number 5 there is a differential between that and splitting. I do recognise how the text fills recognition criteria for splitting and the worry seems reasonable but to me it sounds more like splitting hairs. The kind of thing were I would argue that within probability zero there is difference between "almost never" and "actually never" and for some thing it would make or break things.
If you look at some of the neighboring text, I have some mathematical arguments about what the chances are for N people to all independently play "stag" such that no one plays rabbit and everyone gets the "stag reward". If 3 people flip coins, all three coins come up "stag" quite often. If a "stag" is worth roughly 8 times as much as a rabbit, you could still sanely "play stag hunt" with 2 other people whose skill at stag was "50% of the time they are perfect".   But if they are less skilled than that, or there are more of them, the stag had better be very very very valuable. If 1000 people flip coins then "pure stag" comes up one in every 9.33x10^302 times. Thus, de facto, stag hunts fail at large N except for one of those "dumb and dumber" kind of things where you hear the one possible coin pattern that gives the stag reward and treat this as good news and say "so you're telling me there's a chance!" I think stag hunts are one of these places where the exact same formal mathematical model gives wildly different pragmatic results depending on N, and the probability of success, and the value of the stag... and you have to actually do the math, not rely on emotions and hunches to get the right result via the wisdom one one's brainstem and subconscious and feelings and so on.
6[DEACTIVATED] Duncan Sabien2y
Coin flips are an absolutely inappropriate model for stag hunts; people choosing stag and rabbit are not independent in the way that coin flips are independent; that's the whole point.  Incentives drive everyone toward rabbit; agreements drive people toward stag.  All of the reasoning descending from the choice to model things as coin flips is therefore useless.
For the record, as "arch-moderator", I care about karma for more reasons than just that, in line with Oli's list here.
Although this isn't how I think about karma, on reflection, I think it's a good and healthy frame, and I'm glad you have it and brought it up with your detailed suggestion.
Yeah, my larger position is that karma (and upboats and so on) are brilliant gamifications of "a way to change the location of elements on a webpage". Reddit is a popular website, that many love, for a reason. I remember Digg. I remember K5. I remember Slashdot. There were actual innovations in this space, over time, and part of the brilliance in the improvements was in meeting the needs of a lot of people "where they are currently at" and making pro-social use of many tendencies that are understandably imperfect. Social engineering is a thing, and it is a large part of why our murder rate is so low, and our material prosperity is so high. It is super important and, done well, is mostly good. (I basically just wish that more judges and lawyers and legislators in modern times could program computers, and brought that level of skill to the programming of society.) However, I also think that gamification ultimately should be understood as a "mere" heuristic... as a hack that works on many humans who are full of passions and confusions in predictable ways... If everyone was a sage, I think gamification would be pointless or even counter-productive. A contextually counter-productive heuristic is a bias. In a deep sense we have biases because we sometimes have heuristics that are being applied outside of their training distribution by accident. The context where gamification might not work: Eventually you know you are both the rider and the elephant. Your rider has trained (and is still training) your elephant pretty well, and sometimes even begins to ruefully be thankful that the elephant had some good training, because sometimes the rider falls asleep and it was only the luck of a well-trained elephant that kept them from tragedy.  Anyone who can get to this point (and I'm nowhere close to perfect here, but sometimes in some domains I think I'm getting close)... one barrier to progress that arises as one tries to get the rider and the elephant to play nicely, is th

LW is likely currently on something like a Pareto frontier of several values, where it is difficult to promote one value better without sacrificing others. I think that this is true, and also think that this is probably what OP believes.

The above post renders one axis of that frontier particularly emotionally salient, then expresses willingness to sacrifice other axes for it.

I appreciate that the post explicitly points out that is willing to sacrifice these other axes. It nevertheless skims a little bit over what precisely might be sacrificed.

Let's name some things that might be sacrificed:

(1) LW is a place newcomers to rationality can come to ask questions, make posts, and participate in discussion, hopefully without enormous barrier to entry. Trivial inconveniences to this can have outsized effects.

(2) LW is a kind of bulletin board and coordination center for things of general interest to an actual historical communities. Trivial inconveniences to sharing such information can once again have an outsized effect.

(3) LW is a place to just generally post things of interest, including fiction, showerthoughts, and so on, to the kind of person who is interested in rationality, AI, ... (read more)

I don't think you've made a convincing case that LW is on a Pareto frontier of these values, and I don't know what such a case would look like, either. I've personally made several suggestions here in the comments (for LW feature improvements) that would make some things better without necessarily making anybody worse off. Feature suggestions would take resources to implement, but as far as I can tell the LW team has sufficient resources to act on whatever it considers its highest-EV actions. As for the rest of your post: I appreciate that you mention other values to consider, and that you don't want them to be traded off for one another. In particular, I strongly agree that I do not want to increase barriers to entry for newcomers. But I strongly disapprove of your imputing motives into the OP that aren't explicitly there, or that aren't there without ridiculous numbers of caveats (like the suggestions OP himself flagged as "terrible ideas"). OP even ends with a disclaimer that "this essay is not as good as I wished it would be". In contrast, this entire section of yours reads to me as remarkably uncharitable and in bad faith: If you want to suggest that OP is part of a "genre of rhetoric": make the case that it is, name it explicitly. Make your own words vulnerable, put your own neck out there. Instead of making your own object-level arguments, you're imputing bad motives into the OP, insinuating things without pointing to specific quotes, and suggesting that arguments for your case could be made, but that you won't make the effort to make them. You even end on an applause light ffs. ---------------------------------------- Circling back to the object level of the essay, namely improving the culture here: As I mention in my comment on the Karma system, which I've explicitly singled out in my suggestions for improvement: Your comment is half decent, half terrible, but the only way I have to interact with it is to assign it a single scalar (upvote or downvot

I meant a relative Pareto frontier, vis-a-vis the LW team's knowledge and resources. I think your posts on how to expand the frontier are absolutely great, and I think they (might) add to the available area within the frontier.

"If you want to suggest that OP is part of a "genre of rhetoric": make the case that it is, name it explicitly."

I mean, most of OP is about evoking emotion about community standards; deliberately evoking emotions is a standard part of rhetoric. (I don't know what genre -- ethos if you want to invoke Aristotle -- but I don't think it particularly matters.) OP explicitly says that he would like LW to be smaller -- i.e., sacrifice other values, for the value he's just evoked emotion about. I take this to just be a description of how the essay works, not a pejorative imputation of motives.

I could definitely have done better, and I too went through several drafts, and the one I posted was probably posted because I was tired of editing rather than because it was best. I have removed the sentences in the above that seem most pejorative.

An important note, that I couldn't manage to fold into the essay proper:

I have direct knowledge of at least three people who would like to say positive things about their experience at Leverage Research, but feel they cannot.  People who are fully on board with the fact that their experiences do not erase Zoe's, but whose own very different stories are relevant to understanding what actually went wrong, so that it can be fixed for those who suffered, and prevented in the future.

And at least these three people are not speaking up, because the current state of affairs is such that they feel they can't do so without committing social suicide.

This is a fact about them, not a fact about LessWrong or what would actually happen.

But it sure is damning that they feel that way, and that I can't exactly tell them that they're wrong.

Personally, I think it's clear that there was some kind of disease here, and I'd like to properly diagnose it, and that means making it possible to collect all of the relevant info, not an impoverished subset.

We were missing Zoe's subset.  It's good that we have it now.

It's bad that we're still dealing with an incomplete dataset, though, and it's really bad that a lot of people seem to think that the dataset isn't meaningfully incomplete.

But it sure is damning that they feel that way, and that I can't exactly tell them that they're wrong.

You could have, though.  You could have shown them the many highly-upvoted personal accounts from former Leverage staff and other Leverage-adjacent people.   You could have pointed out that there aren't any positive personal Leverage accounts, any at all, that were downvoted on net.  0 and 1 are not probabilities, but the evidence here is extremely one-sided: the LW zeitgeist approves of positive personal accounts about Leverage.  It won't ostracize you for posting them.

But my guess is that this fear isn't about Less Wrong the forum at all, it's about their and your real-world social scene.  If that's true then it makes a lot more sense for them to be worried (or so I infer, I don't live in California).  But it makes a lot less to bring to bring it up here, in a discussion about changing LW culture: getting rid of the posts and posters you disapprove of won't make them go away in real life.  Talking about it here, as though it were an argument in any direction at all about LW standards, is just a non sequitur.

1[DEACTIVATED] Duncan Sabien2y
Thanks for gathering these.  They are genuinely helpful (several of them I missed). But yes, as you inferred, the people I've talked to are scared about real-life consequences such as losing funding or having trouble finding employment, which are problems they don't currently have but suspect they will if they speak up. I reiterate that this is a fact about them, as oppose to a fact about reality, but they're not crazy to have some weight on it.

When it comes to the real-life consequences I think we're on the same page: I think it's plausible that they'd face consequences for speaking up and I don't think they're crazy to weigh it in their decision-making (I do note, for example, that none of the people who put their names on their positive Leverage accounts seem to live in California, except for the ones who still work there).  I am not that attached to any of these beliefs since all my data is second- and third-hand, but within those limitations I agree.

But again, the things they're worried about are not happening on Less Wrong.  Bringing up their plight here, in the context of curating Less Wrong, is not Lawful: it cannot help anybody think about Less Wrong, only hurt and distract.  If they need help, we can't help them by changing Less Wrong; we have to change the people who are giving out party invites and job interviews.

I expect that many of the people who are giving out party invites and job interviews are strongly influenced by LW. If that's the case, then we can prevent some of the things Duncan mentions by changing LW in the direction of being more supportive of good epistemics (regardless of which "side" that comes down on), with the hope of flow-through effects.
The influence can't be too strong, or they'd be influenced by the zeitgeist's willingness to welcome pro-Leverage perspectives, right?  Or maybe you disagree with that characterization of LW-the-site?
5[DEACTIVATED] Duncan Sabien2y
Things get complicated in situations where e.g. 70% of the group is welcoming and 30% of the group is silently judging and will enact their disapproval later.  And the zeitgeist that is willing to welcome pro-Leverage perspectives might not be willing to actively pressure people to not discriminate against pro-Leverage folk.  Like, they might be fine with somebody being gay, but not motivated enough to step in if someone else is being homophobic in a grocery store parking lot, metaphorically speaking. (This may not describe the actual situation here, of course.  But again it's a fear I feel like I can't dismiss or rule out.)

"three people... would like to say positive things about their experience at Leverage Research, but feel they cannot":

Oof. I appreciate you mentioning that.

(And a special note of thanks, for being willing to put down a concrete number? It helps me try to weigh it appropriately, while not compromising anonymity.)

Navigating the fact that people seem to be scared of coming forward on every side of this, is hard. I would love advice on how to shape this thread better.

If you think of something I can do to make talking about all of {the good, the bad, the neutral, the ugly, and the complicated}, easier? I can't guarantee I'll agree to it, but I really do want to hear it.

Please feel free to reach out to me on LW, anytime in the next 2 months. Not just Duncan, anyone. Offer does expire at start of January, though.

I am especially interested in concrete suggestions that improve the Pareto Frontier of reporting, here. But I'm also pretty geared up to try to steelman any private rants that get sent my way, too.

(In this context? I have already been called all of "possessed, angry, jealous, and bad with secrets." I was willing to steelman the lot, because there is a hint of truth in each, alth... (read more)

Hm... I notice I'm maybe feeling some particular pressure to personally address this one?

Because I called out the deliberate concentration of force in the other direction that happened on an earlier copy of the BayAreaHuman thread.

I am not really recanting that? I still think something "off" happened there.

But I could stand up and give a more balanced deposition.

To be clear? I do think BAH's tone was a tad aggressive. And I think there were other people in the thread, who were more aggressive than that. I think Leverage Basic Facts EA had an even more aggressive comment thread.

I also think each of the concrete factual claims BAH made, did appear to check out with at least one corner of Leverage, according to my own account-collecting (although not always at the same time).

(I also think a few of the LBFEA's wildest claims, were probably true. Exclusion of the Leverage website from Wayback Machine is definitely true*. The Slack channels characterizing each Pareto attendee as a potential recruit, seems... probably true?)

There were a lot of corners of Leverage, though. Several of them were walled off from the corners BAH talked about, or were not very near to it.

For what it's worth, ... (read more)

Have people considered just making a survey and sending it out to former Leverage staff? This really isn't my scene but it seems like while surveys have major issues, it's hard for me to imagine that surveys are worse at being statistically representative than qualitative accounts that went through many selection filters,
Not even pseudonymously?
8[DEACTIVATED] Duncan Sabien2y
Their stories are specific enough that pseudonymity isn't viable.

Strong upvote. Thank you for writing this, it articulates the problems better than I had them in my head and enhances my focus. This deserves a longer reply, but I'm not sure if I'll get to write it today, so I'll respond with my initial thoughts.

What I really want from LessWrong is to make my own thinking better, moment to moment. To be embedded in a context that evokes clearer thinking, the way being in a library evokes whispers. To be embedded in a context that anti-evokes all those things my brain keeps trying to do, the way being in a church anti-evokes coarse language.

I want this too.

In the big, important conversations, the ones with big stakes, the ones where emotions run high—
I don’t think LessWrong, as a community, does very well in those conversations at all.

Regarding the three threads you list, I, others involved in managing of LessWrong, and leading community figures who've spoken to me are all dissatisfied with how those conversations went and believe it calls for changes in LessWrong.

Solutions I am planning or considering:

  • Technological solutions (i.e. UI changes). Currently, I think it's difficult t
... (read more)

Regarding the three threads you list, I, others involved in managing of LessWrong, and leading community figures who've spoken to me are all dissatisfied with how those conversations went and believe it calls for changes in LessWrong.

I'm deeply surprised by this. If there is a consensus among the LW managers and community figures, could one of them write a post about it laying out what was dissatisfactory and what changes they feel need to be made, or at least the result they want from the changes? I know you're a highly conscientious person with too much on zir hands already, so please don't take this upon yourself.

I am also surprised by this! I think this sentence is kind of true, and am dissatisfied with the threads, but I don't feel like my take is particularly well-summarized with the above language, at least in the context of this post (like, I feel like this sentence implies a particular type of agreement with the OP that I don't think summarizes my current position very well, though I am also not totally confident I disagree with the OP). 

I am in favor of experimenting more with some karma stuff, and have been encouraging people to work on that within the Lightcone team. I think there is lots of stuff we could do better, and definitely comparing us to some ideal that I have in my head, I think things definitely aren't going remotely as well as I would like them to, but I do feel like the word "dissatisfied" seems kind of wrong. I think there are individual comments that seem bad, but overall I think the conversations have been quite good, and I am mildly positively surprised by how well they have been going. 

5[DEACTIVATED] Duncan Sabien2y
(As the author of the OP, I think my position is also consistent with "quite good, and mildly positively surprised."  I think the difference is counting up vs. counting down?  I'm curious whether you think quite good when counting down from your personal vision of the ideal LessWrong.)
When counting down we are all savages dancing to the sun gods in a feeble attempt to change the course of history. More seriously though, yeah, definitely when I count down, I see a ton of stuff that could be a lot better. A lot of important comments missing, not enough courage, not enough honesty, not enough vulnerability, not enough taking responsibility for the big picture.
I did indeed mean "dissatisfied" in a "counting down" sense.
The most obvious/annoying issue with karma is false disagreement zero equilibrium controversy tug of war that can't currently be split into more specific senses of voting to reveal that actually there is a consensus. This can't be solved by pre-splitting, it has to act as needed, maybe co-opting the tagging system, with the default tag being "Boostworthy" (but not "Relevant" or anything specific like that), ability to see the tags if you click something, and ability to tag your vote with anything (one tag per voter, so to give a specific tag you have to untag "Boostworthy", and all tags sum up into the usual karma score that is the only thing that shows by default until you click something). This has to be sufficiently inconvenient to only get used when necessary, but then somehow become convenient enough for everyone to use (for that specific comment). On the other hand there is Steam that only has approve/disapprove votes and gives vastly more useful quality ratings than most rating aggregators that are even a little bit more nuanced. So any good idea is likely to make things worse. (Though Steam doesn't have a zero equilibrium problem because the rating is the percentage of approve votes.)
Is it more important to see absolute or relative numbers of votes? To me it seems that if there are many votes, the relative numbers are more important: a comment with 45 upvotes and 55 downvotes is not too different from a comment with 55 upvotes and 45 downvotes; but one of them would be displayed as "-10 karma" and the other as "+10 karma" which seems different a lot. On the other hand, with few votes, I would prefer to see "+1 karma" rather than "100% consensus" if in fact only 1 person has voted. It would be misleading to make a comment with 1 upvote and 0 downvotes seem more representative of the community consensus than a comment with 99 upvotes and 1 downvote. How I perceive the current voting system, is that comments are somewhere on the "good -- bad" scale, and the total karma is a result of "how many people think this is good vs bad" multiplied by "how many people saw this comment and bothered to vote". So, "+50 karma" is not necessarily better than "+10 karma", maybe just more visible; like a top-level comment made immediately after writing the article, versus an insightful comment made three days later as a reply to a reply to a reply to something. But some people seem to have a strong opinion about the magnitude of the result, like "this comment is good, but not +20 good, only +5 good" or "this comment is stupid and deserves to have negative karma, but -15 is too low so I am going to upvote it to balance all those upvotes" -- which drives me crazy, because it means that some people's votes depend on whether they were among the early or late voters (the early voters expressing their honest opinion, the late voters mostly voting the opposite of their honest opinion just because they decided that too much agreement is a bad thing). Here is my idea of a very simple visual representation that would reflect both the absolute and relative votes. Calculate three numbers: positive (the number of upvotes), neutral (the magical constant 7), and negative (the
8Yoav Ravid2y
I think this comes from a place of also seeing karma as reward/punishment, and thinking the reward/punishment is enough/too high, or from a place of seeing the score as representing where it should be relative to other comments, or just from trying to correct for underratedness/overratedness.  I sometimes do this, and think it's alright with the current voting system, but I think it's a flaw of the voting system that it creates this dynamic.
7[DEACTIVATED] Duncan Sabien2y
This is a misconstrual.  The late voters are also expressing their honest opinion, it's just that their honest opinion lies on a policy level rather than a raw stimulus-response level. It's at least as valid (and, I suspect, somewhat more valid) to have preferences of the form "this should be seen as somewhat better than that" than to have preferences of the form "I like this and dislike that."
6Yoav Ravid2y
This is interesting and I would like to see a demo of it. The upside to this suggestion is that since it's only a visual change and doesn't actually change the way karma and voting works, it could get tested and reverted very easily, just needs to be built. 
It could even be displayed to the right from the current karma display, so we could temporarily have both, like this:
5Yoav Ravid2y
Interesting, this gave me an idea for something a bit different. We'll have a list of good attributes a comment can have (Rigor, Effort, Correctness/Accuracy/Precision, Funny, etc.). By default you would have one attribute (perhaps 'Relevant'), and users will be able to add whichever attributes they want (perhaps even custom ones). These attributes will be voteable by users (no limit on how many you can vote on), and will show at the top of the comment together with their score (sorted by absolute value). I'm not sure how it would be used to sort comments or give points to users, though.
3[DEACTIVATED] Duncan Sabien2y
(I expect that having written this post + being friendly with much of the team will result in me being a part of some conversations on this in the near future; if there are summaries I can share here and otherwise it would be a long time before word got out, I'll try to do so.)

While I am not technically a "New User" in the context of the age of my account, I comment very infrequently, and I've never made a forum-level post. 

I would rate my own rationality skills and knowledge at slightly above the average person but below the average active LessWrong member. While I am aware that I possess many habits and biases that reduce the quality of my written content, I have the sincere goal of becoming a better rationalist. 

There are times when I am unsure whether an argument or claim that seems incorrect is flawed or if it is my reasoning that is flawed. In such cases, it seems intuitive to write a critical comment which explicitly states what I perceive to be faulty about that claim or argument and what thought processes have led to this perception. In the case that these criticisms are valid, then the discussion of the subject is improved and those who read the comment will benefit. If the criticisms are not valid, then I may be corrected by a response that points out where my reasoning went wrong, helping me avoid making such errors in the future.

Amateur rationalists like myself are probably going to make mistakes when it comes to criticism of other people's written content, even when we strive to follow community guidelines. My concern with your suggestions is that these changes may discourage users like me from creating flawed posts and comments that help us grow as rationalists. 

I think there's a real danger of that, in practice.

But I've had lots of experience with "my style of moderation/my standards" being actively good for people taking their first steps toward this brand of rationalism; lots of people have explicitly reached out to me to say that e.g. my FB wall allowed them to do just those sorts of first, flawed steps.

A big part of this is "if the standards are more generally held, then there's more room for each individual bend-of-the-rules."  I personally can spend more spoons responding positively and cooperatively to [a well-intentioned newcomer who's still figuring out the norms of the garden] if I'm not also feeling like it's pretty important for me to go put out fires elsewhere.

5[DEACTIVATED] Duncan Sabien2y
Or in other words, that's part of what I was clumsily gesturing at with "Cooperate past the first 'defect' from your interlocutor."  I should've written "first apparent defect."
"If you can find me people capable of being these moderators, I will hire them. I think the number of people who have mastered the standards you propose and are also available is...smaller than I have been able to locate so far." I think the best way to do this would be to ask people to identify a few such comments and how they would have rewritten the comment.
If I may add something, I wish users occasionally had to explain or defend their karma votes a bit. To give one example that really confuses me, currently the top three comments on this thread are: 1. a clarification by OP (Duncan) - makes sense 2. a critical comment which was edited after I criticized it; now my criticism is at ~0 karma, without any comments indicating why. This would all be fine, except the comment generated no other responses, so now I don't even understand why I was the only one who found the original objectionable, or why others didn't like my response to it; and I don't remotely understand the combination of <highly upvoted OP> and <highly upvoted criticism which generates no follow-up discussion>. (Also, after a comment is edited, is there even a way to see the original? Or was my response just doomed to stop making sense once the original was edited?) 3. another critical comment, which did generate the follow-up discussion I expected (EDIT: Have fixed broken links.)
(You've done good work in this post's comment section, IMO.) Maybe if a comment were required in order to strongly upvote or strongly downvote? As someone who does those things fairly often, I wouldn't hate this change. Sitting here imagining a comment I initially wanted to strongly upvote but didn't because of such a rule, I feel okay about the fact that I was deterred, given this site's standards. Or maybe a 1 in 3 chance that a strong upvote will require a comment.
I think this is usually done subconsciously -- people are more motivated to find issues with arguments they disagree with.
3Yoav Ravid2y
How you would test if someone fits the criteria? Can those people be trained?
4[DEACTIVATED] Duncan Sabien2y
I do in fact claim that I could do some combination of identify and train such people, and I claim that I am justified and credible in believing this about myself based on e.g. my experience helping CFAR train mentors and workshop instructors.  I mention this somewhat-arrogant belief because Ruby highlighted me in his comment as someone who might be able to help on related matters. 
5Yoav Ravid2y
Sorry, I think my comment came across as casting doubt on that, but my intention was to ask out of genuine curiosity and interest. I wonder how the process/test might look like.
4[DEACTIVATED] Duncan Sabien2y
Double sorry!  I didn't read you as casting doubt, and didn't have defensive feelings while writing the above. I do believe people can be trained, though I have a hard time boiling down either "how" or why I believe it. As for how to test, I don't have a ready answer, but if I develop one I'll come back and note it here.  (Especially if e.g. Ruby asks me for help and I actually deliver.)
2Yoav Ravid2y
Alright, thanks :) (If anyone else has Ideas I'd be glad to hear them)

I think that smart people can hack LW norms and propagandize / pointscore / accumulate power with relative ease. I think this post is pretty much an example of that:
- a lot of time is spent gesturing / sermoning about the importance of fighting biases etc. with no particularly informative or novel content (it is after all intended to "remind people of why they care".). I personally find it difficult to engage critically with this kind of high volume and low density. 
- ultimately the intent seems to be an effort to coordinate power against types of posters that Duncan doesn't like

I just don't see how most of this post is supposed to help me be more rational. The droning on makes it harder to engage as an adversary, than if the post were just "here are my terrible ideas", but it does so in an arational way.

I bring this up in part because Duncan seems to be advocating that his adherence to LW norms means he can't just propagandize etc.

If you read the OP and do not choose to let your brain project all over it, what you see is, straightforwardly, a mass of claims about how I feel, how I think, what I believe, and what I think should be the case.

I explicitly underscore that I think

... (read more)

If I'm reading you correctly, it sounds like there's actually multiple disagreements you have here--a disagreement with Duncan, but also a disagreement with the current norms of LW.

My impression is primarily informed by these bits here:

I think that smart people can hack LW norms and propagandize / pointscore / accumulate power with relative ease. [...]

If people here really think you can't propagandize or bad-faith accumulate points/power while adhering to LW norms, well, I think that's bad for rationality.

Could you say more about this? In particular, assuming my reading is accurate, I'm interested in knowing (1) ways in which you think the existing norms are inadequate to the task of preventing bad-faith point-scoring, (2) whether you think it's possible to patch those issues by introducing better norms. (Incidentally, if your answer to (2) is "yes", then it's actually possible your position and Duncan's are less incompatible than it might first appear, since you might just have different solutions in mind for the same problem.)

(Of course, it's also possible you think LW's norms are irreparable, which is a fact that--if true--would still be worth drawing attention to, even if it is somewhat sad. Possibly this is all you were trying to do in the parent comment, in which case I could be reading more into what you said than is there. If that's the case, though, I'd still like to see it confirmed.)

Maybe it is good to clarify: I'm not really convinced that LW norms are particularly conducive to bad faith or psychopathic behavior. Maybe there are some patches to apply. But mostly I am concerned about naivety. LW norms aren't enough to make truth win and bullies / predators lose. If people think they are, that alone is a problem independent of possible improvements. 

since you might just have different solutions in mind for the same problem.

I think that Duncan is concerned about prejudicial mobs being too effective and I am concerned about systematically preventing information about abuse from surfacing. To some extent I do just see this as a conflict based on interests -- Duncan is concerned about the threat of being mobbed and advocating tradeoffs accordingly, I'm concerned about being abused / my friends being abused and advocating tradeoffs accordingly. But to me it doesn't seem like LW is particularly afflicted by prejudicial mobs and is nonzero afflicted by abuse.

I don't think Duncan acknowledges the presence of tradeoffs here but IMO there absolutely have to be tradeoffs. To me the generally upvoted and accepted responses to jessicata's post are making a tradeo... (read more)

I like this highlighting of the tradeoffs, and have upvoted it. But:

But to me it doesn't seem like LW is particularly afflicted by prejudicial mobs and is nonzero afflicted by abuse.

... I think this is easier to say when one has never been the target of a prejudicial mob on LessWrong, and/or when one agrees with the mob and therefore doesn't think of it as prejudicial.

I've been the target of prejudicial mobbing on LessWrong.  Direct experience.  And yes, it impacted work and funding and life and friendships outside of the site.

I was not aware of any examples of anything anyone would refer to as prejudicial mobbing with consequences. I'd be curious to hear about your prejudicial mobbing experience.
3[DEACTIVATED] Duncan Sabien2y
I think it's better (for the moment at least) to let Oliver speak to the most salient one, and I can say more later if need be.  I suspect Oliver would provide a more neutral POV.

propagandize / pointscore / accumulate power with relative ease

There's a way in which this is correct denotatively, even though the connotation is something I disagree with.  Like, I am in fact arguing for increasing a status differential between some behaviors that I think are more appropriate for LW and others that I think are less appropriate.  I'm trying at least to be up front about what those behaviors are, so that people can disagree.  e.g. if you think that it's actually not a big deal to distinguish between observation and inference, because people already do a good job teasing those apart.

But yes: I wouldn't use the "power" frame, but there's a way in which, in a dance studio, there's "power" to be had in conforming to the activity of dance and dance instruction, and "less power" in the hands of people not doing those things.

an effort to coordinate power against types of posters that Duncan doesn't like

I don't think this is the case; I want to coordinate power against a class of actions.  I am agnostic as to who is taking those actions, and even specifically called out that if there are people who are above the local law we should be candid about that ... (read more)

If you do happen to feel like listing a couple of underappreciated norms that you think do protect rationality, I would like that.



6[DEACTIVATED] Duncan Sabien2y
Strong upvote. (I think the norms I'm pulling for increase brevity; more consistent standards mean less need to bend over backwards ruling out everything else in each individual case.)

Your OP is way too long (or not sufficiently indexed) for me to, without considerable strain, determine how much or how meaningfully I think this claim is true. Relatedly I don't know what you are referring to here.

All of which is to say that I spend a decent chunk of the time being the guy in the room who is most aware of the fuckery swirling around me, and therefore the guy who is most bothered by it.

I feel like this claim is being used as evidence to bolster a call to action, without being well supported.  Unsupported evidence is always worse than supported, but it's especially grating in this case because providing counter-evidence can reasonably be predicted to be interpreted as a social attack, which people have inhibitions against making (and then when do they do get made, come out in really unproductive ways). I would prefer a norm where, if you're going to claim raw or relative intelligence as a reason to believe you, you need to provide at least a link to evidence.

6[DEACTIVATED] Duncan Sabien2y
That's a fine norm.  What sort of evidence would you be interested in, in this case? I can more easily provide copious evidence of me being aware of fuckery that others aren't (e.g. 100 FB posts over the last year, probably) than of high-enough-intelligence-to-justify "decent chunk."  But I could e.g. dig up my SAT scores or something. I expect that people are sort of adversarially misconstruing "decent chunk" as a claim of something like "the vast majority of the time," or whatever, which I grant I left myself open to but is not a claim I would make (since it isn't true).

I should note that my interest in sharing the FB posts is that they render “Duncan’s claim to rare insight” discussable. I don’t expect them to move my opinion much because we’re FB friends, and I see many of them already.

Now that you’ve acknowledged that this is fair to assess in its own right, I’d like to share my current assessment of your insight levels:

  • You have at least once said things I found extremely novel and useful, although still had major disagreements with. I shared the first part with you privately, although did not update you when I developed more concerns about the model.
  • You have at least once waged a major campaign alone at great social cost, that I appreciate a great deal and respected a lot. I already shared this one privately with you but it seems worth noting in public.
  • Your FB is a mix of things I agree with and disagree with. Some of the disagreements I might change my mind on if we could have a good discussion, and under other circumstances I would be really excited to have those discussions, because I care about this a lot too and people paying sufficient attention are rare. But I (almost?) never do, because just the thought makes me feel weary.
... (read more)
6[DEACTIVATED] Duncan Sabien2y
I do not have, and do not believe I have claimed to have, anything like "certainty in the superiority of my insight."  Happy to just state here explicitly: I don't have anything like certainty in the superiority of my insight. What I have is confidence that, when I'm perceiving that something is going sideways, something is, in fact, going sideways. That's a far cry from always knowing what it is, which is itself a far cry from having any idea how to fix it. I'm confused as to how I'm perceived as claiming superiority of insight when e.g. all I could come up with in the above essay was a set of ideas that I myself identified as terrible and insufficient.
6Stephen Bennett2y
My comment is low context both because I don’t think I’ve seen you and Elizabeth talk before and also because I only skimmed the parent comments. When you say This doesn’t seem to me evidence against you claiming to have superior insight under Elizabeth’s usage of the term. My reading is that she uses the term relatively, I.e. that she believes that you believe your claims about the world are right while others’ claims are wrong (or more likely to be true than others’). Terrible, as you used it in the essay, I took to be in absolute terms, as in “will these interventions help? Idk” I’m more confident in the quote not providing evidence against her usage of “superior insight” than I am in defining her intended meaning.
I would indeed be interested in the FB posts, with the caveat that those then become debatable, and it seems quite possible that that will overwhelm comments on the topic of this post. I also think it would help to declare how you intend to handle people bringing up statements and actions (public or private) not of your choosing. I think deluks handled his comment extremely poorly and I would have done it very differently,* but I do think your claim makes "every bad decision you ever made" relevant, and the overall track record of threads on every bad decision a person ever made is extremely bad. I think the FB posts are better evidence of "being the one to speak up" than "the only one noticing". That's not a criticism: speaking up is really important, and doing it when it matters is a huge service. People who notice but don't act aren’t very helpful. But it is a different claim *I can get into how if you are curious but am worried deluks has poisoned the well on reasonable discussion of that problem. 
5[DEACTIVATED] Duncan Sabien2y
I assume by "very differently," you mean "would not have included outright falsehoods." I don't see the connection between: (approximately) "I'm pretty smart and I am pretty conscientious and also I've hyperfocused on this domain for a long time, so I notice stuff in this domain a lot more often than most people" and [whatever claim you and deluks think I made that is tantamount to an assertion of perfection, or something] Like, it seems that you're asking me to defend some outlandish claim that I have not made.  That's the only situation in which "every bad decision I ever made" would be relevant—if I had claimed to always or overwhelmingly often be competent, or something.  It sounds like "this guy claimed no white ravens!  All we gotta do is find one!" I clearly did not make that claim, as you can see by scrolling up and just reading.  I claimed that I twitch over this stuff, and frequently observe fuckery that other people do not even notice.  
7Said Achmiz2y
FYI, every single one of these posts (yes, I tested all the links) is inaccessible to me, because they require logging into Facebook. (I’m posting this to note that this isn’t a problem specific to that one other post, but seems to be a general problem.)
8Yoav Ravid2y
I was able to see 5-6 in firefox without incognito, and then it asked me to log in (both on ones I already saw and ones I didn't). Seems like some sort of "You have 3 more articles this month" tactic but without telling you.
Are you opening them in incognito browsers? They seem to work straightforwardly for me in non-logged-in browsers and don't know what might be different for you.
2Ben Pace2y
This is groundhog day Ray; we just found out that it doesn't work on Opera and Firefox. (And apparently Chrome Incognito on Windows? I'm confused about the exact line there, because it works on my Chrome Incognito on Mac.)
5Said Achmiz2y
So far, this problem has replicated on every browser on every platform I’ve tried it on, in both regular and private windows. Chrome, Firefox, Opera, on Mac, Windows, Linux… I have not been able to view any of the given posts in any way at all.
  The first of many differences, yes. I also would have emphasized the part where I thought the evidence something was wrong was obvious and you didn't (in ways that were visible to me), not the part where you proactively coordinated evidence sharing when it was personally costly to you, which was a social good. I did not think you were claiming perfection, but I think "It's like being a native French speaker and dropping in on a high school French class in a South Carolina public school" is a very strong claim of superiority, far beyond "I notice stuff in this domain a lot more often than most people". Native speakers can be wrong, but in a disagreement with a disengaged high schooler you will basically always take the word of the native speaker.  I additionally think the problems in iteration I outlined in a sister thread really put a ceiling on your insights, although admittedly that affects analysis and improvements much more than noticing.
7[DEACTIVATED] Duncan Sabien2y
Also, re: evidence that something was wrong was obvious I dunno.  This sounds like an excuse, and an excuse is all that many people will hear, but: My current model is that Brent, whether consciously or unconsciously/instinctively, did in fact do something resembling cultivating me as a shield, by never egregiously misbehaving in my sight.  And many of the other people around me, seeing egregious misbehavior somewhat often, assumed (reasonably) that I must be seeing it, too, and not minding. But after it all started to come out, there were something like a dozen fully dealbreaking anecdotes handed to me by not-necessarily-specifically-but-people-in-the-reference-class-of Rob, Oli, Nate, Logan, Nick, Val, etc., any one of which would have caused me to spring into action, except they just never mentioned it and I was never in the room to see it.
FWIW: I believe you that Brent cultivated you, and I think you talking about that has been really useful in educating people (including me) about how toxic people do that. I do think it had to be some damn strong cultivation to overcome the baseline expectations set by his FB posts, and I'd be interested in hearing you talk about what he did to overcome that baseline- not because I think you were especially susceptible, but because whatever he did worked on a lot of people, and that makes it useful to understand.

Well, for starters, I had unfollowed him on FB by about 2016 as a result of being just continually frustrated by his relentless pessimism.  So I probably missed a whole lot of what others saw as red flags.

This indeed changes my opinion a fair bit, and I should have had it as a more active hypothesis.
4[DEACTIVATED] Duncan Sabien2y
I find this helpful, and I think it's a fair and reasonable reading that I should have ruled out. What I meant by choosing that example in particular was that French contains a lot of sounds which English speakers literally can't perceive at first, until they practice and build up some other background knowledge.  That's ... not entirely different from a claim of superiority, but I tried to defuse the sense of superiority by noting that a lot of it comes from just relentlessly attending to the domain—"it's not that I'm doing anything magic here, many of the people I'm hanging out with are smarter or conscientiouser, it's just that I happen to have put in more reps is all." It didn't work.
Ah, this makes sense and is helpful, and now that you've spelled it out I can see how it connects to other things in the post in ways I didn't before. It also makes cases of failure much less relevant, since no one has all phenomes. Worth noting that I noticed the kerning example seemed very different than the native speaker example, but the "native speaker in a room full of bored teenagers" claim felt so strong I resolved in that direction.
(I agree that there's an issue here with bucketing counter-evidence with social attack, and that a norm of providing evidence is preferable to a silent bucket error; also, it seems likely that there are belief-like claims that (1) are useful to make, even though at least by default they're mixed in with social moves, and that (2) are difficult (costly, say) to provide evidence for. If such claims are common, it might be worth having an epistemic status that can contain that nuance, something like "this claim is acknowledged to be too costly for many pairs of people to reach Aumann agreement on, and shouldn't yet function as a common knowledge belief among groups containing many such pairs, but it's still what I think for what that's worth". Maybe there's a short phrase that already means this, like "from my perspective...", though sadly such things are always diluting.)

Summary: I found this post persuasive, and only noticed after the fact that I wasn't clear on exactly what it had persuaded me of. I think it may do some of the very things it is arguing against, although my epistemic status is that I have not put in enough time to analyze it for a truly charitable reading. 

Disclaimer: I don't have the time (or energy) to put in the amount of thought I'd want to put in before writing this comment. But nevertheless, my model of Duncan wants me to write this comment, so I'm posting it anyway. Feel free to ignore it if it's wrong, useless, or confusing, and I'm sorry if it's offensive or poorly thought out! 

Object-level: I quite liked your two most recent posts on Concentration of Force and Stag Hunts. I liked them enough that I almost sent them to someone else saying "here's a good thing you should read!" It wasn't until I read the comment below by SupposedlyFun that I realized something slightly odd is going on and I hadn't noticed it. I really should have noticed some twinge in the back of my mind on my own, but it took someone else pointing it out for me to catch it. I think you might be guilty of the very thing you're complaining about ... (read more)

Hmm, my two-sentence summary attempt for this post would be: "In recent drama-related posts, the comment section discussion seems very soldier-mindset instead of scout-mindset, including things like up- and down-voting comments based on which "team" they support rather than soundness of reasoning, and not conceding / correcting errors when pointed out, etc. This is a failure of the LW community and we should brainstorm how to fix it."

If that's a bad summary, it might not be Duncan's fault, I kinda skimmed.

Strong upvoted for the effort it takes to write a short, concise thing.  =P

I endorse this as a most-of-it summary, though I think the details matter.

I found this post persuasive, and only noticed after the fact that I wasn't clear on exactly what it had persuaded me of.

I want to affirm that this to me seems like it should be alarming to you. To me a big part of rationality is about being resilient to this phenomenon and a big part of successful rationality norms is banning the tools for producing this phenomenon.

6[DEACTIVATED] Duncan Sabien2y
It is indeed a concern. The alarm is a bit tempered by the fact that this doesn't seem to be a majority view, but "40% of readers" would be deeply problematic and "10% of readers" would still probably indicate some obvious low-hanging fruit for fixing a real issue. Looking at the votes, I don't think it's as low as 4% of readers, which is near my threshold for "no matter what you do, there'll be a swath this large with some kind of problem."
9[DEACTIVATED] Duncan Sabien2y
I do think I'm doing something (in this post specifically) that might be accurately described as "dropping down below the actual level of rigor, to meet the people who are WAY below the level of rigor halfway, and try to encourage them to come up."  I've made an edit to the author's note now that you've helped me notice this. I think my overall model is something like "there are some folk who nominally support the norms but grant themselves exceptions too frequently in practice, and there are some folk who don't actually care all that much about the norms, or who subordinate the norms to something else." (Where "the norms" is the stuff covered in the Sequences and SlateStarCodex and in the lists of things my brain does in the post above and so forth.) And I think I'm arguing that we should encourage the former to better adhere to their own values, and support them in doing so, and maybe disincentivize or disinvite (some fraction or subset) of the latter. re: "But without following a number of the linked comments, I can't say exactly what you think went wrong," I'm happy to detail an example or two, if anyone wants to say "hey, what's wrong with this??" though part of why I didn't detail a large number of them in the OP is that I don't have the spoons for it.

My brain notes, in passing, that Eternal September, and the Septembers that preceded it, can be described in terms of concentration of force.  If a forum has a certain culture, and a bunch of noobs come in without it (instead exhibiting some kind of "mainstream lowest-common-denominator" culture, incompatible with that of the forum)... If they come in one by one, then they'll face negative reinforcement (downvotes and/or critical comments), pushing them to either adopt the forum's culture or leave; if they arrive in a clump, then they'll be in a position to positively support each other, reducing the negative-reinforcement effect.  If enough of them arrive in a group, then the forum's "immune system" may fail to stop them, and they may end up changing the forum's culture.

Today, a sudden influx of users tends to come from some big event.  Like when a hugely popular site drops a link to your forum, or when there's a huge story on your forum that draws a lot of interest from outsiders.  (The Leverage story is one of the latter; a story like Zoe Curzi's is fascinating to humans.  I told a non-rationalist friend about it, and he said it was very juicy gossip. &n... (read more)

4[DEACTIVATED] Duncan Sabien2y
(Because it wasn't a terrible idea. =P)

First of all, thank you, Duncan, for this post. I feel like it captures important perspectives that I've had, and problems that I can see and puts them together in a pretty good way. (I also share your perspective that the post Could Be Better in several ways, but I respect you not letting the perfect be the enemy of the good.)

I find myself irritated right now (bothered, not angry) that our community's primary method of highlighting quality writing is by karma-voting. It's a similar kind of feeling to living in a democracy--yes, there are lots of systems that are worse, but really? Is this really the best we can do? (No particular shade on Ruby or the Lightcone team--making things is hard and I'm certainly glad LW exists and is as good as it is.)

Like, I think I have an idea that might make things substantially better that's not terrible: make the standard signal for quality being a high price on a quality-arbitrated betting market. This is essentially applying the concept of Futarchy to internet forums (h/t ACX and Hanson). (If this is familiar to you, dear reader, feel free to skip to responses to this comment, where I talk about features of this proposal and other ideas.) Here's ... (read more)

To my mind the primary features of this system that bear on Duncan's top-level post are: * High-reputation judges can confidently set the quality signal for a piece of writing, even if they're in the minority. The truth is not a popularity contest, even when it comes to quality. * The emphasis on betting means that people who "upvote" low-quality posts or "downvote" high-quality ones are punished, making "this made me feel things, and so I'm going to bandwagon" a dangerous mental move. And people who make this sort of move would be efficiently sidelined. In concert, I expect that it would be much easier to bring concentrated force down on low-quality bits of writing. Which would, in turn, I think make the quality price/signal a much more meaningful piece of information, instead of the current karma score which is as others noted, is overloaded as a measure.

I like this idea. It has a lot of nice attributes. 

I wrote some in the past about what all the different things are that a voting/karma system on LW is trying to produce, with some thoughts on some proposals that feel a bit similar to this: 

Nice. Thank you. How would you feel about me writing a top-level post reconsidering alternative systems and brainstorming/discussing solutions to the problems you raised?
Seems great! It's a bit on ice this week, but we've been thinking very actively about changes to the voting system, and so right now is the right time to strike the iron if you want to change the teams opinion on how we should change things, and what we should experiment with.
6Yoav Ravid2y
I think this is too complex for a comment system, but upvoted for an interesting and original idea. 
8Ben Pace2y
My sense is that the basic UI interaction of "look at a price and judge it as wrong" has the potential to be surprisingly simple for a comment section. I often have intuitions that something is "overpriced" or "underpriced". But I find the grounding-out process pretty hard to swallow. I'd be spending so much of my time thinking about who was grounding it out and how to model them socially, which is a far more costly operation than my current one that's just "do I think the karma number should go up or down".
4Ben Pace2y
But also strong upvoted for an exciting and original idea.
One obvious flaw with this proposal is that the quality-indicator would only be a measure of expected rating by a moderator. But who says that our moderators are the best judges of quality? Like, the scheme is ripe for corruption, and simply pushing the popularity contest one level up to a small group of elites. One answer is that if you don't like the mods, you can go somewhere else. Vote with your feet, etc. A more turtles-all-the-way-down answer is that the stakeholders of LW (the users, and possibly influential community members/investors?) agree on an aggregate set of metrics for how well the moderators are collectively capturing quality. Then, for each unit of time (eg year) and each potential moderator, set up a conditional prediction market with real dollars on whether that person being a moderator causes the metrics to go up/down compared to the previous time unit. Hire the ones that people predict will be best for the site.

I guess the question is, what is the optimal amount of consensus. Where do we want to be, on the scale from Eternal September to Echo Chamber?

Seems the me that the answer depends on how much correct we are, on average. To emphasise: how much correct we actually are, not how much correct we want to be, or imagine ourselves to be.

On a website where moderators are correct about almost everything, most disagreement is a noise. (It may provide a valuable feedback on "what other people believe", but not on how things actually are.) It is okay to punish disagreement, because in the rare situations where it is correct and you notice it, you can afford the karma hit for opposing the moderators. (And hopefully the moderators are smart enough to start paying attention when a member in good standing surprisingly decides to take a karma hit.)

On a website where moderators are quite often wrong, punishing disagreement means that the community will select for people who share the same biases, or who are good at reading the room.

I believe that people are likely to overestimate how much "other reasonable people" agree with them, which is why echo chambers can happen to people who genuinely see thems... (read more)

I also want to note that this proposal isn't mutually exclusive with other ideas, including other karma systems. It seems fine to have there be an additional indicator of popularity that is distinct from quality. Or, more to my liking, would be a button that simply marks that you thought a post was interesting and/or express gratitude towards the writer, without making a statement about how bulletproof the reasoning was. (This might help capture the essence of Rule Thinkers In, Not Out and reward newbies for posting.)

First, I think promoting and encouraging higher standards is, if you'll pardon the idiom, doing God's work. 

Thank you. 

I'm so appreciative any time any member of a community looks to promote and encourage higher standards. It takes a lot of work and gets a lot of pushback and I'm always super appreciative when I see someone work at it.

Second, and on a much smaller note, if I might offer some......... stylistic feedback?

I'm only speaking here about my personal experience and heuristics. I'm not speaking for anyone else. One of my heuristics — which I darn well know isn't perfectly accurate, but it's nevertheless a heuristic I implicitly use all the time and which I know others use — is looking at language choices made when doing a quick skim of a piece as a first-pass filter of the writer's credibility.

It's often inaccurate. I know it. Still, I do it.

Your writing sometimes, when you care about an issue, seems to veer very slightly into resembling the writing of someone who is heated up about a topic in a way that leads to less productive and coherent thought.

This leads my default reaction to discounting the credibility of the message slightly.

I have to forcibly remind myse... (read more)

9[DEACTIVATED] Duncan Sabien2y
Thanks. =)

Some thoughts on resource bottlenecks and strategy.

There's a lot I like about the set of goals Duncan is aiming for here, and IMO the primary question is one of prioritization.

I do think some high-level things have changed since 2018-or-so. Back when I wrote Meta-tations on Moderation, the default outcome was that LW withered and died, and it was really important people move from FB to LW. Nowadays, LW seems broadly healthy, the team has more buy-in, and I think it's easier to do highly opinionated moderation more frequently for various reasons.

On the other hand, we did just recently refactor the LW team into Lightcone Infrastructure. Most of the team is now working on a broader project of "figure out the most important bottlenecks facing humanity's ability to coordinate on x-risk, and building things that fix that bottleneck" (involving lots of pivoting). Ruby is hiring more people to build more capacity on the LW team but hiring well is a slow process. And most of the plans that seem to accomplish (some version of) what Duncan is pointing to here seem really expensive.

The good news is that we're not money-constrained much these days. The biggest bottlenecked resource is team-atte... (read more)

Something that was previously seemed some-manner-of-cruxy between me and Duncan (but I'm not 100% sure about the flavor of the crux) is "LessWrong who's primary job is to be a rationality dojo" vs "LessWrong who's primary job is to output intellectual progress." 

Where, certainly, there's good reason to think the Intellectual Progress machine might benefit from a rationality dojo embedded in it. But, that's just one of the ideas for how to improve rate-of-intellectual progress. And my other background models point more towards other things as being more important for that.

BUT there is a particular model-update I've had that is new, which I haven't gotten around to writing up yet. (This is less of a reply to Duncan and more to other people I've argued with over the years)

A key piece of my model is that a generative intellectual process looks very different from the finished output. It includes lots of leaps of intuition, inferential distance, etc. In order to get top-thinkers onto LW on a regular basis rather than in small private discords, it's really important for them to be able to think-out-loud without being legible at every step here. And the LW team got a lot of complaint... (read more)

3[DEACTIVATED] Duncan Sabien2y
Strong agreement with this, assuming I've understood it.  High confidence that it overlaps with what Vaniver laid out, and with my interpretation of what Ben was saying in the recent interaction I described under Vaniver's comment. EDIT: One clarification that popped up under a Vanvier subthread: I think the pendulum should swing more in the direction laid out in the OP.  I do not think that the pendulum should swing all the way there, nor that "the interventions gestured at by the OP" are sufficient.  Just that they're something like necessary.

Small addition: LW 1.0 made it so you had to have 10 karma before making a top-level post (maybe just on Main? I don't remember but probably you do). I think this probably matters a lot less now that new posts automatically have to be approved, and mods have to manually promote things to frontpage. But I don't know, theoretically you could gate fraught discussions like the recent ones to users above a certain karma threshold? Some of the lowest-quality comments on those posts wouldn't have happened in that case.

I guess where I'd like to see more moderator intervention would largely be in directing the conversation. For example, by creating threads for the community to discuss topics that you think it would be important for us to talk about.

I think you are getting at something here, Duncan. I've become interested in the following question lately: "How should rationalists conduct themselves if their goal is to promote rationality?" Now, I understand that promoting rationality is not every rationalist's top priority, hence I stated that condition explicitly.

I've been thoroughly impressed by how Toby Ord conducts himself in his writings and interviews. He is kind, respectful, reassuring and most importantly, he doesn't engage in fear-mongering despite working on x-risks. In his EA interview, he said, "Let us not get into criticising each other for working on the second most important thing." I found this stunningly thoughtful and virtuous. I find this to be an excellent example of someone going about achieving their goals effectively.

As much as I like Dawkins and love his books, I will admit that his attitude is sometimes unhelpful towards his own goals. I recall hearing (forgot where) that before a debate on spirituality, Dawkins' interlocutor asked him to read some documents ahead of the discussion. Dawkins showed up having not read the papers and said, "I did not read your documents because I know they are wrong." [ci... (read more)

Suggestions for LW features that could shape its culture by (dis)incentivizing certain behavior (without thinking about how hard they would be to implement):

On How LW Appears to Outside Readers

There's a certain kind of controversial post that inevitably generates meta-discussion of whether people should be allowed to post it here (most recently in this book review). Crucially, the arguments I see there are not "I don't like this" but usually "I'm afraid of what will happen when people who don't like this see it, and associate LW with it". (I found this really tedious, and would prefer a culture where we stick our own heads out and stick to "I don't like this" rather than appealing to third parties.) Also, I wish there were a way to preempt this objection, so as to not fight the same battle over and over.

Other posts are in dispute (as in, have an unusually high fraction of downvotes), like jessicata's post, but a casual reader might only see the post (and maybe its positive karma score), with all the controversy and nuance happening in the utterly impenetrable comments section.

So what might one do about that?

  • Some Reddit-style sites compute a controversy score (percentage of downvote
... (read more)
9Yoav Ravid2y
First of all, thanks for your three comments, I think they provide valuable analysis and suggestions. Another thing I notice about controversial posts: Because of how the front page works, posts that get a lot of comments get more exposure, because they show up in recent discussion, while posts that are correct and valuable but uncontroversial are likely to get less comments (even if everyone who upvotes them left a simple "good post" comment) unless they somehow manage to generate discussion. I'm not entirely sure if it a "bug" or a "feature". On the one hand, posts that are agreed to be good and valuable get drowned out, on the other hand, perhaps it's exactly the controversial posts that deserve attention to resolve the controversy? One way to counteract that is to leave more simple, generic comments like "Thanks for writing this", "This was great", "I enjoyed reading this", etc. People (including me sometimes when I consider making them) worry about not adding anything substantial, but I think that's not a problem, I like seeing these comments and the Karma system should help get the more substantial comments near the top.  That's a social suggestion though, rather than a feature that can be implemented in the website. I don't have an idea for a feature that could deal with that (the main ones currently are the 'magic' sorting on the frontpage, curation, and the frontpage recommendations, but I don't think the latter two have a big impact on this). I like the idea (Had it myself as well) of having a feature that lets users directly give monetary rewards to other users for comments and posts. One of the problems of a Reddit rewards style system is that it's still internet points at the end of the day, and there's a limitation for how much internet points can be worth.  On the other hand, money has obvious utility, and would definitely create an incentive to post very good comments and posts, and to give even more polish to very good comments and posts you w
On the one hand, I like the gesture of commenting nice but relatively empty stuff like "this was great". On the other hand I dislike spam, and this feels kind of redundant with the karma system. Not sure what I think about this. As one random example: I could pay 10$ to any comment which agrees with me, and to any comment which criticizes one of my own critics. I don't even need to say this explicitly, and yet over time it would still absolutely warp discussions. And what if a critic notices that and offers 20$ each? Then we're suddenly in an arms race. I'm not sure how to prevent failure modes like that.
4Yoav Ravid2y
Oh, well the way I imagined it the reward isn't visible or influences how comments are shown. So I guess it could hypothetically create an incentive to make comments that agree with you, but if it's not visible and has no direct influence that seems very unlikely. If it is then it's more likely, but It still doesn't seem like it would be a big problem. Maybe moderators can have automatic monitoring for that kind of thing like they have for mass up/down voting?

For what it's worth, I think some of those terrible ideas are great or close to great.

In particular:

  • Hire a team of well-paid moderators for a three-month high-effort experiment of responding to every bad comment with a fixed version of what a good comment making the same point would have looked like.  Flood the site with training data.
  • Make a fork of LessWrong run by me, or some other hopeless idealist that still thinks that there might be something actually good that we can get if we actually do the thing (but not if we don't).
  • Create an anonymous account with special powers called TheCultureCurators or something, and secretly give the login credentials to a small cadre of 3-12 people with good judgment and mutual faith in one another's good judgment.  Give TheCultureCurators the ability to make upvotes and downvotes of arbitrary strength, or to add notes to any comment or post à la Google Docs, or to put a number on any comment or post that indicates what karma TheCultureCurators believe that post should have.

Rob Bensinger wants me to note that he agrees.


The first one would be costly and annoying to lots of people but also time boxed and super interesting. Training ... (read more)

5[DEACTIVATED] Duncan Sabien2y
I like it, but not as much as I like two-axis proposals, which I think can be done with a smooth enough UI that they don't impose a burden. With something like the below, you can click to weak vote and hold to strong vote, just like we currently do, and can in one click express each of the four following positions: * I like/agree with this point, and furthermore think it's being expressed correctly/is in line with norms of reasoning and discourse I want to see more of on LW (dark blue) * I like/agree with this point, but I want to note objection with how it's being expressed/have reservations about whether it's good rationality or good discourse (pale orange) * I dislike/disagree with this point, and want to note objection with how it's being expressed (dark orange) * I dislike/disagree with this point, but want to endorse/support the way it was arrived at and expressed (pale blue)
Oh yeah, I've seen you post this before, I liked it!
2[comment deleted]2y

Nuance is the cost of precision and the bane of clarity. I think it's an error to feel positively about nuance (or something more specific like degrees of uncertainty), when it's a serious problem clogging up productive discourse, that should be burned with fire whenever it's not absolutely vital and impossible to avoid.

Uh.  I want to make a nuanced response here, distinguishing the difference between "feeling positively about nuance when it's net positive and negatively when its costs exceed its benefits, and trying to distinguish between the net positive case and the net negative case, and addressing the dynamics driving each" and so forth, but your comment above makes me hesitate.

(I also think this.)

EDIT: to clarify/sort-of-summarize, for those who don't want to click through: I think there's a compelling argument to be made that much or even the majority of intellectual progress lies in the cumulative ability to make ever-finer distinctions, i.e. increasing our capacity for nuance.  I think being opposed to nuance is startling, and in my current estimation it's approximately "being opposed to the project of LessWrong."  Since I don't believe that Vladimir is opposed to the project of LessWrong, I declare myself confused.

The benefits of nuance are not themselves nuance. Nuance is extremely useful, but not good in itself, and the bleed-through of its usefulness into positive affect is detrimental to clarity of thought and communication.

Capacity for nuance abstracts away this problem, so might be good in itself. (It's a capacity, something instrumentally convergent. Though things useful for agents can be dangerous for humans.)

I agree with Vladimir, FWIW.

In fact, this is one of the major problems I have with—forgive me for saying so!—your own posts. They are very nuanced! But this makes them difficult, sometimes almost impossible, to understand (not to mention very long); “bane of clarity” seems exactly right to me. (Indeed, I have noticed this tendency in the writing of several members of the LW team as well, and a few others.)

You say:

I think there’s a compelling argument to be made that much or even the majority of intellectual progress lies in the cumulative ability to make ever-finer distinctions, i.e. increasing our capacity for nuance.

There is certainly something to this view. But the counterpoint is that as you make ever finer distinctions, two trends emerge:

  1. The distinctions come to matter less and less—and yet, they impose at least constant, and often increasing, cognitive costs. But this is surely perverse! Cognitive resource expenditures should be proportional to importance/impact, otherwise you end up wasting said resources—talking, and thinking, more and more, about things that matter less and less…

  2. The likelihood that the distinctions you are making, and the patterns you are seeing,

... (read more)
2[DEACTIVATED] Duncan Sabien2y
I disagree with 1 entirely (both parts), and while 2 is sort of logically necessary, that doesn't mean the effect is as large as you imply with "increases dramatically," nor that it can't be overcome.  c.f. it's not what it looks like. (Reply more curt than usual for brevity's sake.  =P)
I think of robustness/redundancy as the opposite of nuance for the purposes of this thread. It's not the kind of redundancy where you set up a lot of context to gesture at an idea from different sides, specify the leg/trunk/tail to hopefully indicate the elephant. It's the kind of redundancy where saying this once in the first sentence should already be enough, the second sentence makes it inevitable, and the third sentence preempts an unreasonable misinterpretation that's probably logically impossible. (But then maybe you add a second paragraph, and later write a fictional dialogue where characters discuss the same idea, and record a lecture where you present this yet again on a whiteboard. There's a lot of nuance, it adds depth by incising the grooves in the same pattern, and none of it is essential. Perhaps there are multiple levels of detail, but then there must be levels with little detail than make sense out of context, on their own, and the levels with a lot of detail must decompose into smaller self-contained points. I don't think I'm saying anything that's not tiresomely banal.)
5Said Achmiz2y
Note that the linked content is inaccessible for those without a Facebook account.
7[DEACTIVATED] Duncan Sabien2y
...false?  I just opened it in an incognito window and it worked fine.  All my posts are public. But anyway here's the text:
6Said Achmiz2y
In a regular window (Firefox): In a private window (Firefox): In a regular window (Opera): In a private window (Opera): Firefox 78.5.0esr (Mac); Opera 80.0.4170.63 (Mac). EDIT: Tested also with Firefox 94.0.1 (Windows) and Chrome 95.0.4638.69 (Windows), with identical results. Your posts are not accessible without a Facebook account.
Huh, I see the post plus a big "log in" bar at the bottom on Safari 15612. (Mac), and the same without the bar in an incognito tab Chrome 94.0.4606.71 (Mac). These don't overlap with any of the things you tried, but it's strange to me that our results are consistently different.
I can no longer see it when not logged in, even though I did before. Maybe we triggered a DDoS mitigation thingie? Edit: Removed incorrect claim about how this worked (before seeing Said's response).
2Said Achmiz2y
No, this is not correct. All of my tests were conducted on a desktop (1080p) display, at maximum window width.
Yes, sorry, I got too excited about the absurd hypothesis supported by two datapoints, posted too soon, then tried to reproduce, and it no longer worked at all. I had the time to see the page in firefox incognito window on the same system where I'm logged in and in a normal firefox window from a different Linux username that never had facebook logged in. Edit: Just now it worked again twice, and after that it no longer did. Bottom line: Public facebook posts are not really public, at least today, they are only public intermittently.

There is a part of Sequences which I am too lazy to find now, which goes approximately like this: "If you make five maps of the same city, and you make those maps correctly, then the maps should be the same. So if you make five maps of the same city, and you find differences between them (for example, some streets A and B intersect on one map, but run parallel on another map), it means that you made a mistake somewhere, and the maps are not as good as you wish them to be. Nonetheless, you cannot fix this mistake by merely adjusting some of the maps to fit the other ones. The sameness of the maps is a desired outcome... but it must happen naturally, as a result of all maps correctly representing the same city... not artificially, as a result of adjusting the maps to fit each other."

I get a similar feeling from some of your posts (including this one, also the punch bug). It seems to me that you care a lot about being right; and that is a good thing, and it's kinda what this community is trying to be about. And you seem strongly frustrated by people coming to conclusions dramatically different from yours; which indeed means that someone is wrong about something. And I agree that this ... (read more)

My sense is that I am disagreeing with (a set of) specific things.

The bulk update that I'm pushing for is not "switch my opinion to everything Duncan says," but "start looking for ways to make the smaller, each-nameable-in-its-own-right slips in rationality happen less often."

I don't think I'm making a meta-argument about disagreement being wrong, except insofar as I'm asserting a belief that LessWrong ought to be for a specific thing, and that, in the case where there is consensus about that thing, other things should be deprioritized.  I'm not even claiming that I'm definitely right about the thing LW ought to be for!  But if it's about that thing, or chooses to become so, then it needs to be less about the other thing.

If we had a consensus about "this comment is more rational, and that comment is less rational", then reminding people to upvote the rational comments and downvote the irrational comments might result in karma scores that everyone would agree with. (Modulo the fact already mentioned somewhere in this discussion that some comments are seen by more people than other comments, which would still result in more karma for the same degree of rationality.) (Plus some other issues, such as: what if someone writes a comment containing one rational and one irrational paragraph; should we penalize needlessly long or hard-to-read comments; what if the comment is not quite good but contains a rare and important idea; etc.) Thing is, I don't believe we have this consensus. Some comments are obviously rational, some are obviously irrational, but there are many where different people have a different opinion. Technically, this can be measured. Like, find a person you believe to be so rational that you are satisfied with their level of rationality, who comments and votes on LW. Then find a long thread where you both voted, and check how many comments you upvoted/downvoted/ignored the same, and how many times you disagreed (not just upvote vs downvote, but also e.g. upvote vs no vote). My guess is that you overestimate how much your votes would match. My understanding of your complaint is that people are often voting on comments regardless of their rationality. Which certainly happens. But in a parallel reality where all of us consistently tried our best to really only vote for good arguments... I think you would assume much greater consensus in votes than I would.
Rationality doesn't make sense as a property of comments. It's a quality of cognitive skills that work well (and might generate comments). Any judgement of comments according to rationality of algorithms that generated them is an ad hominem equivocation, the comments screen off the algorithms that generated them.
8[DEACTIVATED] Duncan Sabien2y
Mmm, I think this is a mistake. I think that you're correct to point at a potential trap that people might slip into, of confusing the qualities of a comment with the properties of the algorithm that generated it.  I think this is a thing people do, in fact, do, and it's a projection, and it's an often-wrong projection. But I also think that there's a straightforward thing that people mean by "this comment is more rational than that one," and I think it's a valid use of the word rational in the sense that 70+ out of 100 people would interpret it as meaning what the speaker actually intended. Something like: * This is more careful with its inferences than that * This is more justified in its conclusions than that * This is more self-aware about the ways in which it might be skewed or off than that * This is more transparent and legible than that * This causes me to have an easier time thinking and seeing clearly than that ... and I think "thinking about how to reliably distinguish between [this] and [that] is a worthwhile activity, and a line of inquiry that's likely to lead to promising ideas for improving the site and the community."

I'm specifically boosting the prescriptivist point about not using the word "rational" in an inflationary way that doesn't make literal sense. Comments can be valid, explicit on their own epistemic status, true, relevant to their intended context, not making well-known mistakes, and so on and so forth, but they can't be rational, for the reason I gave, in the sense of "rational" as a property of cognitive algorithms.

I think this is a mistake

Incidentally, I like the distinction between error and mistake from linguistics, where an error is systematic or deliberatively endorsed behavior, while a mistake is intermittent behavior that's not deliberatively endorsed. That would have my comment make an error, not a mistake.

2[DEACTIVATED] Duncan Sabien2y
I like it.
6[DEACTIVATED] Duncan Sabien2y
I agree that the consensus doesn't exist. In part, that's why several of my suggestions depended on a small number of relatively concrete observables (like distinguishing inference from observation). But also, I think that a substantial driver of the lack of consensus/spread of opinion lies in the fact that the population of LessWrong today, in my best estimation, contains a lot of people who "ought not to be here," not in the sense that they're bad or wrong or anything, but in the sense that a gym ought mostly only contain people interested in doing physical activity and a library ought mostly only contain people interested in looking at books.  There is some number of non-central or non-bought-in members that a given population can sustain, and right now I think LessWrong is holding more than it can handle. I think a tighter population would still lack consensus in the way you highlight, but less so.
9Rana Dexsin2y
FWIW, I'm someone who believes myself to have the occasional useful contribution on LW, but I also have an intuitive sense of being “dangerously non-central” here, with the first word of that expanding to something like “likely to be welcomed anyway, but in a way which would do more collateral damage to community alignment (via dilution) than is broadly recognized in a way that people are willing to act on”. I apply a significant amount of secondary self-restraint on those grounds to what I post, possibly not enough (though my thoughts about what an actually appropriate strategy would be to apply here are too muddled to say that with confidence), and my emotional sense endorses my use of this restraint (in particular, it doesn't cause noticeable feelings of hostility or rejection in either direction). I'm saying this out loud partly in case anyone else who's had similar first-person experiences would otherwise feel awkward about describing them here and therefore result in a cluster of evidence being missing; I don't know how large that group would be.
4Said Achmiz2y
“My Kind of Reflection”.
5Yoav Ravid2y
Also in the modesty argument and No License To Be Human

I agree that it is important to defend and concentrate the force of rationality.

However, I think the difficulty in the original context that this post is about might be that community drama is hard to address in a good way. One major constraint in understanding any topic is having a high flow of representative or unusually useful information from the topic. But for community drama, this is really hard, because it interferes in people's privacy, it involves events that happened in the past, it involves adding up many small interactions, it may be dependent on social relationships, etc..

As such, I think it's going to be really hard for anyone to prove or disprove the validity of complaints about community dynamics in online writing. The best approach I can think of right now is to just allow people to speak up about their impressions without too much burden of proof, to test if others have the same impressions. Maybe there are other, better approaches, but I don't think standards of argumentation can solve them without doing something about the information bottleneck.

Reading this post I kinda feel like you are failing to take your own advice about gardening in a certain way.

Like it feels good to call people out on their BS and get a conversation going, but also I think there's some alternative version of these two posts that's not about building up a model and trying to convince people that something is wrong that must be done about it, and instead a version that just fights the fight at the object level against particular posts, comments, etc. as part of a long slog to change the culture through direct action that others will see and emulate through a shift in the culture.

My belief here is that you can't really cause much effective, lasting change by just telling people they're doing something that sucks and is causing a problem. Rarely will they get excited about it and take up the fight. Instead you just have to fight it out, one weed at a time, until some corner of the garden is plucked and you have a small team of folks helping you with the gardening, then expanding from there.

If LW is not that place and folks don't seem to be doing the work, then maybe LW is simply not the sort of thing that can be what you'd like it to be. Heck, maybe th... (read more)

5Said Achmiz2y
FWIW, I agree with “direct action”, but that only works if the site moderators / admins are not opposed to that action (and, preferably, even support it). If they are, then “direct action” doesn’t work, and only persuasive posts have any chance of working.
3[DEACTIVATED] Duncan Sabien2y
Can confirm from experience.
5[DEACTIVATED] Duncan Sabien2y
You say this as if I do not do it, a lot, and get downvoted, a lot.   This is a totally valid hypothesis imo, and one I keep very close to the forefront. I agree. I am for that reason not putting all my eggs in the this-working-out basket.  <3 (Appreciated and upvoted, in case my tone is not clear.)
4Gordon Seidoh Worley2y
Haha, true, I think you and I have occasionally gotten into it directly with regards to this. I guess to that point I'm not sure the norms you want are actually the norms the community wants. I know for myself the norms I want don't always seem to be the ones the community wants, but I guess I accept this as different people want different things, and I'm just gonna push for the world to be more how I'd like it. I guess that's what you're doing, too, but in a way that feels more forceful, especially in that you sometimes advocate for stuff not belonging rather than to be allowed but argued against. This is likely some deep difference of opinion, but I see LW like a dojo, and you have to let people mess up, and it's more effective to let people see the correction in action rather than for it to go away because you push so hard that everyone is afraid to make mistakes. I get the vibe that you want LW to make corrections so hard that we'd see engagement drop below critical mass, like what happened with LW 1.0 by other means, but that might be misinterpreting what you want to see happen (although you're pretty clear about some of your ideas, so I'm not that uncertain about this).
4[DEACTIVATED] Duncan Sabien2y
TBC, I don't [want] us to drop below critical mass.  The point of disagreement is whether a cleaner standard would result in that or not, and I have a strong suspicion that it would not, in no small part because it wouldn't change the behavior of the high-quality contributors we already have at all, and it would bring in some high-quality contributors who aren't here because the comments are Too Much Headache.

I'd like to throw out some more bad ideas, with fewer disclaimers about how terrible they are because I have less reputation to hedge against.

Inline Commenting

I very strongly endorse the point that it seems bad that someone can make bad claims in a post, which are then refuted in comments which only get read by people who get all the way to the bottom and read comments. To me the obvious (wrong) solution is to let people make inline comments. If nothing else, having a good way within comments to point to what part of the post you want to address feels like a strict win, and given that we already have pingbacks I think letting sufficiently good comments exist alongside the post would also be good. This could also be the kind of thing that a poster can enable or disable, and that a reader can toggle visibility on.

Personal Reputation

I don't have great models for how reputation should or does work on LessWrong. The second of these is testable though - I'd be curious to see what happened if prominent accounts, before commenting, flipped a coin, and in half of all cases posted through a random alt. Of course it may not be a bad thing if respected community figures get more consideration,... (read more)

You can link to comments, so that is an easy technical solution. As ever , it's mainly a cultural problem: if good quality criticism were upvoted, it would appear at the top of the comments anyway, and bit be buried.

On existing LW systems, and on how they shape and incentivize discussions here:

  • Moderation-by-moderator is expensive and doesn't scale. So sites like Reddit or Less Wrong use a karma system. Some impacts of this system:
  • Whether a post or comment is good or bad is determined by a single scalar, i.e. karma.
    • So how do you vote on a comment you consider important and valuable for the most part, but which contains one sentence you consider very wrong? Maybe upvote it while explaining your disagreement?
    • Or what do you do with comments that seem high-effort but very wrongheaded? I want to incentivize effort, even if it occasionally produces wrongheaded results; but upvoting would suggest agreement.
    • What if a comment looks correct and receives lots of upvotes, but over time new info indicates that it's substantially incorrect? Past readers might no longer endorse their upvote, but you can't exactly ask them to rescind their upvotes, when they might have long since moved on from the discussion.
    • How much karma a comment gets depends to a significant extent on how many views it gets, i.e. on a) how early it's posted, b) on how much traffic the post will get overall, and c) on whether it's a top-lev
... (read more)

A few related thoughts: if we do push farther upon the epistemic hygiene axis, it may be worth writing a Sequence on "Living Epistemically Clean" or something along those lines, so that the standards for discussion are clear and we have a guide to upholding those standards. Specification and implementation, if you will. Such a sequence could potentially also cover, say, implementing TAPs for epistemic hygiene in the outside world. It would probably be useful to have more info on how to address your list of "things your brain tries to do" as a community.

I notice that I value some way of allowing new users to grow accustomed to writing on LW. I imagine that my first post may not be epistemically clean enough to pass the standards you're pointing towards (which isn't a point against your standards) and that others new to the site may be discouraged by, say, a ban. The help reviewing drafts that LW currently offers helps, and that's probably a useful place to focus efforts towards helping people uphold the standards you want to put up. Potentially have multiple levels of users--specific karma amounts or moderator approval would be required to comment in some areas?

Slightly meta: I noti... (read more)

7[DEACTIVATED] Duncan Sabien2y
I'm working on such a sequence for genpop; I had not considered tweaking it to serve as a bridge to LW particularly.  A good idea.

Misc. thoughts on the object level:

  • I understand that the community should (and hopefully could) do better, to optimize for a certain culture, that the status quo is Not Good Enough. However, I'm not clear on how widespread you consider this problem to be. My personal impression is that there are <5 controversial posts per month, posts which demand a ton from LW's culture. These are high-stakes events where we noticeably fall short of what is asked of us, and so such occasions could hence benefit a lot from e.g. better culture, or more explicit moderation. The rest of the site, it seems to me, is handled decently well by the karma system. What's your take on that?
    • E.g. regarding several of your explicitly-acknowledged-as-terrible suggestions: I'm not clear on what kind of comment you have in mind that would warrant a temporary ban but which would not get downvoted into oblivion. And similarly, the "team of well-paid moderators" part suggests to me you see a site-wide problem, not just a "we fall short in the high-stakes cases" problem.
    • In any case, if the problem is specifically the high-stakes controversial posts, that opens up a different class of solutions: e.g. a mod could flag
... (read more)
7Said Achmiz2y
121 147 unique commenters, as of this writing. EDIT: Method for count is as follows: on GreaterWrong, turn on the anti-kibitzer feature. Find the lexically-last commenter identifier; in this case, it is ‘EQ’, which is the 121st identifier (26 * 4 + 17 = 121). EDIT 2: Whoops, I can’t count. Obviously it should be 26 * 5 + 17 = 147.
Hmm, I got 146.  My method was: load the page, use command-F to expand the comments, search the page for "[+]" and click on all of them; then select all, copy, and run: $ pbpaste | egrep '\[-\]' | sed -E 's/[0-9]+[a-z]+$//' | sort | uniq -c | sort -nr 62 [-]jessicata 46 [-]Benquo 41 [-]ChristianKl 36 [-]Unreal 35 [-]Duncan_Sabien 32 [-]Viliam 30 [-]habryka 22 [-]TekhneMakre 22 [-]AnnaSalamon 20 [-]Rob Bensinger ... Then pipe the whole thing into wc, yielding 146.  (Also, this method captured 918 comments out of the 924 the page currently reports; there are 8 deleted comments, so I'm not sure exactly what explains the difference; oh well, it seems close enough.)
3Said Achmiz2y
Well, for one thing, I made a dumb arithmetic mistake, so my result should’ve been 147, not 121. That’s actually still off by 2 from yours (because GreaterWrong labels the OP as such, without a letter identifier, resulting in a total of 148). I do not know why this is.
4Yoav Ravid2y
How do you check how much unique pageviews a post got?
As I tried to link to in the original post, Oliver Habryka posted a screenshot of LW's Google Analytics page for roughly October 2021. I'm referencing the two topmost linked URLs (rows 2 and 3), plus the "unique pageviews" column.

Recap:  Concentration of force is

At each relevant moment, you want to project locally superior or overwhelming force. having the most [resources] actually present, or perhaps this just means having the right [resources] pointed in the right directions.

Taking this analogy to forum moderation, this seems to mean the ability 

  • to call in a sufficient number of participants
  • of suitable qualification (e.g., moderator or specific skill)
  • on short notice
  • to police a particular post.

Policing in the civilian sense of e.g., the original Metropolitan Police)  

Additionally, information is needed: The ability to detect posts requiring policing, but I understand that the LW moderation team already has this ability. 


In fact, we're honestly well past the 80/20—LessWrong is at least an 85/37 by this point

8Rohin Shah2y
I assume that meant "instead of 80% of the value for 20% of the effort, we're now at least at 85% of the value for 37% of the effort", which parses fine to me

One thing we are working on in the Guild of the ROSE is a sort of accreditation or ranking system, which we informally call the "belt system" because it has many but not all of the right connotations. It is possible to have expertise in how to think better and it's desirable to have a way of recognizing people who demonstrate their expertise, for a variety of reasons. Currently the ranking system is planned to be a partly based on performance within the courses we are providing, and party based in objective tests of skill ("belt tests"). But we are still experimenting with various ideas and haven't rolled it out.

I'm confused.

In the counterfactual where lesswrong had the epistemic and moderation standards you desire, what would have been the result of the three posts in question, say three days after they were first posted? Can you explain why, using the standards you elucidated here?

(If you've answered this elsewhere, I apologize).

Full disclosure: I read all three of those posts, and downvoted the third post (and only that one), influenced in part by some of the comments to that post.

The three posts would all exist.

The first one would be near zero, karmawise, and possibly slightly negative.  It would include a substantial disclaimer, up front, noting and likely apologizing for the ways in which the first draft was misleading and underjustified.  This would be a result of the first ten comments containing at least three highly upvoted ones pointing that out, and calling for it.

The second post would be highly upvoted; Zoe's actual writing was well in line with what I think a LWer should upvote.  The comments would contain much less piling on and being real confident and rampant extrapolation; they would be focused mainly on "okay, how do we integrate this new data (which we largely take at face value and assume to be true)?  What are the multiple worlds with which it is compatible?  Which worlds that we previously thought possible have been ruled out by this new information?"

They would be doing a lot of split and commit, in other words.  Most people in the comments there seemed to have at max a single hypothesis, and to be collecting confirmation rather than seeking falsification.

(The support for Zoe would be unchanged; that part is ... (read more)

"What are the multiple worlds with which it is compatible? Which worlds that we previously thought possible have been ruled out by this new information?" Thanks for spelling it out like this, that is quite helpful for me. Even though the idea behind it was clear to me before, I intend to implement those two specific questions more into my thinking routines.

I guess long-time lurkers/new posters like me are part of the problem(though obviously I assume most online only LW members didn't engage with a California drama post). I still think LW is a great place for discussion and just being exposed to new ideas and good feedback, but I'm probably dragging down the sanity level.

Re fear: I think the SSC situation made it clear that LW and rationalist adjacent spaces are more public than users might think, maybe people are hesitant because they don't want to get twitter blasted or show up as a snap in an NYTimes arti... (read more)

4Said Achmiz2y
I think that “attempting to aristocratize/oligarchize the site” might (or might not—I am undecided) have desirable consequences… but I think it would be a mistake to base this on karma scores. (See this old comment of mine for reasoning.)

It's a Trojan horse.  It's just such good-thoughts-wrapped-in-bad-forms that people give a pass to, which has the net effect of normalizing bad forms.

Should this be "bad thoughts wrapped in good forms ... normalizing bad thoughts"?

No. The content of the comment is good. The bad is that it was made in response to a comment that was not requesting a response or further elaboration or discussion (or at least not doing so explicitly; the quoted comment does not explicitly point at any part of the comment it's replying to as being such a request). My read of the situation is that person A shared their experience in a long comment, and person B attempted to shut them down / socially-punish them / defend against the comment by replying with a good statement about unhealthy dynamics, implying that person A was playing into that dynamic, without specifying how person A played into that dynamic, when it seems to me that in fact person A was not part of that dynamic and person B was defending themselves without actually saying what they're protecting nor how it's being threatened. This occurs to me as bad form, and I believe it's what Duncan is pointing at.

I see the vision described as something like a community of people who want to do argument mapping together, which involves lots of exposing of tacit linked premises. I think a reason no such community exists (in any appreciable size) is that that mode of discourse is more like discovery rather than creation, as if all of the structure of arguments is already latent within the people arguing and the structure of the argument itself. The intuition then becomes reliable structure->reliable output. Creation, generativity is much messier and involves people surfacing their reactions to things without fully accounting for the reactions others might have (incl. negative), because non predicted reactions are, like, the whole point. There are a large class of persons I would have hedged my comment out more substantially with, but on the basis of past interactions and writing, I consider Aella an adult (in the high-bar 99th% emotional reflective ability sense). I didn't really think about how not having that context would affect how it was perceived.
3[DEACTIVATED] Duncan Sabien2y
This is an interesting and relevant point. In particular, it sparks in me a thought like: Okay, if you're creating your argument and exposing it sort of at the same time, in a high-bandwidth back-and-forth, then you have to have some degree of trust in your conversational partner and audience.  Like, you have to have faith that they won't "gotcha" super hard, that you will be able to revisit and rephrase, that you will be able to change your mind, that if you skip a step they'll ask about it rather than attacking it, that they won't pour a bunch of projections and assumptions onto what you said and then be loath to let them go, etc. Which I think actually becomes a (weak) argument for higher standards in general?  Because if the discourse is overall cleaner, then it becomes easier to do things like "interpret Romeo's comment charitably, and if you have a problem with it, just cooperatively fill in the cracks." Whereas if things are slipping all over the place, there's a kind of race-to-the-bottom that makes it harder to extend that charity and good faith in each individual case, which makes people less willing to share their thoughts in the first place, since avoiding the gotchas takes so much effort.
Right and in my mind canon people are free to respond strongly like 'I think this is a central example of something that is really bad but am having a hard time articulating, can you say more about what you mean?' Because obviously in most communities people would be like wait what? Why would you give the person more ammo after they just said you are bad? To which you'd then have to attempt to point them to the idea that if you are actually confused as badly as they think you might be that would be super valuable to know.

I commend your vision of LessWrong.

I expect that if something like it is someday achieved, it'll mostly be done the hard way through moderation, example-setting and simply trying as hard as possible to do the right thing until most people do the right thing most of the time.

But I also expect that the design of LessWrong on a software level will go a long way towards enabling, enforcing and encouraging the kinds of cultural norms you describe. There are plenty of examples of a website's culture being heavily influenced by its design choices - Twitter's 280-... (read more)

But it’s not actually doing the thing, and as far as I can tell it’s not really trying to do the thing, either—not in the way that blue-tribe Americans are actually trying to do something about racism, from pushing for institutional change all the way down to individuals taking personal responsibility for aiding strangers-in-need.

Is this really what you want? The way that blue-tribe Americans are “actually trying to do something about racism” seems to have almost no chance of actually doing anything about racism, nor even to be designed or intended to h... (read more)

5[DEACTIVATED] Duncan Sabien2y
It's possible I should change that section.  The most important piece of it that I do intend is that we've tipped past "Someone's being racist in the parking lot!  Oh, well, not my problem." to "Someone's being racist in the parking lot!  This threatens an aspect of the fabric that I care about, that makes it my problem." I agree with you that a lot (most?) of what the blue tribe is doing re: racism specifically is counterproductive.
6Said Achmiz2y
I really don’t think that this is an accurate description of what happened (and is happening), in the “racism” case. What’s more, it elides those critical aspects of the situation to which I was trying to point. However, possibly this is a distracting avenue of discussion at this time, so I am content to let this thread end here. I will only say that I strongly recommend thinking about the analogy between these two situations in greater detail.

Personally, I found your highlighting of that paragraph in particular useful, and believe the piece is stronger now that I've removed it, and would not mind if you went into more detail.  This is a tangent/digression, but it's a relevant one, so as long as it's headlined as such I don't think it's distracting.

(This is actually one of my private "rules of engagement" and something I've got an essay in the works about: changes-of-topic are exhausting when they're not self-aware about being changes-of-topic, but if they're clearly identified then that cost goes way down.)

Meta comments on this post:

I appreciate your commitment to precision, specificity, and disclaimers, to rule out what you don't mean and to pinpoint exactly what you do mean. I aspire to clear communication, too, and it's obvious to me that you've put orders of magnitude more effort and skillpoints into it than I could ever imagine doing in my own attempts to communicate. (The best I can personally do is write in a longwinded manner, since my attempts at concise language are too easy to misunderstand.)

That said, this has downsides, too. For instance, I'm no... (read more)

point at small things as if they are important

Taking unimportant things seriously is important. It's often unknown that something is important, or known that it isn't, and that doesn't matter for the way in which it's appropriate to work on details of what's going on with it. General principles of reasoning should work well for all examples, important or not. Ignoring details is a matter of curiosity, allocating attention, it shouldn't impact how the attention that happens to fall on a topic treats it.

general enthusiasm for even rather dull and tediou

... (read more)

Where bad commentary is not highly upvoted just because our monkey brains are cheering, and good commentary is not downvoted or ignored just because our monkey brains boo or are bored.

Suggestion: give our monkey brains a thing to do that lets them follow incentives while supporting (or at least not interfering with) the goal. Some ideas:

  • split upvotes into "this comment has the Right effect on tribal incentives" and "after separating out its impact on what side the reader updates towards, this comment is still worth reading"
  • split upvotes into flair (a
... (read more)
1Alex Vermillion2y
I think the second bullet is called the "Slashdot" model where I've heard it after a site that implemented it famously, but I am pretty amused by the first point too. Something like a few layers of vote would be kind of fun because of how frequently I have to split them, like * This is correct / some amount incorrect * This was a good attempt at being correct / This was an imperfect attempt at being correct * This demonstrates good norms / This demonstrates unwanted norms I'm not advocating this because I haven't thought it out well, but I may return to this in the future.

Strong upvote. Also, I first noticed these types of dynamics at a large scale in the comments on Duncan's Dragon Army proposal (I'm linking to the Medium version since the LW version seems to be gone).

That thread (the subset of it that was happening on LW 1.0) was one of the things that convinced me to build LW 2.0 (I was already working on it, but wasn't sure how much I would commit to it). Because that thread was really quite bad, and a lot of it had to do with deep site architecture things that were hard to change.

The post has a lot of things going on. I am chipping on a it somewhat one spoonful at a time.

I didn't engage with the main drama directly as I am not around there.

It feels like a story about the a stag hunting party that encountered a bear. A bit meatier but a different beast and dangerous. Then some hunters go "nope, I didn't sign up for this" or just flee in abject horror. Those that stay get mauled because a small group can't deal with a full bear.

With a regular stag hunt it elicts frustration. Repeated failures might make starvation creep closer but us... (read more)

I spend a lot of time around people who are not as smart as me, and I also spend a lot of time around people who are as smart as me (or smarter), but who are not as conscientious, and I also spend a lot of time around people who are as smart or smarter and as conscientious or conscientiouser, but who do not have my particular pseudo-autistic special interest and have therefore not spent the better part of the past two decades enthusiastically gathering observations and spinning up models of what happens...
All of which is to say that I spend a decent chu

... (read more)

Make wildly overconfident assertions that it doesn't even believe (that it will e.g. abandon immediately if forced to make a bet).

This seems like an important failure mode that I think a ton of people, including myself, fall victim to. I can't actually think of a reference post for it though. Does it exist? Really, I think it deserves it's own tag. I recall hearing it discussed in the rationality community, but moreso in passing, not as a focal point of something.

2[DEACTIVATED] Duncan Sabien2y
I guess there may not be a direct, central post, but I did write this not long ago.
Hire a team of well-paid moderators for a three-month high-effort experiment of responding to every bad comment with a fixed version of what a good comment making the same point would have looked like.  Flood the site with training data.

What's so terrible about this idea? I imagine the main way it could go wrong is not being able to find enough people willing to do it / accidentally having too low a bar and being overwhelmed by moderators who don't know what they are doing and promote the wrong norms. But I feel like there are probably enough people o... (read more)

6[DEACTIVATED] Duncan Sabien2y
On reflection, it's of a slightly different character than other items on the list. (Each item on the list is "terrible" for somewhat different reasons/has a somewhat different failure mode.) For that one, the main reason I felt I should disclaim it is "here's the part where I try to spend tens of thousands of someone else's money," and it feels like that should be something of a yellow flag.
2Daniel Kokotajlo2y
It's only a yellow flag if you are spending the money. If you are uninvolved and e.g. the Lightcone team is running the show, then it's fine. (But I have no problem with you doing it either)

Standards are not really popular. Most people don’t like them. Half the people here, I think, don’t even see the problem that I’m trying to point at. Or they see it, but they don’t see it as a problem.

I think a good chunk of LW’s current membership would leave or go quiet if we actually succeeded at ratcheting the standards up.

I don’t think that’s a bad thing. I’d like to be surrounded by people who are actually trying. And if LW isn’t going to be that place, and it knows that it isn’t, I’d like to know that, so I can go off and found it (or just give up)

... (read more)
3[DEACTIVATED] Duncan Sabien2y
I do not have specific insider knowledge, no. I do know that you and I have clashed before on what constitutes adhering to the standards, though I believe (as I believe you believe) that in such situations it should be possible to simply take the conversation up a meta level.

On the topic of standards

On the current margin, I am interested in users taking more risks with individual comments and posts, not less. People take more risk when the successes are rewarded more, and when the failures are punished less. I generally encourage very low standards for individual comments, similar to how I have very low standards for individual word-choice or sentence structure. I want to reward or punish users for their body of contributions rather than pick each one apart and make sure it's "up to standard". (As an example, see the how this moderation notice is framed as a holistic evaluation of a user's contributions, not about a single comment.)

So yes, Said, I am broadly opposed to substantially increasing the standards applied to each individual comment or paragraph. I am much more in favor of raising the amount of reward you can get for putting in remarkable amounts of effort and contributing great insights and knowledge. I think your support of Eliezer by making and your support of Gwern with the site re-design are examples of the kind of things I think will make people like Eliezer and Gwern feel like their best writing is rewarded, rather ... (read more)

Re: standards:

What you say makes sense if, and only if, the presence of “bad” content is costless.

And that condition has (at least) these prerequisites:

  1. Everyone (or near enough) clearly sees which content is bad; everyone agrees that the content is bad, and also on what makes it bad; and thus…

  2. … the bad content is clearly and publicly judged as such, and firmly discarded, so that…

  3. … nobody adopts or integrates the bad ideas from the bad content, and nobody’s reasoning, models, practices, behavior, etc. is affected (negatively) by the bad content; and relatedly…

  4. … the bad content does not “crowd out” the good content, bad ideas from it do not outcompete opposing good ideas on corresponding topics, the bad ideas in the bad content never become the consensus views on any relevant subjects, and the bad reasoning in the bad content never affects the norms for discussion (of good content, or of anything) on the site (e.g., is never viewed by newcomers, taken to be representative, and understood to be acceptable).

If, indeed, these conditions obtain, then your perspective is eminently reasonable, and your chosen policy almost certainly the right one.

But it seems very clear to m... (read more)

What you say makes sense if, and only if, the presence of “bad” content is costless.

"Iff" is far too strong. I agree that the "if" claim holds. However, I think that what Ben says also makes sense if the bad/high-variance content has costs which are less than its benefits. Demanding costlessness imposes an unnecessarily high standard on positions disagreeing with your own, I think.

Contrasting your position with Ben's, I sense a potential false dichotomy. Must it be true that either we open the floodgates and allow who-knows-what on the site in order to encourage higher-variance moves, or we sternly allow only the most well-supported reasoning? I think not. What other solutions might be available? 

The first—but surely not best—to come to mind is the curation < LW review < ??? pipeline, where posts are subjected to increasing levels of scrutiny and rewarded with increasing levels of visibility. Perhaps there might be some way for people to modulate "how much they update on a post" by "the amount of scrutiny the post has received." I don't think this quite fights the corrosion you point at. But it seems like something is possible here, and in any case it seems to me too early to conclude there is only one axis of variation in responses to the situation (free-wheeling vs strict).

6Said Achmiz2y
Re: other solutions: I have repeatedly suggested/advocated the (to me, fairly obvious) solution where (to summarize / crystallize my previous commentary on this): 1. People post things on their personal LW blogs. Post authors have moderation powers on their personal-blog posts. 2. Things are posted to the front page only if (but not necessarily “if”!) they are intended to be subject to the sort of scrutiny wherein we insist that posts live up to non-trivial epistemic/etc. standards (with attendant criticism, picking-apart, analysis, etc.; and also with attendant downvotes for posts judged to be bad). Importantly, post authors do not have moderation powers in this case, nor the ability to decide on moderation standards for comments on their posts. (In this case a post might be front-paged by the author, or, with the author’s consent, by the mods.) 3. Posts that go to the front page, are evaluated by the above-described process, and judged to be unusually good, may be “curated” or what have you. In this case, it would be proper for the community to judge personal-blog posts, that have not been subjected to “frontpage-level” scrutiny, as essentially ignorable. This would go a long way toward ensuring that posts of the “jam-packed with bullshit” type (which would either be posted to personal blogs only, or would go to the front page and be mercilessly torn apart, and clearly and publicly judged to be poor) would be largely costless. I agree with you that this sort of setup would not quite solve the problem, and also that it would nonetheless improve the situation markedly. But the LW team has consistently been opposed to this sort of proposal.

It sounds to me like posting on your High-Standards-Frontpage is a very high effort endeavor, an amount of effort that currently only around 3-30 posts each year have put into them. I've thought of this idea before with the name "LW Journal" or "LW Peer Review", which also had a part where it wasn't only commenters critiquing your post, but we paid a few people full-time for reviewing of the posts in this pipeline, and there was also a clear pass/failure with each submission. (Scott Garrabrant has also suggested this idea to me in the past, as a publishing place for his papers.)

I think the main requirement I see is a correspondingly larger incentive to write something that passes this bar. Else I mostly expect the same fate to befall us as with LW 1.0, where Main became increasingly effortful and unpleasant for authors to post to, such that writers like Scott Alexander moved away to writing on their personal blogs.

(I'm generally interested to hear ideas for what would be a big reward for writers to do this sort of thing. The first ones that come to my mind are "money" and "being published in physical books".)

I do think that something like this would really help the site in certain ... (read more)

So I think the Review is pretty good at getting good old content, but I think the thing Said is talking about should happen more quickly, and should be more like Royal Society Letters or w/e. Actually, I wonder about Rohin's newsletters as a model/seed. They attract more scrutiny to things, but they come with the reward of Rohin's summary (and, presumably, more eyeballs than it would have gotten on its own). But also people were going to be writing those things for their own reasons anyway. I think if we had the Eliezer-curated weekly newsletter of "here are the LW posts that caught my interest plus commentary on them", we would probably think the reward and scrutiny were balanced. Of course, as with any suggestion that proposes spending Eliezer-time on something, I think this is pretty dang expensive--but the Royal Society Letters were also colossally expensive to produce.
4[DEACTIVATED] Duncan Sabien2y
I would likely do this from my own motivation (i.e. not necessarily need money) if I were given at least one of: a) guaranteed protection from the badgunk comments by e.g. three moderators willing to be dependably high-effort down in the comments b) given the power to hide badgunk comments pending their author rewriting them to eliminate the badgunk c) given the power to leave inline commentary on people's badgunk comments The only thing holding me back from doing something much more like what Said proposes is "LW comment sections regularly abuse and exhaust me."  Literally that's the only barrier, and it's a substantial one.  If LW comment sections did not regularly abuse and exhaust me, such that every post feels like I need to set aside fifty hours of life and spoons just in case, then I could and would be much more prolific. (To be clear: some people whose pushback on this post was emphatically not abuse or exhausting include supposedlyfun, Said, Elizabeth, johnswentsworth, and agrippa.)
5Said Achmiz2y
Would you accept this substitute: “A site/community culture where other commenters will reliably ‘call out’ (and downvote) undesirable comments, and will not be punished for doing so (and attempts to punish them for such ‘vigilante-style’ a.k.a. ‘grassroots’ ‘comment policing’ will themselves be punished—by other commenters, recursively, with support from moderators if required).”
5[DEACTIVATED] Duncan Sabien2y
Yes, absolutely.  Thanks for noting it.  That substitute is much more what the OP is pushing for. EDIT: With the further claim that, once such activity is reliable and credible, its rate will also decrease.  That standards, clearly held and reliably enforced, tend to beget fewer violations in the first place, and that, in other words, I don't think this would be a permanent uptick in policing.

Strong-upvote. I want to register that I have had disagreements with Said in the past about this, and while I am still not completely sure whether I agree with his frame, recent developments have in fact caused me to update significantly towards his view.

I suspect this is true of others as well, such that I think Said's view (as well as associated views that may differ in specifics but agree in thrust) can no longer be treated as the minority viewpoint. (They may still be the minority view, but if so I don't expect it to be a small minority anymore, where "small" might be operationalized as "less than 1 in 5 people on this site".)

There are, at the very least, three prominent examples that spring to mind of people advocating something like "higher epistemic standards on LW": Duncan, Said, and (if I might be so bold) myself. There are, moreover, a smattering of comments from less prolific commenters, most of whom seem to express agreement with Duncan's OP. I do not think this is something that should be ignored, and I think the site may benefit from some kind of poll of its userbase, just to see exactly how much consensus there is on this.

(I recognize that the LW/Lightcone team may n... (read more)

2Ben Pace2y
I am generally in favor of people running polls and surveys about information they're interested in.  (Here's a very random one I did, and looking through search I see people have done them on general demographics, nootropics, existential risk, akrasia, and more.)
5Ben Pace2y
I'm pretty confused by your numbered list, because they seem directly in contradiction with how scientific journals have worked historically. Here's a quote from an earlier post of mine: I think that many-to-most papers published in scientific journals are basically on unhelpful questions and add little to the field, and I'd bet some of the proofs are false. And yet it seems to be very rarely published that they're unhelpful or wrong or even criticized in the journals. People build on the good content, and forget about the rest. And journals provide a publishing house for the best ideas at a given time. (Not too dissimilar to the annual LW Review.) It seems to me that low-quality content is indeed pretty low cost if you have a good filtering mechanism for the best content, and an incentive for people to produce great content. I think on the margin I am interested in creating more of both — better filtering and stronger incentives. That is where my mind currently goes when I think of ways to improve LessWrong.
1Said Achmiz2y
This seems like a very odd response given the fact of the replication crisis, the many cases of scientific knowledge being forgotten for decades after being discovered, the rise of false or inaccurate (and sometimes quite harmful!) models, etc. I think that (in many, or most, scientific fields) people often don’t build on the good content, and don’t forget about the bad content; often, the reverse happens. It’s true that it’s “very rarely published that [bad papers/proofs are] unhelpful or wrong or even criticized in the journals”! But this is, actually, very bad, and is a huge reason why more and more scientific fields are being revealed to be full of un-replicable nonsense, egregious mistakes, and even outright fraud! The filtering mechanisms we have are actually quite poor. The “papers in scientific journals” example / case study seems to me to yield clear, strong support for my view.
7Ben Pace2y
It's very worthwhile to understand the ways in which academia has died over the last 60 years or so, and part of it definitely involves failures in the journal system. But the axis of public criticism in journals doesn't seem at all to have been what changed in the last 60 years? Insofar as you think that's a primary reason, you seem to be explaining a change by pointing to a variable that has not changed. In replying to your proposed norms, it's not odd to point out that the very mechanism of labeling everything that's bad as bad and ensuring we have common knowledge of it, was not remotely present when science was at its most productive — when Turing was inventing his machines, or when Crick & Watson were discovering the structure of DNA. In fact it seems to have been actively opposed in the journal system, because you do not get zero criticism without active optimization for it. That is why it seems to me to be strong evidence against the system you propose. There may be a system that works via criticizing everything bad in public, but when science was most successful it did not, and instead seems to me to be based around the system I describe (a lot of submissions and high reward for success, little punishment for failure).

As a note on balance, I think you addressed the (predicted) costs of raising standards but not the costs of the existing standards.

I know of at least three people that I believe Ben Pace would consider high-quality potential contributors who are not writing and commenting much on LessWrong because the comments are exhausting and costly in approximately the ways I'm gesturing at.

(We have some disagreement but more overlap than disagreement.)

And I myself am quite likely to keep adding essays but am extremely unlikely to be anything other than a thought-dropper, at the current level of you-can-get-away-with-strawmanning-and-projecting-and-bullying-and-various-other-violations-of-existing-nominal-norms-in-the-comments-and-it'll-be-highly-upvoted.

I think that should be part of the weighing, too.  Like, the cost of people feeling cringier is real, but so too is the cost of people who don't even bother to show up, because they don't feel safe doing so.

I also note that I strongly claim that a generally cleaner epistemic pool straightforwardly allows for more experimentation and exploration.  If there's more good faith in the atmosphere (because there's less gunk causing people to... (read more)

because the comments are exhausting and costly in approximately the ways I'm gesturing at.

(We have some disagreement but more overlap than disagreement.)

As I understand Ben Pace, he's saying something like "I want people to take more risks so that we find more gold", and you're replying with something like "I think people will take more risks if we make the space more safe, by policing things like strawmanning."

It seems central to me to somehow get precise and connected to reality, like what specific rules you're suggesting policing (strawmanning? projecting? Everything in the Sequences?), and maybe look at some historic posts and comments and figure out which bits you would police and which you wouldn't. (I'm really not sure if this is in the 'overlap' space or the 'disagreement' space.)

Strong-upvote as well for the specificity request; the place where I most strongly expect attempts at "increasing standards" to fail is the point where people realize that broad agreement about direction does not necessarily translate to finer agreement about implementation, and I expect this is best avoided by sharing gears-level models as quickly and as early during the initial discussion as possible. As I wrote in another comment:
1[DEACTIVATED] Duncan Sabien2y
I note that I've already put something like ten full hours into creating exactly these types of examples, and that fact sort of keeps getting ignored/people largely never engage with them. Perhaps you are suggesting a post that does that-and-nothing-but-that?
I think I am suggesting "link to things when you mention them." Like, if I want to argue with DanielFilan about whether or not a particular garment "is proper" or not, it's really not obvious what I mean, whereas if I say "hey I don't think that complies with the US Flag Code", most of the work is done (and then we figure out whether or not section j actually applies to the garment in question, ultimately concluding that it does not). Like, elsewhere you write: I currently don't think there exists a 'straightforward list of rationality 101 principles and practices' that I could link someone to (in the same way that I can link them to the Flag Code, or to literal Canon Law). Like, where's the boundary between rationality 101 and rationality 102? (What fraction of rationality 101 do the current 'default comment guidelines' contain?) Given the absence of that, I think you're imagining much more agreement than exists. Some like the "Double crux" style, but Said disliked it back in 2018 [1] [2] and presumably feels the same way now. Does that mean it's in the canon, like you suggest in this comment, or not? [Edit: I recall that at some point, you had something that I think was called Sabien's Rules? I can't find it with a quick search now, but I think having something like that which you can easily link to and people can either agree with or disagree with will clarify things compared to your current gesturing at a large body of things.]
2[DEACTIVATED] Duncan Sabien2y
Sabien's Sins is linked in the OP (near the end, in the list of terrible ideas). I will probably make a master linkpost somewhere in my next four LW essays.  Thanks.
Where? Is it the quoted lines?
4[DEACTIVATED] Duncan Sabien2y
Huh, not sure how I missed that; thanks for pointing it out.
2Said Achmiz2y
Indeed, my opinion of “double crux” has not improved since the linked comments were written.