You make many good points about skills and group membership. Perhaps this one is really not that relevant to your point, but it does seem worth mentioning for any aspiring rationalist considering your scenario.
You don't mention the skill sets I think are most likely to contribute to that scenario.
I'd think after hearing that story that most likely explanation was that Adam irritated Bella.
The most relevant skills seem like communication, social grace, and emotion management.
The sequences frequently mention communication, but seldom mention social grace or emotion management. Not getting defensive when someone is being abrasive seems like largely an emotion management skill.
Communication is hard, so it's pretty likely that Adam and Bella weren't on exactly the same page about what was under discussion. They could've both been mostly right since they were talking about somewhat different things (you said the exchange was brief, making this more likely). Both transmissive and receptive communication skills are hard, particularlly for complex topics.
The skill of telling someone they're wrong without being abrasive is real and difficult. I don't think you mentioned it.
The skill of being abrasively told you're wrong and not letting it irritate you is also real and difficult.
I think this is worth mentioning because people seem to fairly frequently assume that being a rationalist does or should mean your emotions don't play a role in your thinking. This is wildly unrealistic. Being a rationalist provides some resistance to the worst forms of emotions interfering with logic (since the love of truth is an emotion that can counteract many other influences), but it doesn't nearly provide full immunity.
Assuming you're not biased or otherwise affected by emotion because you're a rationalist is very irrational.
Sometimes people ask me why someone behaving irrationally is present at a rationalist meetup. This essay is a partial answer to why I think that question is confused. The short version is, we are not set up such that any particular list of skills is a requirement.
Once upon a time I was talking to someone at a rationalist meetup, and they described a conversation they’d had with another attendee. Let's call these two Adam and Bella. Bella, it transpired, had said something incorrect, Adam had corrected them, and Bella got defensive. A bit of back and forth transpired.
(Names have been changed to protect the innocent and guilty alike.)
Adam later talks to me and asks, “why would a rationalist not be glad to learn that they were wrong? Don’t we want to learn to be less wrong?”
(One reason is that Adam is wrong and Bella is right. I have passed through denial and into acceptance at the rate that people show up to the 'believe true things' club, then assume the only explanation for why regulars do not agree on their pet contrarian take is that everybody else is wrong. And of course, being told a fact is not the same as being convinced that fact is true. Assume for the moment that Adam is right on the object level here.)
I’ve had several variations on this conversation. There is a genuine point here, and one I think is important. Why would a member of a website called LessWrong, fully aware of the Litanies of Tarski and Gendlin, not update cleanly? Since it’s come up repeatedly, I would like a canonical explanation for why I think this happens and why I’m not more upset about it. First, a few equivalencies for other identities.
When I look at that list, one answer that jumps out at me is that “mathematician” describes a very wide range of skills.
There is no common census on when someone starts being referred to as any of those identities. “Doctor” is pretty clear, you have to have a medical degree and they don’t have those in stock at Walmart. “Marine” is very explicit, and I confidently expect every marine with all four limbs intact can kick my ass (and quite a few can kick my ass with less limbs than that.) But calling ten year-old Screwtape with his TI-83 BASIC calculator and a look of screwed up concentration a programmer isn’t obviously incorrect, despite the fact that he didn’t even know there was a SortA() function. “Mathematician” is defined as “an expert in or student of mathematics” which means that any kid studying arithmetic for the first time counts.
“Swimmer” is what you yell when someone goes overboard while white water rafting, whether or not they can swim.
Descriptively, “Rationalist” is sometimes used to describe people who show up to house parties in a certain social circle. I've made my particular piece with this. "Rationalist" is just another word with fuzzy category boundaries, and I think at this point it's descriptively correct that the word points as much to the social group as to the skillset.
Let's get a little more complicated.
I have a friend, a great musician, she’s been making music for decades now. Do you think it’s reasonable to assume she knows how to play a G chord?
What if I say she’s a drummer?
I’m not being obtuse for no reason here. There’s lots of sub-specialization in lots of fields. Many people dabble - guitarists who can sing a little, violinists who can do the cello - but again you have to be careful with your terms and assumptions. I think of myself as a decent programmer, but I’ve never written a compiler. Some people who have written compilers haven’t ever stood up a web server. Brazilian Jiu Jitsu is great, but I don’t think it contains a side kick.
Just so, there’s people in the rationality community I expect to strictly beat me me on calibration and betting, but who I think I could outmatch in anticipating my own failures (virtue of humility.) Likewise, I know some people with a comprehensive understanding of cognitive science who are just as overconfident in practice as the man on the street.
I want to be clear about what I’m not saying. I’m not saying I don’t care. I’m not saying rounding yourself out isn’t useful. I’m not saying preferences don’t matter; some authors just prefer first person and that’s fine. I’m also not saying that knowing a skill means literally never making future mistakes; sometimes a singer has a cold or just hits a flat note.
And all of that assumes that there’s some kind of ladder or path to becoming more rational, which posters on LessWrong or attendees at an ACX meetup are inevitably going to encounter. If that’s what you think, you’re obviously going to different meetups than I am.
I like Duncan Sabien’s Make More Grayspaces.
If you think of the target space as a place where An Unusual Good Thing Happens, based on five or six special preconditions that make that target space different from the broader outside world...
...then the grayspace is the place where those five or six special preconditions are being laid down. The Unusual Good Thing isn’t actually happening in the grayspace. It isn’t even being attempted. And The Unusual Good Thing isn’t being exposed to people who don’t have the prereqs down pat.
If a math Ph.D. doesn’t know how to do long division, now I agree we have a problem. Likewise if a black belt in Karate can’t do a side kick. These are also true at earlier levels! Anyone with a high school diploma should have the long division thing down.[1]
"Ah," you might say, "but rationality does have its answer! We can read the sequences!"
Eh.
So first off, there's the Astral Codex Ten groups. It's not obvious from ACX that you should read the sequences. Maybe you just want to read ACX, and some people will just have been reading for the last few weeks or months before they joined.
Then you run into the fact that this is kind of rough on newcomers, and newcomers won't stop coming. I know of multiple local groups that read all the sequence posts! It usually takes them a couple years! None of them wanted to restart from the beginning even though some new people had joined halfway through.
Plus, I'm just skeptical that reading the sequences once, years ago, is enough to remember it?
This thing we do, this collective effort to have a real art of rationality? I’m not convinced we have a real grayspace for most of it. I want to congratulate the Guild of the Rose for being a place where they say ‘hey, try changing your mind, and admit it out loud. We’re keeping track of who has done it and who hasn’t.’ It’s one of three paths and guild members can pick what paths to go down, so I can’t quite be told someone’s been a member of the Guild for years and confidently know they can change their mind without flinching, but they’re closer than anyone else I know.
What would it take?
I think it would take three things.
Even if we had one, “rationalist” seems as reasonable a term to apply to someone just starting out as “chess player” or “writer.” I’m not begrudging anyone the term.
(Okay. I am a little bit begrudging the people who aren’t even trying. If you call yourself a chess player and you show up to chess club, you either gotta already know how the knight moves or be willing to have someone show you. If you’re not interested - well, look, I’d like to find a place for the people who are. But the socials are still fine, I run 'em myself and I have a good time at the ones I attend. And as a collective we really have not made it easy to get good at the skills.)
Just, consider all this next time you’re tempted to assume someone should already have a skill.
Lest anyone think I'm casting stones at others here, I did not have long division down when I got my diploma. I think this demonstrated a mistake on the part of the school system. (Short version, I swapped schools a lot and the new school sort of assumed the old one had covered it.)