This is a special post for quick takes by bgold. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Related: Reason as memetic immune disorder
I like the idea that having some parts of you protected from yourself makes them indirectly protected from people or memes who have power over you (and want to optimize you for their benefit, not yours). Being irrational is better than being transparently rational when someone is holding a gun at your head. If you could do something, you would be forced to do it (against your interests), so it's better for you if you can't.
But, what now? It seems like rationality and introspection is a bit like defusing a bomb -- great if you can do it perfectly, but it kills you when you do it halfways.
It reminds me of a fantasy book which had a system of magic where wizards could achieve 4 levels of power. Being known as a 3rd level wizard was a very bad thing, because all 4th level wizards were trying to magically enslave you -- to get rid of a potential competitor, and to get a powerful slave (I suppose the magical cost of enslaving someone didn't grow up proportionally to victim's level).
To use an analogy, being biologically incapable of reaching 3rd level of magic might be an evolutionary advantage. But at the same time, it would prevent you from reaching the 4th level, ever.
Thanks for including that link - seems right, and reminded me of Scott's old post Epistemic Learned Helplessness
The only difference between their presentation and mine is that I’m saying that for 99% of people, 99% of the time, taking ideas seriously is the wrong strategy
I kinda think this is true, and it's not clear to me from the outset whether you should "go down the path" of getting access to level 3 magic given the negatives.
Probably good heuristics are proceeding with caution when encountering new/out there ideas, remembering you always have the right to say no, finding trustworthy guides, etc.
Depression as a concept doesn't make sense to me. Why on earth would it be fitness enhancing to have a state of withdrawal, retreat, collapse where a lack of energy prevents you from trying new things? I've brainstormed a number of explanations:
I'm partial to the explanation offered by the Predictive Processing Model, that depression is an extreme form of low confidence. As SSC write:
imagine the world’s most unsuccessful entrepreneur. Every company they make flounders and dies. Every stock they pick crashes the next day. Their vacations always get rained-out, their dates always end up with the other person leaving halfway through and sticking them with the bill.
What if your job is advising this guy? If they’re thinking of starting a new company, your advice is “Be really careful – you should know it’ll probably go badly”.
if sadness were a way of saying “Things are going pretty badly, maybe be less confidence and don’t start any new projects”, that would be useful...
Depression isn’t normal sadness. But if normal sadness lowers neural confidence a little, maybe depression is the pathological result of biological processes that lower neural confidence.
But I still don't understand why the behaviors we often see with depression - isolation, lack of energy - are 'longterm adaptive'. If a particular policy isn't working, I'd expect to see more energy going into experimentation.
[TK. Unfinished because I accidentally clicked submit and haven't finished editing the full comment]
I think you're asking too much of evolutionary theory here. Human bodies do lots of things that aren't longterm adaptive -- for example, if you stab them hard enough, all the blood falls out and they die. One could interpret the subsequent shock, anemia, etc. as having some fitness-enhancing purpose, but really the whole thing is a hard-to-fix bug in body design: if there were mutant humans whose blood more reliably stayed inside them, their mutation would quickly reach fixation in the early ancestral environment.
We understand blood and wound healing well enough to know that no such mutation can exist: there aren't any small, incrementally-beneficial changes which can produce that result. In the same way, it shouldn't be confusing that depression is maladaptive; you should only be confused if it's both maladaptive and easy to improve on. Intuitively it feels like it should be -- just pick different policies -- but that intuition isn't rooted in fine-grained understanding of the brain and you shouldn't let it affect your beliefs.
On a group selection level it might make lots more sense to have certain people get into states where they're very unlikely to procreate.
On of the finding of data-driven models of evolution of the last decades, is that group selection mostly isn't strong enough to create effects.
My views come more from listening to experts and not from looking at specifics. When studying bioinformatics that's basically what they told us about the result of researching genetics with computer models. Afterwards when talking to experts, I also heard the same sentiments that most claims of group selection shouldn't be trusted.
I too have heard that group selection is not well believed it just seems so out of sync with my understanding of systems theory that I'm skeptical about taking people's word on it.
Since we can sequence genomes we know how many changes need to happen for the difference between organisms. We know that gene drift destroys features for which there isn't selection pressure to keep them like our ability to make our own Vitamin C.
It seems to me like the moving pieces that are needed for computer models are there, so I would trust experts opinions of people on the topic more strongly then would be warranted 30 years ago where opinions were mostly based on intellectual arguments.
Is the clearest "win" of a LW meme the rise of the term "virtue signaling"? On the one hand I'm impressed w/ how dominant it has become in the discourse, on the other... maybe our comparative advantage is creating really sharp symmetric weapons...
Do I understand it correctly that you believe the words "virtue signaling", or at least their frequent use, originates on LW? What is your evidence for this? (Do you have a link to what appears to be the first use?)
In my opinion, Robin Hanson is more likely suspect, because he talks about signaling all the time. But I would not be surprised to hear that someone else used that idiom first, maybe decades ago.
In other words, is there anything more than "I heard about 'virtue signaling' first on LW"?
https://twitter.com/esyudkowsky/status/910941417928777728
I remember seeing other claims/analysis of this but don't remember where
When EY says our community he means more then just LW but the whole rationalist diaspora as well towards which Robin Hanson can be counted.