You understand that one of the most important aspects of things like that is that it's dangerous to publicly question them, right?
Think of 10 commonly accepted truths that you'd be afraid to question in public, preferably ones that you would feel uncomfortable even questioning in your own mind. Things where if you did question them, you'd expect to hear either a two-sentence "argument for" treated as obviously overcoming all possible objections, or a vehement, impassioned refusal to even talk about it because "the question is Settled". Extra points if it's been Settled by Science(TM).
Probably at least one or two of them will be false, or only sometimes or partly true, or have far less ironclad support than you initially feel they do. Maybe more than that. Of course, some of them will also be 100 percent true, but you'll have generated some candidates.
Take, for example, your own example of eugenics. Forced sterilization is discredited. Mass extermination is discredited. But those are means, not ends. They are not what the word "eugenics" meant to the people advocating for it. "Eugenics" just means any change in reproductive actions made based on the obvious truth that some human heritable characteristics are more desirable than others.
If you know that 25 percent of your offspring will definitely die after short lives of horrible pain, and you therefore personally choose not to have any children, then you are absolutely practicing eugenics. The fact that that sort of choice is, in fact, sometimes looked askance at, and treated as somehow equivalent to rounding other people up and putting them into camps, or even seen as a dangerous step on the path to rounding people up and putting them in camps, is a harmful orthodoxy of the kind you are looking for. And it does affect the medical advice that people get.
If you want a guess as to something that's already started to change, and that may in the future be anathema, one real possibility would be the idea that it's OK to eat meat.
Yes, I think my focus is ideally less on "debate specific examples" (I can easily think of many that I think would be extremely controversial, some of which have been brought up in the comments) and more on what sort of meta-rules would be appropriate to use in order to try and protect ourselves more generally and be the type of community that doesn't fall for this stuff.
OK, then I'd suggest trying to nip them in the bud, because once they get momentum behind them I suspect they're almost impossible to stop until they've sort of run their natural course. And the only thing I can think of to do that is to apply the same old rules of evidence and argumentation that we all know and love.
On edit: And after they've become hard to challenge, I guess it couldn't hurt to just not feed them, and perhaps subtly bringing up issues and ways of thinking that might not be obviously related on the surface, but might lead people to think twice. I don't think attacking those things head on usually works; it just hardens opinions and makes you a pariah. If there are physical actions associated with the problem, you can also find ways not to contribute to those or even to sabotage them.
I think if a person cannot point to several opinions they currently have that are regarded as abhorrent or stupid by most people, then it is unlikely that they would actually have held "correct"* opinions on the matters mentioned above, and other similar matters.
*i.e. opinions regarded as correct in <current year>.
Intelligence is no antitode. The philosopher Heidegger was closely allied with the Nazis. The most famous economist J M Keynes was Director of the British Eugenics Society (1937-1944).
I do hold several such opinions but there is no way I am going to state them in public. One thing that has not changed is the intolerance for divergent opinions. If anything it has become worse.
I am old enough to have seen many changes such that opinions regarded as totally abhorrent have now become the orthodoxy. And the old opinion is now regarded as abhorrent. I see the new generation quietly adopt the new opinion and easily condemn those who grew up in earlier times.
A few years back a young less-wronger informed me how grateful he was to have grown up in a time and place where he had a peer group with correct opinions on all the important issues. My thought was that it was mostly likely that the reason he thought those opinions were correct was because they were held by his peer group. Not especially because they are correct.
We actually had a session on this at the local LW where we tried to imagine current beliefs that a future generation would regard as terrible.
One scenario someone came up with was that society became much more conservative (plausability from the idea that coservatives and the like tend to have more children) and many of the current 'woke' beliefs would be seen as very regrettable and harmful.
Another was a kind of Idiocracy scenario where the policies of our time were regarded as a catastrophe because they were dysgenic (e.g subsidies for low-IQ single mothers etc). I do stress these were scenarios we came up with, not beliefs we hold.
I don't think rationalists or Effective Altruists would have been taken in much by the Nazis, but I worry that when it comes to whether we would have supported eugenics, many of us might have failed the test -- after all, this was very much scientific and medical orthodoxy at the time.
Why? Before the national socialists were successful in elections they had strong support in universities. At the time in Germany, if you were in the market for an idealistic ideology that you want to self-identify with, national socialism and communism were the strongest options.
I do think that plenty of EA's join EA to be able to self-identify in a way that convinces them they and other EA's are morally superior compared to other people.
How are we to know that we aren't making similar errors today?
Based only on priors, the probability we aren't is very low indeed. A better question is, given an identified issue, how can change happen? One main problem is that contra-orthodox information on moral issues tends not to travel easily.
One exercise you can try is imagining a world where your currently popular belief is as unpopular as eugenics is now. Almost no one thinks your belief is plausible; most people are dumbfounded or angered by your sincere assertions, and ascribe bad motives to you. Some get mad just because you make an argument that might indirectly support that view. Take 5 minutes to think about what it would be like to experience such a world. If you find yourself less attached to the belief, you might be unduly influenced by its current popularity.
(If you're inclined to contrarianism, imagine the opposite.)
Similarly, there was an extended period in the 1980s and 1990s where many scientists and media figures believed that techniques known as "recovered memory therapy" were helping people recover suppressed memories of abuse they had actually suffered as children -- now the general view is that this therapy was creating false memories, not bringing hidden true ones to the surface.
I think the general view is that some of the memories are false.
My read is that the rationalist/EA communities wouldn’t contest whether eugenics was a good idea in principle, but would notice the glaring moral issues with the ways that it was actually pursued.
Interesting post. What are in your opinion some of those ideas held by a significant fraction of the community that could be shown in the future to be akin to eugenics?
I prefer not to get into specific examples here (several have been brought up in comments to varying degrees of controversy), but rather to discuss the broader meta question of how best to be a community that avoids falling for things.
The anti-malaria programs are a good candidate for future moral outrage.
The politically correct people of the future may decide that malaria was a part of African cultural heritage, and the developed countries destroyed it in yet another unforgivable act of colonialism. Note that this opinion will sound much less absurd if the malaria is actually successfully eradicated planet-wide. Then it will become much easier to claim that the harmfulness of malaria was just a myth invented by the oppressors as an excuse for its eradication. Bonus points if you can point out actual harm to humans caused by the anti-malaria efforts; such as some insecticides having harmful side-effects on human health.
Alternatively, the lack of actually using gene drive mosquitos in the wild at scale and letting a lot of people die while we wait could also be seen as a moral tragedy.
(content warning: murder, medical abuse)
A few years ago, my old middle school and another school in its district were renamed; they had originally been named after once-esteemed educational figures who were now in disrepute thanks in large part to their links to eugenics. I believe at the time I may have thought that it was odd that the schools had not been renamed sooner -- eugenics had already been discredited for decades at that point.
But at the time that these men lived, eugenics was not discredited. In fact, it was extremely popular. While eventually things changed, much of society was taken in first -- with terrible consequences. Unjust laws were passed based on eugenic principles in many nations, perhaps most notably in Nazi Germany. Hundreds of thousands of people were killed under government eugenics programs while hundreds of thousands more were involuntarily sterilized. Nowadays these acts are widely viewed as murder and abuse, but it took society some time to reach that conclusion and in the meantime things went pretty badly.
Worse still, eugenics is not the only such disgrace. You may well have heard of the lobotomy, an invasive medical procedure that is now widely viewed as destructive and abusive. However, when the lobotomy was first developed, its creator was not condemned as a medical abuser and removed from his position -- in fact, he received the Nobel Prize for his creation! Tens of thousands of people were lobotomized before lobotomies became viewed as abusive and were banned or greatly restricted in many jurisdictions.
Similarly, there was an extended period in the 1980s and 1990s where many scientists and media figures believed that techniques known as "recovered memory therapy" were helping people recover suppressed memories of abuse they had actually suffered as children -- now the general view is that this therapy was creating false memories, not bringing hidden true ones to the surface. Unfortunately, before recovered memory therapy was discredited its "revelations" led to many people being falsely accused of child abuse and in some cases even Satanic activity, and some had their reputations destroyed or were unjustly imprisoned on the basis of these "recovered" memories.
All three of these cases -- eugenics, lobotomies, and recovered memory therapy -- are instances where bad moral and medical thinking led to grave abuses. Worse still, all of them were popular within the last hundred years -- one doesn't exactly have to go back to the days of bloodletting and trepanation to find these sorts of errors.
To me, this seems to suggest a worrisome problem. How are we to know that we aren't making similar errors today? What can we do to try and protect ourselves from these sorts of mistakes?
There is a famous work called "Who Goes Nazi" that asks one to consider who among one's acquaintances would be likely to support the Nazi regime. (It was written in 1941 when this was very much a live concern.) I don't think rationalists or Effective Altruists would have been taken in much by the Nazis, but I worry that when it comes to whether we would have supported eugenics, many of us might have failed the test -- after all, this was very much scientific and medical orthodoxy at the time.
To put things another way, I think a lot of us wouldn't be fooled by demagogues but that scientific/medical misconduct might be something else. I'm curious whether anyone has especially good ideas or suggestions as to how best to think about these things -- ideally I would like to be able to say "EA is the kind of movement that wouldn't have supported eugenics even when many scientists and doctors did", but I don't feel confident in that statement at this time.
(I should clarify also that this is not meant as an insult, but rather as a case for moral examination and reflection.)