I've actually been doing research on the Semmelweiss reflex, ethics, and the problems of unbearable knowledge and defensive unknowing in the context of making decisions about existential threats, with an eye on how to solve the problem of giving people more tools and cognitive resources. AI alignment is one of the existential threats of interest. (Essentially, I have been quietly working the same problem you are publicly working towards solving.)
I think you have some great points, but I have a lot to add.
Unfortunately, I lack a lot of the jargon of LW, and operate from different heuristics, so accidentally alienate people easily. Within the social mores of Less Wrong, what's appropriate? To message you privately, to write a very long comment, to create a separate post? What would you prefer? I would really love to talk about this without accidentally blowing anything up.
Perhaps instead of thinking of myths as eternal Truths, its more useful to think of them as heuristics, or easy ways to capture and communicate complex ideas. Tech for thinking complex thoughts. Humans rarely have the capacity to think abstractly, so we mostly reason by metaphor to something concrete.
From this perspective, stories and metaphors give shape to the kinds of thoughts people can think.
I think of Chinese Cheng Yu, the 4 character idioms that refer to stories that have morals or lessons, as being the most compressed form of this. Most cultures, from ancient Greece to rural America, have some form of this.
When s someone references a Less Wrong phrase that conveys a complex idea via a metaphor, like Cupstacking, the Map is not the Territory, they are doing the same thing. To some extent, one of the projects of Less Wrong is to replace and expand a vernacular.
On the other hand, recently Americans have been fed a glut of superhero, Western, and epic fantasy stories that have a single clear Good guy and Bad guy, and a straight forward arc of the person winning (usually though violence) being the Good guy, or at least Our guy. There is a struggle for power, honor, respect, justice, safety, the girl (or her memory), and all that is right. Whoever opposes the Protagonist/POV character is the bad guy by default.
This story has very little to do with universal truths, but it does shape thinking, especially when so often repeated.
This phenomenon seems to match up with existing search terms like structural dissociation, double-binds, and double-bookkseping. It's an interesting and sympathetic exploration.
Another possible solution to the hostile telepath problem, at least the angry mom. Instead of self-protection and appeasement aa a core strategy, you could be curious about the other person's internal experience. Perhaps under their anger is distress. Perhaps they want you to connect with their distress.
Perhaps you could telepath back.
Perhaps they want you to care about their experience, not just engage in self protection so you can continue to access the resources of the relationship.
This isn't always true. It isn't always possible, epecially for a child totally dependent on a parents' resources.
But as an adult, this is emotional connection is often what is being requested.
What you are saying about persuasion is important in both directions. Have you every encountered agnotology? It's part of sociology, and it looks at the creation of unknowing. In the cases you list above, including tobacco and lead, there has been research into the ways that industries marshalled money, resources, and persuasion to create doubt and prevent regulation.
So there's maybe an additional important heuristic, which is that profit will motivate individuals to ignore harm, and if they have power and money, they will use institutions to persuade people not to notice.
It's not the entirety of the difficulty, but it is something that ethics might help to correct.