This would be more powerful with some examples, e.g. "we've all learned that Lamarck thought species evolved by passing on acquired traits, but Darwin told us that species can't evolve that way but do so through selection on preinherited traits; but here's a quote from Origin of Species in which Darwin explicitly endorses Lamarckian selection." Repeat for Kuhn, Kant, etc.

This inevitably involves getting rid of detritus that no longer serves you, but that’s only in service of pursuing your ideal life

https://en.wikipedia.org/wiki/Garden-path_sentence

The balance between rights and obligations has probably been understood from time immemorial.

Perhaps, but your understanding is not the understanding I'm familiar with.

I've understood this balance to be something more intimate: If I have the right to live, you have the obligation not to take my life. If I have the right to speak, you have the obligation not to muzzle me. If I have the right to healthcare, somebody has the obligation to provide me with that healthcare. Or things along those lines.

Your formulation, in which there are a set of possible rights and obligations and you need to fill one pocket with rights and the other pocket with obligations in such a way that you can still walk a straight line, is some other species of animal.

Things I'd like to know:

1. What is the baseline things-going-tits-up-mentally rate for people similarly situated to those who take on meditation, and how does that compare to the rate for people who begin to meditate? There's a smell of "the plural of anecdote is data" about this post. People at high risk for mental illness can go around the bend while meditating? Well, they can go around the bend watching TV or chatting with Claude too. How much more dangerous is meditation than the typical range of alternative activities?
2. There's bad, and then there's Bad. Ask a reformed alcoholic whether there were any negative side effects of giving up alcohol, and they'll tell you it was a bit like an anal probe of the soul with the devil's pitchfork for a month or two at least. Is that a cautionary tale that should steer you away from sobriety, or just par for that otherwise worthy course? Some of the practitioners of meditation (esp. in the Buddhist tradition) think we're most of us addicts to the chimerical delights of the senses, and that of course it'll be a struggle to overcome that. FWIW.
3. Is this like psychedelics, where if you take them in the context of a long-standing ritual practice with lots of baked-in wisdom, things will probably go okay or at least they'll know how to give you a soft pillow to land on if you get too far out there; but if you take them in some arbitrary context there's no telling how it'll turn out? How do outcomes look for people who meditate in an institutional context with feedback from a seasoned veteran vs. those who meditate based on e.g. enthusiastic blog posts?

Not saying you're wrong, but answers to things like this would help me know what to do with your observations.

I'm pretty sure I'm ontologically clueless in the way you describe. There are some very fundamental things in my worldview that I suspect are misapprehensions but that I don't know how to confidently replace with anything better. The yet-unimagined right answer to any of them could potentially knock out the pillars holding up much of what I think I know. It's unsettling when I pause to think about it, but I muddle through.

Might be worth mentioning Kuhn's "paradigm shifts" as examples of branches of knowledge jumping from one local maximum to another one, and having to resort their conceptual categories thereafter.

Alasdair MacIntyre's histories of ethical philosophy also highlight how sometimes when a field jumps from one local maximum to another, it brings along old conceptual categories that no longer can find a place but they continue to haunt as weird, ghostly apparitions.

I wonder if what makes alcohol superior to pharmacologically-similar drugs like diazepam in terms of socializing and bonding has less to do with the substance and its effects and more to do with the rituals and folklore around consumption.

Partially, I think alcohol just "comes on" much more quickly, and thus the drug and its effects are more tightly-linked mentally. But all of the hocus-pocus around e.g. mixing cocktails or discussing vintages or hops varieties or what-not, and then holding the potion in your hands in its specially-shaped glass, and yada yada... that's a lot of extra magic juju being added to the spell.

Put diazepam or gabapentin or what-have-you (or, who knows?, placebo?) in some exotically-shaped vehicle and administer it in some public and unusual ritual with its own exacting connoisseurship, and maybe you get everything you need.

Yeah, GHB is kind of nice in being alcohol-like without some of the worst effects. But I think the thin line between effective-dose and unconscious-dose makes it unlikely to fill this niche (remembering a Burning Man 20 years or so ago in which we inadvertently posed as Camp Jonestown Massacre after a couple dozen people took just a little too much and ended up splayed at random locations).

Thinking slightly outside the box here... seems that most of the benefits you identify could also be satisfied by a drug (e.g. valium) that has similar effects without being so freaking toxic (though I understand valium is similarly addictive, alas). Maybe we need to bring back recreational valium. Casual pop-culture reading suggests to me that back in the 70s-80s, valium & quaaludes & the like were quite the thing in party circles.

Some (actually, Much) additional nuance about empathy and its components that might add some context to this discussion at this link: https://www.lesswrong.com/posts/SMziBSCT9fiz5yG3L/notes-on-empathy

In particular, as other commenters have already pointed out, there is something about "perspective taking" (the shift from considering how you feel about what they feel to considering how they feel) that may be resulting in the OP's frustration.