Sure. I'm really glad to hear. This is not my community, but you did explicitly ask.
This is just off the top of my head, and I don't mean it to be a final complete and correct list. It's just to give you a sense of some things I've encountered, and to help you and your org think about how to empower people and help them flourish. Academia uses a lot of these to avoid the geek-MOP-sociopath cycle.
I'm assuming your institution wants to follow an academic model, including teaching, mentorship, hiearchical student-teacher relationships, etc.
An open question is when you have a duty of care. My rule of thumb is (1) when you or the org is explicitly saying "I'm your teacher", "I'm your mentor"; (2) when you feel a power imbalance with someone because this relationship has arisen implicitly; (3) when someone is soliciting this role from you, whether you want it or not.
If you're a business making money, that's quite different, just say "we're going to use your body and mind to make money" and you've probably gotten your informed consent. :)
* Detection
1. Abuse is non-Gaussian. A small number of people may experience a great deal, while the majority see nothing wrong. That means that occasional random sampling is not going to identify problems. There are a lot of comments here saying "XYZ org (etc) was great, I saw nothing bad" — this is not a good signal.
2. Women and people from marginalized groups are at much higher risk. They're less able to trust a random stranger, and they're also less able to appeal to social norms or law enforcement. They also are at higher risk if they do report.
Somebody in the comments said that many of the people reporting abuse are trans, and "trans people suffer from mental illness more", so maybe they're just crazy and everything was actually pretty OK.
Hopefully this reasoning looks as crazy to you as it does to me; in the 1970s people would have said the same about gay people, but now we realize that a lot of that was due to homophobia (etc), and a lot of it was due to the fact that gay people, being marginalized, made soft targets for manipulation, blackmail, etc.
3. Always take reports seriously, even if "the person seems weird".
* Prevention
4. The most obvious thing is use common sense. If they didn't need it at Solvay in 1927, you probably don't need it now.
For example, avoid weird hyper-personal psychological interventions (circling, debugging, etc). Therapy is a regulated profession for good reasons, and the evidence-based therapies we know about have safeguards (e.g., asymmetric privacy, theraputic alliance, regulations about sexual activity and business relationships, boards that manage complaints, etc.)
5. Obey the law. Don't allow underage drinking, illegal drug use, drunk driving, etc., and don't allow others to allow it. Have a zero tolerance policy on this (if that feels like a buzzkill, you can say it has to do with liability).
The reason for this is (IMO) actually quite interesting. It's not that the law is necessarily a good guide to morals. It's more that abusers tend to be out of control (because they have a psychological disorder, because they think they're above or beyond ordinary requirements, or because they're abusing drugs themselves, etc), and violating the law is a sign of this.
The extreme example I know of is the high school personality cult I mentioned above. The colleague in question was (in retrospect) terrifying: he engaged in animal abuse and setting fires (two of the Macdonald triad), and the young men in his cult engaged in sexual abuse.
In the end, however, he was "busted" (fired) for statutory rape. The other stuff going on was too fuzzy, gradual, and excusable to hit people's radar at first (think boiling frog). But SR is a bright line, and if someone's crazy enough to cross that line, it's a signal that other things are off as well.
6. Preserve personal-professional boundaries with students/mentees. A baseline assumption is that you shouldn't really know much about anyone's personal life -- who they're dating, what their mental problems are, what kind of sex they like. It's not forbidden knowledge, but if you (or someone else in the org) does, you might ask: to what end? Is this helping them thrive?
Similarly, respect when someone wants those boundaries, or when they want to re-establish them.
Dating and sexual relationships across the student-teacher boundary should be completely out.
* Mitigation
7. When powerful people in your group say that an abusive person is an advsior, that sends a message to vulnerable people that they ought to, or need to, tolerate abuse by that person in other contexts. If you believe a person is abusing vulnerable people to whom you, or the org, owe a duty of care, you ought to cut communication with that person.
8. Don't give charismatic or "high performing" people a pass. There's no real correlation between excellence and being abusive -- if anything, the positive correlation with drug and mental problems makes it a negative correlation. Meanwhile, the same thing that can enable abuse (dark triad traits) can also appear as high performance.
9. Done right, none of this requires drama. Among other things, if your org is aware of (1) through (6), abusers will go elsewhere. Having zero tolerance also makes it a lot easier to help good-faith people not abuse unintentionally -- you can step in before things go off the rails, when the stakes are low, and save important relationships.
EDIT: since you asked where academia is falling short. I'd say it falls short in (1) and (2), is sort of OK in (3) in part because of Title IX and similar things, is good in (4) in part because there are long-standing traditions of common sense, and in (5) because lawyers, and falls short in (6), (7), and (8).
What's being described here seems to be violating all eight rules at different levels. Most obvious to me from the outside is (1), (4), (5), (6) and (7).
EDIT2: since this came up. Good practice is that the vulnerable person can't waive these concerns.
For example, the answer to "but I want to do [intense weird psychological thing] with my mentor" should be "not as long as you or this mentor remains with the org", or at the very least "not as long as this mentor remains with the org with a duty of care towards you".
I want to provide an outside view that people might find helpful. This is based on my experience as a high school teacher (6 months total experience), a professor at an R1 university (eight years total experience), and someone who has mentored extraordinarily bright early-career scientists (15 years experience).
It’s very clear to me that the rationalist community is acting as a de facto school and system of interconnected mentorship opportunities. In some cases (CFAR, e.g.) this is explicit.
Academia also does this. It has ~1000 years of experience, dating from the founding of the University of Cambridge, and has learned a few things in that time.
An important discovery is that there are serious responsibilities that come with attending on “young” minds (young in quotes; generically the first quarter of life, depending on era, that’s <15 up to today around <30). These minds are considered inherently vulnerable, who need to be protected from manipulation, boundary violations, etc. It’s been discovered that making this a blanket and non-negotiable rule has significant positive epistemic and moral effects that haven’t been replicated with alternatives.
Even before academic institutions, this is seen: consider the extensive discussions in the Socratic dialogues. In later eras it is implicit in the phrase in loco parentis. Historically this has appeared as concern for the soul, or (in the post-religious era) psychological health. Importantly, the young minds are not allowed to waive this concern.
I see two places where the rationalist community has absolutely failed in implementing this responsible practice. The outcomes have been not sane.
Reading Scott Alexander’s comment above, the paragraph beginning with “I want to clarify that I don’t dislike Vassar”, my feeling is that I don’t care if powerful/influential people in the community are ok with the person, think it’s well-intentioned. Simply giving the high-status "he's a friend" mostly cancels out the effect of formal disinvitations, but it's clear that multiple vulnerable people, knowingly or not, are reporting harm.
I do not buy AI risk on this scale/urgency. But even if I did, I would consider the immediate duty of care to override these concerns. If I didn’t want to do that, I would not work with vulnerable young minds.
A final remark. When I was a high school teacher, it was a residential setting. A colleague decided to start a personality cult among the young men. It got extraordinarily messed up and abusive extraordinarily quickly (two weeks). The man was a sociopath; the young men were not, but they engaged in sexual abuse. This happened because of lax oversight from the principal who was in the process of retiring/handing over the reins.
I hope this helps. I wish potentially vulnerable young people (in this era, everyone under thirty) who see the rationalist community as a source of guidance and mentorship to take care of each other and demand more from influential people.
Please contact me privately if you have any concerns about what I’ve written above.
Edit: minor edits; miscounted years since my first faculty appointment.
This seems like the beginning of a very good discussion, but:
To that end, I'll suggest that if you like we chat offline. I'm in NYC, for example, and you're welcome to get in touch via PM.