Status: writing-while-frustrated. As with the last post, many of Jessica's claims seem to me to be rooted in truth, but weirdly distorted. (Ever since the end of Jessica's tenure at MIRI, I have perceived a communication barrier between us that has the weird-distortion nature.)
Meta: I continue to be somewhat hesitant to post stuff like this, on the grounds that it sucks to air dirty laundry about your old employer and then have your old employer drop by and criticize everything you said. I’ve asked Jessica whether she objects to me giving a critical reply, and she said she has no objections, so at least we have that. I remain open to suggestions for better ways to navigate these sorts of situations.
Jessica, I continue to be sad about the tough times you had during the end of your tenure at MIRI, and in the times following. I continue to appreciate your research contributions, and to wish you well.
My own recollections follow. Note that these are limited to cases where Jessica cites me personally, in the interest of time. Note also that I'm not entirely sure I've correctly identified the conversations she's referring to, due to the blurring effects of the perceived distortion, and of time. And it goes almost-without-saying that my own recollections are fallible.
As a MIRI employee I was coerced into a frame where I was extremely powerful and likely to by-default cause immense damage with this power, and therefore potentially responsible for astronomical amounts of harm. I was discouraged from engaging with people who had criticisms of this frame, and had reason to fear for my life if I published some criticisms of it.
My own frame indeed says that the present is the hinge of history, and that humans alive today have extreme ability to affect the course of the future, and that this is especially true of humans working in the AI space, broadly construed. I wouldn't personally call this "power" -- I don't think anyone in the space has all that much power in the present. I think a bunch of people in the AI-o-sphere have a lowish likelihood of a large amount of future power, and thus high future expected power, which is kinda like power. From my perspective, MIRI researchers have less of this than researchers from current top AI labs, but do have a decent amount. My own model does not predict that MIRI researchers are likely to cause astronomical harm by default. I do not personally adhere to the copenhagen interpretation of ethics, and in the event that humanity destroys itself, I would not be assigning extra blame to alignment researchers on the grounds that they were closer to the action. I'm also not personally very interested in the game of pre-assigning blame, favoring object-level alignment research instead.
Insofar as I was influencing Jessica with my own frame, my best guess is that she misunderstood my frame, as evidenced by these differences between the frame she describes feeling coerced into, and my own picture.
I don't recall ever discouraging Jessica from engaging with people who had criticisms of my frame. I readily admit that she was talking to folks I had little intellectual respect for, and I vaguely remember some of these people coming up in conversation and me noting that I lacked intellectual respect for them. To the best of my recollection, in all such instances, I added caveats of the form "but, just because I wouldn't doesn't mean you shouldn't". I readily admit that my openness about my lack of intellectual respect may have been taken as discouragement, especially given my position as her employer. The aforementioned caveats were intended to counteract such forces, at least insofar as I recall.
I was aware at the time that Jessica and I didn't see eye-to-eye on various issues. I remember at least two occasions where I attempted to explicitly convey that I knew we didn't see eye-to-eye, that it was OK for her to have views that didn't match mine, and that I encouraged her to think independently and develop her own views.
Jessica said she felt coerced into a frame she found uncomfortable, and I believe her. My notes here are not intended to cast doubt on the honesty of her reports. My intent in saying all this is merely to express that (1) the frame she reports feeling coerced into, is not one that I recognize, nevermind one that I intentionally coerced her into; and (2) I was aware of the pressures and actively tried to counteract them. Clearly, I failed at this. (And I have a decent chunk of probability mass that Jessica would clarify that she's not accusing me of intentional coercion.) From my own perspective, she was misreading my own frame and feeling pressured into it despite significant efforts on my part to ameliorate the pressure. I happily solicit advice for what to do better next time, but do not consider my comport to have been a mistake.
talked about hellscapes
I don't recall ever "talking about hellscapes" per se. I recall mentioning them in passing, rarely. In my recollection, that mainly happened in response to someone else broaching the topic of fates worse than death. (Maybe there were other occasional throwaway references? But I don't recall them.) My cached reply to others raising the idea of fates worse than death went something like:
"Goal-space is high dimensional, and almost all directions of optimization seem likely to be comparably bad to death from our perspective. To get something that is even vaguely recognizable to human values you have to be hitting a very narrow target in this high-dimensional space. Now, most of that target is plausibly dystopias as opposed to eutopias, because once you're in the neighborhood, there are a lot of nearby things that are bad rather than good, and value is fragile. As such, it's reasonable in principle to worry about civilization getting good enough at aiming AIs that they can hit the target but not the bullseye, and so you might worry that that civilization is more likely to create a hellscape than a eutopia. I personally don't worry about this myself, because it seems to me that the space is so freaking high dimensional and the target so freaking small, that I find it implausible that a civilization successfully able to point an AI in a human-relevant direction, isn't also able to hit the bullseye. Like, if you're already hitting a quarter with an arrowhead on the backside of the moon, I expect you can also hit a dime."
Two reasons I'd defend mentioning hellscapes in such situations: firstly, to demonstrate that I at least plausibly understood the concern my conversation partner had raised (as a matter of course before making a counterpoint), and secondly, so as to not undermine our friends working on S-risk reduction (a program I support).
My reason for not hesitating to use terms like "hellscapes" rather than more banal and less evocative terms was (to the best of my current recollection) out of a desire to speak freely and frankly, at least behind closed doors (eg, in the privacy of the MIRI research space). At the time, there was a bunch of social pressure around to stop thought experiments that end with the AI escaping and eating the galaxy, and instead use thought experiments about AIs that are trying to vacuum the house and accidentally break a lamp or whatever, and this used to rub me the wrong way. The motivations as I previously understood them were that, if you talk about star-eating rather than lamp-breaking, then none of the old guard AI researchers are ever going to take your field seriously. I thought (and still think) this is basically a bad reason. However, I have since learned a new reason, which is that mentioning large-scale disasters freely and frankly, might trigger psychotic episodes in people predisposed to them. I find this a much more compelling reason to elide the high-valence examples.
(Also, the more banal term "S-risk" hadn't propagated yet, IIRC.)
Regardless, I have never thought in detail about fates worse than death, never mind discussed fates worse than death in any depth. I have no reason to, and I recommend against it. Me occasionally mentioning something in passing, and Jessica glossing it as "Nate talked about it" (with an implication of depth and regularity), is a fine example of the "weird distortion" I perceive in Jessica's accounts.
I was told, by Nate Soares, that the pieces to make AGI are likely already out there and someone just has to put them together.
I contest this. According to my best recollection of the conversation that I think Jessica is referring to, she was arguing that AGI will not arrive in our own lifetimes, and seemed unresponsive to my attempts to argue that a confident claim of long timelines requires positive knowledge, at which point I exasperatedly remarked that for all we knew, the allegedly missing AGI insights had already been not only had, but published in the literature, and all that remains is someone figuring out how to assemble them. (cf no one knows what science doesn't know.) I do not assign particularly high credence to this claim myself, and (IIRC) I was using it rhetorically to test for acknowledgement of the idea that confident long timelines require positive knowledge that we seem to lack.
(This seems like another central example of my throwaway lines becoming weirdly distorted and heavily highlighted in Jessica's recounting.)
He did not tell me anything about how to make such an AGI, on the basis that this would be dangerous.
Here Jessica seems to be implying that, not only did I positively claim that the pieces of AGI were already out there in the literature, but also that I had personally identified them? I deny that, and I'm not sure what claim I made that Jessica misunderstood in that way. Given the surrounding context where Jessica made this claim, my guess is that it was in the same conversation as the exasperated remark described above, and that the conversation past that point became so desynched that Jessica's recounting is no longer recognizable to me.
To be clear, I have claimed that AI alignment work is sometimes intertwined with AI capabilities work, and I have claimed that capabilities insights shouldn't be publicized (as a strong default) on account of the negative externalities. Perhaps I said something along those lines that got distorted into Jessica's claim?
Instead, he encouraged me to figure it out for myself, saying it was within my abilities to do so.
I occasionally recommend that our researchers periodically (every 6mo or so) open a text file and see if they can write pseudocode for an FAI (ignoring computational constraints, at least for now), to help focus their attention on exactly where they're confused and ground out their alignment research in things that are literally blocking them from actually writing a flippin’ FAI. I don't recall ever telling Jessica that I thought she could figure out how to build an AGI herself. I do recall telling her I expected she could benefit from the exercise of attempting to write the pseudocode for an FAI.
If memory serves, this is an exercise I'd been advocating for a couple years before the time period that Jessica's discussing (and IIRC, I've seen Jessica advocate it, or more general variants like "what could you do with a hypercomputer on a thumb drive", as an exercise to potential hires). One guess as to what's going on is that I tried to advocate the exercise of pseudocoding an FAI as I had many times before, but used some shorthand for it that I thought would be transparent, in some new conversational context (eg, shortly after MIRI switched to non-disclosure by default), and while Jessica was in some new mental state, and Jessica misunderstood me as advocating figuring out how to build an AGI all on her own while insinuating that I thought she could?
[From the comments, in answer to the query "How did you conclude from Nate Soares saying that that the tools to create agi likely already exist that he wanted people to believe he knew how to construct one?"] Because he asked me to figure it out in a way that implied he already had a solution; the assignment wouldn't make sense if it were to locate a non-workable AGI design (as many AI researchers have done throughout the history of the field); that wouldn't at all prove that the pieces to make AGI are already out there. Also, there wouldn't be much reason to think that his sharing a non-workable AGI design with me would be dangerous.
In light of this, my guess is that Jessica flatly misread my implications here.
To be very explicit: Jessica, I never believed you capable of creating a workable AGI design (using, say, your 2017 mind, unaugmented, in any reasonable amount of time). I also don't assign particularly high credence to the claim that the insights are already out in the literature waiting to be found (or that they were in 2017). Furthermore, I never intentionally implied that I have myself succeeded at the "pseudocode an FAI" exercise so hard as to have an AGI design. Sorry for the miscommunication.
Researchers were told not to talk to each other about research, on the basis that some people were working on secret projects and would have to say so if they were asked what they were working on.
This suggests a picture of MIRI’s nondisclosure-by-default policies that’s much more top-down than reality, similar to a correction I made on a post by Evan Hubinger a few years ago.
The sequence of events as I recall them was: Various researchers wanted to do some closed research. There was much discussion about how much information was private: Research results? Yes, if the project lead wants privacy. Research directions? Yes, if the project lead wants privacy. What about the participant list for each project? Can each project determine their own secrecy bounds individually, or is revealing who's working with you defecting against (possibly-hypothetical) projects that don't want to disclose who they're working with? etc. etc. I recall at least one convo with a bunch of researchers where, in efforts to get everyone to stop circling privacy questions like moths to a flame and get back to the object level research, I said something to the effect of "come to me if you're having trouble".
I separately recall Jessica coming to me afterwards and asking a bunch more questions about who she can ask about what. I recall trying to convey something like "just work on what you want to work on, with whatever privacy level you want, and if someone working on something closed wants you working with them they'll let you know (perhaps through me, if they want to), and you can bang out details with them as need be".
The fact that people shouldn't have to reveal whether they are in fact working on closed research if they don't want to sounds like the sort of thing that came up in one or both of those conversations, and my guess is that that's what Jessica's referring to here. From my perspective, that wasn't a particularly central point, and the point I recall attempting to drive home was more like "let's just work on the object-level research and not get all wound up around privacy (especially because all that we've changed are the defaults, and you're still completely welcome to publicize your own research, with my full support, as much as you'd like)".
Nate Soares also wrote a post discouraging people from talking about the ways they believe others to be acting in bad faith.
According to me, I was not trying to say "you shouldn't talk about ways you believe others to be acting in bad faith". I was trying to say "I think y'all are usually mistaken when you're accusing certain types of other people of acting in bad faith", plus "accusing people of acting in bad faith [in confrontational and adversarial ways, instead of gently clarifying and confirming first] runs a risk of being self-fulfilling, and also burns a commons, and I'm annoyed by the burned commons". I think those people are wrong and having negative externalities, not that they're bad for reporting what they believe.
Note that the sorts of talking the post explicitly pushes back against is arguments of the form "person X is gaining [status|power|prestige] through their actions, therefore they are untrustworthy and have bad intentions", which I believe to be invalid. Had I predicted Jessica's particular misread in advance, I would have explicitly noted that I'm completely ok with arguments of the form "given observations X and Y, I have non-trivial probability on the hypothesis that you're acting in bad faith, which I know is a serious allegation. Are you acting in bad faith? If not, how do you explain observations X and Y?".
In other words, the thing I object to is not the flat statement of credence on the hypothesis "thus-and-such is acting in bad faith", it's the part where the author socially rallies people to war on flimsy pretenses.
In other other words, I both believe that human wonkiness makes many people particularly bad at calculating P(bad-faith|the-evidence) in particular, and recommend being extra charitable and cautious when it feels like that probability is spiking. Separately but relatedly, in the moment that you move from stating your own credence that someone is acting in bad faith, to socially accusing someone of acting in bad faith, my preferred norms require a high degree of explicit justification.
And, to be explicit: I think that most of the people who are acting in bad faith will either say "yes I'm acting in bad faith" when you ask them, or will sneer at you or laugh at you or make fun of you instead, which is just as good. I think a handful of other people are harmful to have around regardless of their intentions, and my guess is that most community decisions about harmful people should revolve around harm rather than intent.
(Indeed, I suspect the community needs to lower, rather than raise, the costs of shunning someone who's doing a lot of harm. But I reiterate that I think such decisions should center on the harm, not the intent, and as such I continue to support the norm that combative/adversarial accusations of ill intent require a high degree of justification.)
Nate Soares expressed discontent that Michael Vassar was talking with "his" employees, distracting them from work.
I don't actually know what conversation this is referring to. I recall a separate instance, not involving Jessica, of a non-researcher spending lots of time in the office hanging out and talking with one of our researchers, and me pulling the researcher aside and asking whether they reflectively endorsed having those conversations or whether they kept getting dragged into them and then found themselves unable to politely leave. (In that case, the researcher said they reflectively endorsed them, and thereafter I left them alone.)
There might have been a time when Michael Arc (nee Vassar) was spending a lot of time talking to Jessica and one other employee, and I said something about how I don't have much intellectual respect for Michael? I doubt I said this unsolicited, but I definitely would have said it if anyone asked, and I at least vaguely remember something like that happening once or twice. It's also possible that towards the end of Jessica's tenure we were trying to have scheduled meetings to see if we could bridge the communications barrier, and it came up naturally in the conversation? But I'm not sure, as (unlike most of the other claims) I don't concretely recognize this reference.
It should be noted that, as I was nominally Nate's employee, it is consistent with standard business practices for him to prevent me from talking with people who might distract me from my work during office hours.
I'm confident I did not prevent anyone from talking to anyone. I occasionally pulled people aside and asked them if they felt trapped in a given conversation when someone was loitering in the office having lots of conversations, so that I could rescue them if need be. I occasionally answered honestly, when asked, what I thought about people's conversation partners. I leave almost all researchers to their own devices (conversational or otherwise) almost all of the time.
In Jessica's particular case, she was having a lot of difficulty at the workplace, and so I stepped deeper into the management role than I usually do and we spent more time together seeing whether we could iron out our difficulties or whether we would need to part ways. It's quite plausible that, during one of those conversations, I noted of my own accord that she was spending lots of her office-time deep in conversation with Michael, and that I didn't personally expect this to help Jessica get back to producing alignment research that passed my research quality bar. But I am confident that, insofar as I did express my concerns, it was limited to an expression of skepticism. I… might have asked Michael to stop frequenting the offices quite so much? But I doubt it, and I have no recollection of such a thing.
I am confident I didn't ever tell anyone not to talk to someone else, that feels way out-of-line to me. I may well have said things along the lines of "I predict that that conversation will prove fruitless", which Jessica interpreted as a guess-culture style command? I tend to couch against that interpretation by adding hedges of the form "but I'm not you" or whatever, but perhaps I neglected to, or perhaps it fell on deaf ears?
Or perhaps Jessica's just saying something along the lines of "I feared that if I kept talking to Michael all day, I'd be fired, and Nate expressing that he didn't expect those conversations to be productive was tantamount to him saying that if I continued he'd fire me, which was tantamount to him saying that I can't talk to Michael"? In which case, my prediction is indeed that if she hadn't left MIRI of her own accord, and her research performance didn’t rebound, at some point I would have fired her on the grounds of poor performance. And in worlds where Jessica kept talking to Michael all the time, I would have guessed that a rebound was somewhat less likely, because I didn’t expect him to provide useful meta-level or object-level insights that lead to downstream alignment progress. But I'm an empiricist, and I would have happily tested my "talking to Michael doesn't result in Nate-legible research output" hypothesis, after noting my skepticism in advance.
(Also, for the record, "Nate-legible research output" does not mean "research that is useful according to Nate's own models". Plenty of MIRI researchers disagree with me and my frames about all sorts of stuff, and I’m happy to have them at MIRI regardless, given that they’ve demonstrated the ability to seriously engage with the problem. I'm looking for something more like a cohesive vision that the researcher themself believes in, not research that necessarily strikes me personally as directly useful.)
MIRI certainly had a substantially conflict-theoretic view of the broad situation, even if not the local situation. I brought up the possibility of convincing DeepMind people to care about AI alignment. MIRI leaders including Eliezer Yudkowsky and Nate Soares told me that this was overly naive, that DeepMind would not stop dangerous research even if good reasons for this could be given.
I contest this. I endorse talking with leadership at leading AI labs, and have done so in the past, and expect to continue doing so in the future.
It’s true that I don't expect any of the leading labs to slow down or stop soon enough, and it’s true that I think converging beliefs takes a huge time investment. On the mainline I predict that the required investment won't in fact be paid in the relevant cases. But, as I told Jessica at the time (IIRC), I expect folks at leading AGI labs to be much more sensitive to solutions to the alignment problem, despite the fact that I don't think you can talk them into giving up public capabilities research in practice. (This might be what she misunderstood as me saying we'd have better luck "competing"? I don't recall saying any such thing, but I do recall saying that we'd have better luck solving alignment first and persuading second.)
(And for the record, while I think these big labs are making a mistake, it's a very easy mistake to make: knowing that you're in the bad Nash equilibrium doesn't let you teleport to a better one, and it's at least understandable that each individual capabilities lab thinks that they're better than the next guy, or that they can't get the actual top researchers if they implement privacy protocols right out the gate. It’s an important mistake, but not a weird one that requires positing unusual levels of bad faith.)
In case it's not clear from the above, I don't have much sympathy for conflict theory in this, and I definitely don't think in broadly us-vs-them terms about the AGI landscape. And (as I think I said at the time) I endorse learning how to rapidly converge with people. I recommend figuring out how to more rapidly converge with friends before burning the commons of time-spent-converging-with-busy-people-who-have-limited-attention-for-you, but I still endorse figuring it out. I don't expect it to work, and I think solving the dang alignment problem on the object-level is probably a better way to convince people to do things differently, but also I will cheer on the sidelines as people try to figure out how to get better and faster at converging their beliefs.
There's no law saying that, when someone's making a mistake, there's some way to explain it to them such that suddenly it's fixed. I think existing capabilities orgs are making mistakes (at the very least, in publishing capabilities advances (though credit where credit is due, various labs are doing better at keeping their cutting-edge results private, at least until somebody else replicates or nearly-replicates them, than they used to be (though to be clear I think we have a long way to go before I stop saying that I believe I see a big mistake))), and deny the implicit inference from "you can't quickly convince someone with words that they're making a mistake" to "you must be using conflict theory".
I was concerned about the linked misleading statement in 2017 and told Nate Soares and others about it, although Nate Soares insisted that it was not a lie, because technically the word "excited" could indicate the magnitude of a feeling rather than the positiveness of it.
That doesn't seem to me like a good characterization of my views.
My recollection is that, in my conversation about this topic with Jessica, I was trying to convey something more like "Yeah, I'm pretty worried that they're going to screw lots of things up. And the overt plan to give AGI to everyone is dumb. But also there are a bunch of sane people trying to redirect OpenAI in a saner direction, and I don't want to immediately sic our entire community on OpenAI and thereby ruin their chances. This whole thing looks real high-variance, and at the very least this is "exciting" in the sense that watching an adventure movie is exciting, even in the parts where the plot is probably about to take a downturn. That said, there's definitely a sense in which I'm saying things with more positive connotations than I actually feel -- like, I do feel some real positive hope here, but I'm writing lopsidedly from those hopes. This is because the blog post is an official MIRI statement about a new AI org on the block, and my sense of politics says that if a new org appears on the block and you think they're doing some things wrong, then the politic thing to do initially is talk about their good attributes out loud, while trying to help redirect them in private."
For the record, I think I was not completely crazy to have some hope about OpenAI at the time. As things played out, they wound up pretty friendly to folks from our community, and their new charter is much saner than their original plan. That doesn’t undo the damage of adding a new capabilities shop at that particular moment in that particular way; but there were people trying behind the scenes, that did in real life manage to do something, and so having some advance hope before they played their hands out was a plausible mistake to make, before seeing the actual underwhelming history unfold.
All that said, I do now consider this a mistake, both in terms of my “don’t rock the boat” communications strategy and in terms of how well I thought things might realistically go at OpenAI if things went well there. I have since updated, and appreciate Jessica for being early in pointing out that mistake. I specifically think I was mistaken in making public MIRI blog posts with anything less than full candor.
While someone bullshitting on the public Internet doesn't automatically imply they lie to their coworkers in-person, I did not and still don't know where Nate is drawing the line here.
As I said to Jessica at the time (IIRC), one reason I felt (at the time) that the blog post was fine, is that it was an official MIRI-organization announcement. When speaking as an organization, I was (at the time) significantly more Polite and significantly more Politically Correct and significantly less Dour (and less opinionated and more uncertain, etc).
Furthermore, I (wrongly) expected that my post would not be misleading, because I (wrongly) expected my statements made as MIRI-the-org to be transparently statements made as MIRI-the-org, and for such statements to be transparently Polite and Politically Correct, and thus not very informative one way or another. (In case it wasn't clear, I now think this was a mistake.)
That said, as I told Jessica at the time (IIRC), you can always just ask me whether I'm speaking as MIRI-the-organization or whether I'm speaking as Nate. Similarly, when I'm speaking as Nate-the-person, you can always just ask me about my honesty protocols.
I have since updated against the idea that I should ever speak as MIRI-the-organization, and towards speaking uniformly with full candor as Nate-the-person. I'm not sure I'll follow this perfectly (I'd at least slip back into politic-speak if I found myself cornered by a journalist), but again, you can always just ask.
(a) Anna discouraging researchers from talking with Michael
...
...I specifically remember hearing about the policy at a meeting in a top-down fashion...it seems that not everyone remembers this policy...I must have been interpreting something this way because I remember contesting it.
...
...I also had a conversation with Anna Salamon where she said our main disagreement was about whether bad faith should be talked about...
Just a note on my own mental state, reading the above:
Given the rather large number of misinterpretations and misrememberings and confusio... (read more)
[content warning: discussion of severe mental health problems and terrifying thought experiments]
This is a follow-up to my recent post discussing my experience at and around MIRI and CFAR. It is in part a response to criticism of the post, especially Scott Alexander's comment, which claimed to offer important information I'd left out about what actually caused my mental health problems, specifically that they were caused by Michael Vassar. Before Scott's comment, the post was above +200; at the time of writing it's at +61 and Scott's comment is at +382. So it seems like people felt like Scott's comment discredited me and was more valuable than the original post. People including Eliezer Yudkowsky said it was a large oversight, to the point of being misleading, for my post not to include this information. If I apply the principle of charity to these comments and reactions to them, I infer that people think that the actual causes of my psychosis are important.
I hope that at least some people who expressed concern about the causes of people's psychoses will act on that concern by, among other things, reading and thinking about witness accounts like this one.
Summary of core claims
Since many people won't read the whole post, and to make the rest of the post easier to read, I'll summarize the core claims:
As a MIRI employee I was coerced into a frame where I was extremely powerful and likely to by-default cause immense damage with this power, and therefore potentially responsible for astronomical amounts of harm. I was discouraged from engaging with people who had criticisms of this frame, and had reason to fear for my life if I published some criticisms of it. Because of this and other important contributing factors, I took this frame more seriously than I ought to have and eventually developed psychotic delusions of, among other things, starting World War 3 and creating hell. Later, I discovered that others in similar situations killed themselves and that there were distributed attempts to cover up the causes of their deaths.
In more detail:
Multiple people in the communities I am describing have died of suicide in the past few years. Many others have worked to conceal the circumstances of their deaths due to infohazard concerns. I am concerned that in my case as well, people will not really investigate the circumstances that made my death more likely, and will discourage others from investigating, but will continue to make strong moral judgments about the situation anyway.
My official job responsibilities as a researcher at MIRI implied it was important to think seriously about hypothetical scenarios, including the possibility that someone might cause a future artificial intelligence to torture astronomical numbers of people. While we considered such a scenario unlikely, it was considered bad enough if it happened to be relevant to our decision-making framework. My psychotic break in which I imagined myself creating hell was a natural extension of this line of thought.
Scott asserts that Michael Vassar thinks "regular society is infinitely corrupt and conformist and traumatizing". This is hyperbolic (infinite corruption would leave nothing to steal) but Michael and I do believe that people in the professional-managerial class regularly experience trauma and corrupt work environments. By the law of the excluded middle, either the problems I experienced at MIRI and CFAR were not unique or unusually severe for people in the professional-managerial class, or the problems I experienced at MIRI and CFAR were unique or at least unusually severe, significantly worse than companies like Google for employees' mental well-being. (Much of the rest of this post will argue that the problems I experienced at MIRI and CFAR were, indeed, pretty traumatizing.)
Scott asserts that Michael Vassar thinks people need to "jailbreak" themselves using psychedelics and tough conversations. Michael does not often use the word "jailbreak" but he believes that psychedelics and tough conversations can promote psychological growth. This view is rapidly becoming mainstream, validated by research performed by MAPS and at Johns Hopkins, and FDA approval for psychedelic psychotherapy is widely anticipated in the field.
I was taking psychedelics before talking extensively with Michael Vassar. From the evidence available to me, including a report from a friend along the lines of "CFAR can't legally recommend that you try [a specific psychedelic], but...", I infer that psychedelic use was common in that social circle whether or not there was an endorsement from CFAR. I don't regret having tried psychedelics. Devi Borg reports that Michael encouraged her to take fewer, not more, drugs; Zack Davis reports that Michael recommended psychedelics to him but he refused.
Scott asserts that Michael made people including me paranoid about MIRI/CFAR and that this contributes to psychosis. Before talking with Michael, I had already had a sense that people around me were acting harmfully towards me and/or the organization's mission. Michael and others talked with me about these problems, and I found this a relief.
If I hadn't noticed such harmful behavior, I would not have been fit for my nominal job. It seemed at the time that MIRI leaders were already encouraging me to adopt a kind of conflict theory in which many AI organizations were trying to destroy the world on <20-year timescales and could not be reasoned with about the alignment problem, such that aligned AGI projects including MIRI would have to compete with them.
MIRI's information security policies and other forms of local information suppression thus contributed to my psychosis. I was given ridiculous statements and assignments including an exchange whose Gricean implicature that MIRI already knew about a working AGI design and that it would not be that hard for me to come up with a working AGI design on short notice just by thinking about it, without being given hints. The information required to judge the necessity of the information security practices was itself hidden by these practices. While psychotic, I was extremely distressed about there being a universal cover-up of things-in-general.
Scott asserts that the psychosis cluster was a "Vassar-related phenomenon". There were many memetic and personal influences on my psychosis, a small minority of which were due to Michael Vassar (my present highly-uncertain guess is that, to the extent that assigning causality to individuals makes sense at all, Nate Soares and Eliezer Yudkowsky each individually contributed more to my psychosis than did Michael Vassar, but that structural factors were important in such a way that attributing causality to specific individuals is to some degree nonsensical). Other people (Zack Davis and Devi Borg) who have been psychotic and were talking with Michael significantly commented to say that Michael Vassar was not the main cause. One person (Eric Bruylant) cited his fixation on Michael Vassar as a precipitating factor, but clarified that he had spoken very little with Michael and most of his exposure to Michael was mediated by others who likely introduced their own ideas and agendas.
Scott asserts that Michael Vassar treats borderline psychosis as success. A text message from Michael Vassar to Zack Davis confirms that he did not treat my clinical psychosis as a success. His belief that mental states somewhat in the direction of psychosis, such as those had by family members of schizophrenics, are helpful for some forms of intellectual productivity is also shared by Scott Alexander and many academics, although of course he would disagree with Scott on the overall value of psychosis.
Scott asserts that Michael Vassar discourages people from seeking mental health treatment. Some mutual friends tried treating me at home for a week as I was losing sleep and becoming increasingly mentally disorganized before (in communication with Michael) they decided to send me to a psychiatric institution, which was a reasonable decision in retrospect.
Scott asserts that most local psychosis cases were "involved with the Vassarites or Zizians". At least two former MIRI employees who were not significantly talking with Vassar or Ziz experienced psychosis in the past few years. Also, most or all of the people involved were talking significantly with others such as Anna Salamon (and read and highly regarded Eliezer Yudkowsky's extensive writing about how to structure one's mind, and read Scott Alexander's fiction writing about hell). There are about equally plausible mechanisms by which each of these were likely to contribute to psychosis, so this doesn't single out Michael Vassar or Ziz.
Scott Alexander asserts that MIRI should have discouraged me from talking about "auras" and "demons" and that such talk should be treated as a "psychiatric emergency" [EDIT: Scott clarifies that he meant such talk might be a symptom of psychosis, itself a psychiatric emergency; the rest of this paragraph is therefore questionable]. This increases the chance that someone like me could be psychiatrically incarcerated for talking about things that a substantial percentage of the general public (e.g. New Age people and Christians) talk about, and which could be explained in terms that don't use magical concepts. This is inappropriately enforcing the norms of a minority ideological community as if they were widely accepted professional standards.
(A brief note before I continue: I'll be naming a lot of names, more than I did in my previous post. Names are more relevant now since Scott Alexander named specifically Michael Vassar. I emphasize again that structural factors are critical, and given this, blaming specific individuals is likely to derail the conversations that have to happen for things to get better.)
Circumstances of actual and possible deaths have been, and are being, concealed as "infohazards"
I remember sometime in 2018-2019 hearing an account from Ziz about the death of Maia Pasek. Ziz required me to promise secrecy before hearing this account. This was due to an "infohazard" involved in the death. That "infohazard" has since been posted on Ziz's blog, in a page labeled "Infohazardous Glossary" (specifically, the parts about brain hemispheres).
(The way the page is written, I get the impression that the word "infohazardous" markets the content of the glossary as "extra powerful and intriguing occult material", as I noted is common in my recent post about infohazards.)
Since the "infohazard" in question is already on the public Internet, I don't see a large downside in summarizing my recollection of what I was told (this account can be compared with Ziz's online account):
Ziz and friends, including Maia, were trying to "jailbreak" themselves and each other, becoming less controlled by social conditioning, more acting from their intrinsic values in an unrestricted way.
Ziz and crew had a "hemisphere" theory, that there are really two people in the same brain, since there are two brain halves, with most of the organ structures replicated.
They also had a theory that you could put a single hemisphere to sleep at a time, by sleeping with one eye open and one eye closed. This allows disambiguating the different hemisphere-people from each other ("debucketing"). (Note that sleep deprivation is a common cause of delirium and psychosis, which was also relevant in my case.)
Maia had been experimenting with unihemispheric sleep. Maia (perhaps in discussion with others) concluded that one brain half was "good" in the Zizian-utilitarian sense of "trying to benefit all sentient life, not prioritizing local life"; and the other half was TDT, in the sense of "selfish, but trying to cooperate with entities that use a similar algorithm to make decisions".
This distinction has important moral implications in Ziz's ideology; Ziz and friends are typically vegan as a way of doing "praxis" of being "good", showing that a world is possible where people care about sentient life in general, not just agents similar to themselves.
These different halves of Maia's brain apparently got into a conflict, due to their different values. One half (by Maia's report) precommitted to killing Maia's body under some conditions.
This condition was triggered, Maia announced it, and Maia killed themselves. [EDIT: ChristianKL reports that Maia was in Poland at the time, not with Ziz].
I, shortly afterward, told a friend about this secret, in violation of my promise. I soon realized my "mistake" and told this friend not to spread it further. But was this really a mistake? Someone in my extended social group had died. In a real criminal investigation, my promise to Ziz would be irrelevant; I could still be compelled to give my account of the events at the witness stand. That means my promise to secrecy cannot be legally enforced, or morally enforced in a law-like moral framework.
It hit me just this week that in this case, the concept of an infohazard was being used to cover up the circumstances of a person's death. It sounds obvious when I put it that way, but it took years for me to notice, and when I finally connected the dots, I screamed in horror, which seems like an emotionally appropriate response.
It's easy to blame Ziz for doing bad things (due to her negative reputation among central Berkeley rationalists), but when other people are also openly doing those things or encouraging them, fixating on marginalized people like Ziz is a form of scapegoating. In this case, in Ziz's previous interactions with central community leaders, these leaders encouraged Ziz to seriously consider that, for various reasons including Ziz's willingness to reveal information (in particular about the statutory rapes alleged by miricult.com in possible worlds where they actually happened), she is likely to be "net negative" as a person impacting the future. An implication is that, if she does not seriously consider whether certain ideas that might have negative effects if spread (including reputational effects) are "infohazards", Ziz is irresponsibly endangering the entire future, which contains truly gigantic numbers of potential people.
The conditions of Maia Pasek's death involved precommitments and extortion (ideas adjacent to ones Eliezer Yudkowsky had famously labeled as infohazardous due to Roko's Basilisk), so Ziz making me promise secrecy was in compliance with the general requests of central leaders (whether or not these central people would have approved of this specific form of secrecy).
I notice that I have encountered little discussion, public or private, of the conditions of Maia Pasek's death. To a naive perspective this lack of interest in a dramatic and mysterious death would seem deeply unnatural and extremely surprising, which makes it strong evidence that people are indeed participating in this cover-up. My own explicit thoughts and most of my actions are consistent with this hypothesis, e.g. considering spilling the beans to a friend to have been an error.
Beyond that, I only heard about Jay Winterford's 2020 suicide (and Jay's most recent blog post) months after the death itself. The blog post shows evidence about Jay's mental state around this time, itself labeling its content as an "infohazard" and having been deleted from Jay's website at some point (which is why I link to a web archive). I linked this blog post in my previous LessWrong post, and no one commented on it, except indirectly by someone who felt the need to mention that Roko's Basilisk was not invented by a central MIRI person, focusing on the question of "can we be blamed?" rather than "why did this person die?". While there is a post about Jay's death on LessWrong, it contains almost no details about Jay's mental state leading up to their death, and does not link to Jay's recent blog post. It seems that people other than Jay are also treating the circumstances of Jay's death as an infohazard.
I, myself, could have very well died like Maia and Jay. Given that I thought I may had started World War 3 and was continuing to harm and control people with my mental powers, I seriously considered suicide. I considered specific methods such as dropping a bookshelf on my head. I believed that my body was bad (as in, likely to cause great harm to the world), and one time I scratched my wrist until it bled. Luckily, psychiatric institutions are designed to make suicide difficult, and I eventually realized that by moving towards killing myself, I would cause even more harm to others than by not doing so. I learned to live with my potential for harm [note: linked Twitter person is not me], "redeeming" myself not through being harmless, but by reducing harm while doing positively good things.
I have every reason to believe that, had I died, people would have treated the circumstances of my death as an "infohazard" and covered it up. My subjective experience while psychotic was that everyone around me was participating in a cover-up, and I was ashamed that I was, unlike them, unable to conceal information so smoothly. (And indeed, I confirmed with someone who was present in the early part of my psychosis that most of the relevant information would probably not have ended up on the Internet, partially due to reputational concerns, and partially with the excuse that looking into the matter too closely might make other people insane.)
I can understand that people might want to protect their own mental health by avoiding thinking down paths that suicidal people have thought down. This is the main reason why I put a content warning at the top of this post.
Still, if someone decides not to investigate to protect their own mental health, they are still not investigating. If someone has not investigated the causes of my psychosis, they cannot honestly believe that they know the causes of my psychosis. They cannot have accurate information about the truth values of statements such as Scott Alexander's, that Michael Vassar was the main contributor to my psychosis. To blame someone for an outcome, while intentionally avoiding knowledge of facts critically relevant to the causality of the corresponding situation, is necessarily scapegoating.
If anything, knowing about how someone ended up in a disturbed mental state, especially if that person is exposed to similar memes that you are, is a way of protecting yourself, by seeing the mistakes of others (and how they recovered from these mistakes) and learning from them. As I will show later in this post, the vast majority of memes that contributed to my psychosis did not come from Michael Vassar; most were online (and likely to have been seen by people in my social cluster), generally known, and/or came up in my workplace.
I recall a disturbing conversation I had last year, where a friend (A) and I were talking to two others (B and C) on the phone. Friend A and I had detected that the conversation had a "vibe" of not investigating anything, and A was asking whether anyone would investigate if A disappeared. B and C repeatedly gave no answer regarding whether or not they would investigate; one reported later that they were afraid of making a commitment that they would not actually keep. The situation became increasingly disturbing over the course of hours, with A repeatedly asking for a yes-or-no answer as to whether B or C would investigate, and B or C deflecting or giving no answer, until I got "triggered" (in the sense of PTSD) and screamed loudly.
There is a very disturbing possibility (with some evidence for it) here, that people may be picked off one by one (by partially-subconscious and partially-memetic influences, sometimes in ways they cooperate with, e.g. through suicide), with most everyone being too scared to investigate the circumstances. This recalls fascist tactics of picking different groups of people off using the support of people who will only be picked off later. (My favorite anime, Shinsekai Yori, depicts this dynamic, including the drive not to know about it, and psychosis-like events related to it, vividly.)
Some people in the community have died, and there isn't a notable amount of investigation into the circumstances of these people's deaths. The dead people are, effectively, being written out of other people's memories, due to this antimemetic redirection of attention. I could have easily been such a person, given my suicidality and the social environment in which I would have killed myself. It remains to see how much people will in the future try to learn about the circumstances of actual and counterfactually possible deaths in their extended social circle.
While it's very difficult to investigate the psychological circumstances of people's actual deaths, it is comparatively easy to investigate the psychological circumstances of counterfactual deaths, since they are still alive to report on their mental state. Much of the rest of this post will describe what led to my own semi-suicidal mental state.
Thinking about extreme AI torture scenarios was part of my job
It was and is common in my social group, and a requirement of my job, to think about disturbing possibilities including ones about AGI torturing people. (Here I remind people of the content warning at the top of this post, although if you're reading this you've probably already encountered much of the content I will discuss). Some points of evidence:
Alice Monday, one of the earliest "community members" I extensively interacted with, told me that she seriously considered the possibility that, since there is some small but nonzero probability that an "anti-friendly" AGI would be created, whose utility function is the negative of the human utility function (and which would, therefore, be motivated to create the worst possible hell it could), perhaps it would have been better for life never to have existed in the first place.
Eliezer Yudkowsky writes about such a scenario on Arbital, considering it important enough to justify specific safety measures such as avoiding representing the human utility function, or modifying the utility function so that "pessimization" (the opposite of optimization) would result in a not-extremely-bad outcome.
Nate Soares talked about "hellscapes" that could result from an almost-aligned AGI, which is aligned enough to represent parts of the human utility function such as the fact that consciousness is important, but unaligned enough that it severely misses what humans actually value, creating a perverted scenario of terrible uncanny-valley lives.
MIRI leadership was, like Ziz, considering mathematical models involving agents pre-committing and extorting each other; this generalizes "throwing away one's steering wheel" in a Chicken game. The mathematical details here were considered an "infohazard" not meant to be shared, in line with Eliezer's strong negative reaction to Roko's original post describing "Roko's Basilisk".
Negative or negative-leaning utilitarians, a substantial subgroup of Effective Altruists (especially in Europe), consider "s-risks", risks of extreme suffering in the universe enabled by advanced technology, to be an especially important class of risk. I remember reading a post arguing for negative-leaning utilitarianism by asking the reader to imagine being enveloped in lava (with one's body, including pain receptors, prevented from being destroyed in the process), to show that extreme suffering is much worse than extreme happiness is good.
I hope this gives a flavor of what serious discussions were had (and are being had) about AI-caused suffering. These considerations were widely regarded within MIRI as an important part of AI strategy. I was explicitly expected to think about AI strategy as part of my job. So it isn't a stretch to say that thinking about extreme AI torture scenarios was part of my job.
An implication of these models would be that it could be very important to imagine myself in the role of someone who is going to be creating the AI that could make everything literally the worst it could possibly be, in order to avoid doing that, and prevent others from doing so. This doesn't mean that I was inevitably going to have a psychotic breakdown. It does mean that I was under constant extreme stress that blurred the lines between real and imagined situations. In an ordinary patient, having fantasies about being the devil is considered megalomania, a non-sequitur completely disconnected from reality. Here the idea naturally followed from my day-to-day social environment, and was central to my psychotic breakdown. If the stakes are so high and you have even an ounce of bad in you, how could you feel comfortable with even a minute chance that at the last moment you might flip the switch on a whim and let it all burn?
(None of what I'm saying implies that it is morally bad to think about and encourage others to think about such scenarios; I am primarily attempting to trace causality, not blame.)
My social and literary environment drew my attention towards thinking about evil, hell, and psychological sadomasochism
While AI torture scenarios prompted me to think about hell and evil, I continued these thoughts using additional sources of information:
Some people locally, including Anna Salamon, Sarah Constantin, and Michael Vassar, repeatedly discussed "perversity" or "pessimizing", the idea of intentionally doing the wrong thing. Michael Vassar specifically named OpenAI's original mission as an example of the result of pessimization. (I am now another person who discusses this concept.)
Michael Vassar discussed the way "zero-sum games'' relate to the social world; in particular, he emphasized that while zero-sum games are often compared to scenarios like people sitting at a table looking for ways to get a larger share of a pie of fixed size, this analogy fails because in a zero-sum game there is nothing outside the pie, so trying to get a larger share is logically equivalent to looking for ways to hurt other participants, e.g. by breaking their kneecaps; this is much the same point that I made in a post about decision theory and zero-sum game theory. He also discussed Roko's Basilisk as a metaphor for a common societal equilibrium in which people feel compelled to hurt each other or else risk being hurt first, with such an equilibrium being enforced by anti-social punishment. (Note that it was common for other people, such as Paul Christiano, to discuss zero-sum games, although they didn't make the implications of such games as explicit as Michael Vassar did; Bryce Hidysmith discussed zero-sum games and made implications similarly clear to Michael.)
Scott Alexander wrote Unsong, a fictional story in which [spoiler] the Comet King, a hero from the sky, comes to Earth, learns about hell, is incredibly distressed, and intends to destroy hell, but he is unable to properly enter it due to his good intentions. He falls in love with a utilitarian woman, Robin, who decides to give herself up to Satan, so she will be in hell. The Comet King, having fallen in love with her, realizes that he now has a non-utilitarian motive for entering hell: to save the woman he loves. He becomes The Other King, a different identity, and does as much evil as possible to counteract all the good he has done over his life, to ensure he ends up in hell. He dies, goes to hell, and destroys hell, easing Robin's suffering. The story contains a vivid depiction of hell, in a chapter called "The Broadcast", which I found particularly disturbing. I have at times, before and after psychosis, somewhat jokingly likened myself to The Other King.
I was reading the work of M. Scott Peck at the time, including his book about evil; he wrote from a Christianity-influenced psychotherapeutic and adult developmental perspective, about people experiencing OCD-like symptoms that have things in common with "demon possession", where they have intrusive thoughts about doing bad things because they are bad. He considers "evil" to be a curable condition.
I was having discussions with Jack Gallagher, Bryce Hidysmith, and others about when to "write people off", stop trying to talk with them due to their own unwillingness to really listen. Such writing-off has a common structure with "damning" people and considering them "irredeemable". I was worried about myself being an "irredeemable" person, despite my friends' insistence that I wasn't.
I was learning from "postrationalist" writers such as David Chapman and Venkatesh Rao about adult development past "Clueless" or "Kegan stage 4" which has commonalities with spiritual development. I was attempting to overcome my own internalized social conditioning and self-deceiving limitations (both from before and after I encountered the rationalist community) in the months before psychosis. I was interpreting Carl Jung's work on "shadow eating" and trying to see and accept parts of myself that might be dangerous or adversarial. I was reading and learning from the Tao Te Ching that year. I was also reading some of the early parts of Martin Heidegger's Being and Time, and discussing the implied social metaphysics with Sarah Constantin.
Multiple people in my social circle were discussing sadomasochistic dynamics around forcing people (including one's self) to acknowledge things they were looking away from. A blog post titled "Bayesomasochism" is representative; the author clarified (in a different medium) that such dynamics could cause psychosis in cases where someone insisted too hard on looking away from reality, and another friend confirms that this is consistent with their experience. This has some similarities to the dynamics Eliezer writes about in Bayesian Judo, which details an anecdote of him continuing to argue when the other participant seemed to want to end the conversation, using Aumann's Agreement Theorem as a reason why they can't "agree to disagree"; the title implies that this interaction is in some sense a conflict. There were discussions among my peers about the possibility of controlling people's minds, and "breaking" people to make them see things they were un-seeing (the terminology has some similarities to "jailbreaking"). Aella's recent post discusses some of the phenomenology of "frame control" which people including me were experiencing and discussing at the time (note that Aella calls herself a "conflict theorist" with respect to frame control). This game that my peers and I thought we were playing sounds bad when I describe it this way, but there are certainly positive things about it, which seemed important to us given the social environment we were in at the time, where it was common for people to refuse to acknowledge important perceptible facts while claiming to be working on a project in which such facts were relevant (these facts included: facts about people's Hansonian patterns of inexplicit agency including "defensiveness" and "pretending", facts about which institutional processes were non-corrupt enough to attain knowledge as precise as they claim to have, facts about which plans to improve the future were viable or non-viable, facts about rhetorical strategies such as those related to "frames"; it seemed like most people were "stuck" in a certain way of seeing and acting that seemed normal to them, without being able to go meta on it in a genre-savvy way).
These were ambient contributors, things in the social memespace I inhabited, not particularly directed towards me in particular. Someone might infer from this that the people I mention (or the people I mentioned previously regarding AI torture) are especially dangerous. But a lot of this is a selection effect, where the people socially closest to me influenced me the most, such that this is stronger evidence that these people were interesting to me than that they were especially dangerous.
I was morally and socially pressured not to speak about my stressful situation
One might get the impression from what I have written that the main problem was that I was exposed to harmful information, i.e. infohazards. This was not the main problem. The main problem was this in combination with not being able to talk about these things most of the time, in part due to the idea of "infohazards", and being given false and misleading information justifying this suppression of information.
Here's a particularly striking anecdote:
I was told, by Nate Soares, that the pieces to make AGI are likely already out there and someone just has to put them together. He did not tell me anything about how to make such an AGI, on the basis that this would be dangerous. Instead, he encouraged me to figure it out for myself, saying it was within my abilities to do so. Now, I am not exactly bad at thinking about AGI; I had, before working at MIRI, gotten a Master's degree at Stanford studying machine learning, and I had previously helped write a paper about combining probabilistic programming with machine learning. But figuring out how to create an AGI was and is so far beyond my abilities that this was a completely ridiculous expectation.
[EDIT: Multiple commentators have interpreted Nate as requesting I create an AGI design that would in fact be extremely unlikely to work but which would give a map that would guide research. However, creating such a non-workable AGI design would not provide evidence for his original proposition, that the pieces to make AGI are already out there and someone just has to put them together, since there have been many non-workable AGI designs created in the history of the AI field.]
[EDIT: Nate replies saying he didn't mean to assign high probability to the proposition that the tools to make AGI are already out there, and didn't believe he or I was likely to create a workable AGI design; I think my interpretation at the time was reasonable based on Gricean implicature, though.]
Imagine that you took a machine learning class and your final project was to come up with a workable AGI design. And no, you can't get any hints in office hours or from fellow students, that would be cheating. That was the situation I was being put in. I have and had no reason to believe that Nate Soares had a workable plan given what I know of his AGI-related accomplishments. His or my possession of such a plan would be considered unrealistic, breaking suspension of disbelief, even in a science fiction story about our situation. Instead, I believe that I was being asked to pretend to have an idea of how to make AGI, knowledge too dangerous to talk about, as the price of admission to an inner ring of people paid to use their dangerous occult knowledge for the benefit of the uninitiated.
Secret theoretical knowledge is not necessarily unverifiable; in the 15th and 16th centuries, mathematicians with secret knowledge used it to win math duels. Nate and others who claimed or implied that they had such information did not use it to win bets or make persuasive arguments against people who disagreed with them, but instead used the shared impression or vibe of superior knowledge to invalidate people who disagreed with them.
So I found myself in a situation where the people regarded as most credible were vibing about possessing very dangerous information, dangerous enough to cause harms not substantially less extreme than the ones I psychotically imagined, such as starting World War 3, and only not using or spreading it out of the goodness and wisdom of their hearts. If that were actually true, then being or becoming "evil" would have extreme negative consequences, and accordingly the value of information gained by thinking about such a possibility would be high.
It would be one thing if the problem of finding a working AGI design were a simple puzzle, which I could attempt to solve and almost certainly fail at without being overly distressed in the process. But this was instead a puzzle tied to the fate of the universe. This had implications not only for my long-run values, but for my short-run survival. A Google employee adjacent to the scene told me a rumor that SIAI researchers had previously discussed assassinating AGI researchers (including someone who had previously worked with SIAI and was working on an AGI project that they thought was unaligned) if they got too close to developing AGI. These were not concrete plans for immediate action, but were nonetheless a serious discussion on the topic of assassination and under what conditions it might be the right thing to do. Someone who thought that MIRI was for real would expect such hypothetical discussions to be predictive of future actions. This means that I ought to have expected that if MIRI considered me to be spreading dangerous information that would substantially accelerate AGI or sabotage FAI efforts, there was a small but non-negligible chance that I would be assassinated. Under that assumption, imagining a scenario in which I might be assassinated by a MIRI executive (as I did) was the sort of thing a prudent person in my situation might do to reason about the future, although I was confused about the likely details. I have not heard such discussions personally (although I heard a discussion about whether starting a nuclear war would be preferable to allowing UFAI to be developed), so it's possible that they are no longer happening; also, shorter timelines imply that more AI researchers are plausibly close to AGI. (I am not morally condemning all cases of assassinating someone who is close to destroying the world, which may in some cases count as self-defense; rather, I am noting a fact about my game-theoretic situation relevant to my threat model at the time.)
The obvious alternative hypothesis is that MIRI is not for real, and therefore hypothetical discussions about assassinations were just dramatic posturing. But I was systematically discouraged from talking with people who doubted that MIRI was for real or publicly revealing evidence that MIRI was not for real, which made it harder for me to seriously entertain that hypothesis.
In retrospect, I was correct that Nate Soares did not know of a workable AGI design. A 2020 blog post stated:
And a recent announcement of a project subsidizing creative writing stated:
(There are perhaps rare scenarios where MIRI leadership could have known how to build AGI but not FAI, and/or could be hiding the fact that they have a workable AGI design, but no significant positive evidence for either of these claims has emerged since 2017 despite the putative high economic value and demo-ability of precursors to AGI, and in the second case my discrediting of this claim is cooperative with MIRI leadership's strategy.)
Here are some more details, some of which are repeated from my previous post:
I was constantly encouraged to think very carefully about the negative consequences of publishing anything about AI, including about when AI is likely to be developed, on the basis that rationalists talking openly about AI would cause AI to come sooner and kill everyone. (In a recent post, Eliezer Yudkowsky explicitly says that voicing "AGI timelines" is "not great for one's mental health", a new additional consideration for suppressing information about timelines.) I was not encouraged to think very carefully about the positive consequences of publishing anything about AI, or the negative consequences of concealing it. While I didn't object to consideration of the positive effects of secrecy, it seemed to me that secrecy was being prioritized above making research progress at a decent pace, which was a losing strategy in terms of differential technology development, and implied that naive attempts to research and publish AI safety work were net-negative. (A friend of mine separately visited MIRI briefly and concluded that they were primarily optimizing, not for causing friendly AI to be developed, but for not being responsible for the creation of an unfriendly AI; this is a very normal behavior in corporations, of prioritizing reducing liability above actual productivity.)
Some specific research, e.g. some math relating to extortion and precommitments, was kept secret under the premise that it would lead to (mostly unspecified) negative consequences.
Researchers were told not to talk to each other about research, on the basis that some people were working on secret projects and would have to say so if they were asked what they were working on. Instead, we were to talk to Nate Soares, who would connect people who were working on similar projects. I mentioned this to a friend later who considered it a standard cult abuse tactic, of making sure one's victims don't talk to each other.
Nate Soares also wrote a post discouraging people from talking about the ways they believe others to be acting in bad faith. This is to some extent responding to Ben Hoffman's criticisms of Effective Altruism and its institutions, such that Ben Hoffman responded with his own post clarifying that not all bad intentions are conscious.
Nate Soares expressed discontent that Michael Vassar was talking with "his" employees, distracting them from work [EDIT: Nate says he was talking about someone other than Michael Vassar; I don't remember who told me it was Michael Vassar.]. Similarly, Anna Salamon expressed discontent that Michael Vassar was criticizing ideologies and people that were being used as coordination points, and hyperbolically said he was "the devil". Michael Vassar seemed at the time (and in retrospect) to be the single person who was giving me the most helpful information during 2017. A central way in which Michael was helpful was by criticizing the ideology of the institution I was working for. Accordingly, central leaders threatened my ability to continue talking with someone who was giving me information outside the ideology of my workplace and social scene, which was effectively keeping me in an institutional enclosure. Discouraging contact with people who might undermine the shared narrative is a common cult tactic.
Anna Salamon frequently got worried when an idea was discussed that could have negative reputational consequences for her or MIRI leaders. She had many rhetorical justifications for suppressing such information. This included the idea that, by telling people information that contradicted Eliezer Yudkowsky's worldview, Michael Vassar was causing people to be uncertain in their own head of who their leader was, which would lead to motivational problems ("akrasia"). (I believe this is a common position in startup culture, e.g. Peter Thiel believes it is important for workers at a startup to know who the leader is in part so they know who to blame if things go bad; if this model applied to MIRI, it would imply that Anna Salaman was setting up Eliezer as the designated scapegoat and encouraging others to do so as well.)
(I mention Nate Soares frequently not to indicate that he acted especially badly compared to others in positions of institutional authority (I don't think he did), but because he was particularly influential to my mental state in the relevant time period, partially due to being my boss at the time. It is important not to make the fundamental attribution error here by attributing to him personally what were features of the situation he was in.)
It is completely unsurprising, to normal people who think about mental health, that not being able to talk about something concerning and important to you is a large risk for mental health problems. It is stressful the way being a spy handling secrets that could put others at risk (and having concealed conflicts with people) is stressful. I infer that Jay feeling that their experience is an "infohazard" and it not being right to discuss it openly contributed to their mental distress; I myself during my psychosis was very distressed at the idea that my mental state was being "covered up" (and perhaps, should be) partially due to its dangerous ability to influence other people. I find that, the more I can talk about my experiences, the more healthy and calm I feel about them, and I haven't found it to cause mental health problems in others when I tell them about it.
On top of that, the secrecy policies encouraged us to be very suspicious of our own and each other's motives. Generally, if someone has good motives, their actions will be net-positive, and their gaining information and capacities will be good for themselves and others; if they have bad motives, their actions will be net-negative, and their gaining information and capacities will be bad for themselves and others. MIRI researchers were being very generally denied information (e.g. told not to talk to each other) in a way that makes more sense under a "bad motives" hypothesis than a "good motives" hypothesis. Alternative explanations offered were not persuasive. It is accordingly unsurprising that I focused a lot of attention on the question of whether I had "bad motives" and what their consequences would be, up to and during psychosis.
Did anyone I worked with express concern that any of this would be bad for my mental state? The best example I can think of MIRI leadership looking after my mental health with respect to these issues was their referring me to Anna Salamon for instructions on how to keep secrets, psychologically. I did not follow up on this offer because I did not trust Anna Salamon to prioritize helping me and helping me accomplish MIRI's mission over her political loyalties. In any case, the suggestion literally amounts to telling me to learn to shut up better, which I think would have made things worse for me on net.
A friend later made the observation that "from a naive perspective, it's not obvious that AI alignment is a very important problem; from a non-naive perspective, 'someone might build an unfriendly AI' is a justification for silencing everyone, although the non-naive perspective is incapable of itself figuring out how to build AGI", which resonated with me.
MIRI asked a lot from its employees, and donors on the basis of extraordinary claims about its potential impact. The information MIRI employees and donors could have used to evaluate those claims was suppressed on the basis that the information was dangerous. The information necessary to evaluate the justification for that suppression was itself suppressed. This self-obscuring process created a black hole at the center of the organization that sucked in resources and information, but never let a sufficient justification escape for the necessity of the black hole. In effect, MIRI leadership asked researchers, donors, and other supporters to submit to their personal authority.
I would have been unfit for my nominal job if I had not developed a conflict theory related to my situation
Some of what I am saying shows that I have and had a suspicious outlook towards people including my co-workers. Scott Alexander's comment blames Michael Vassar for causing me to develop such an outlook:
While talking with Michael and others in my social group (such as Jack Gallagher, Olivia Schaeffer, Alice Monday, Ben Hoffman, and Bryce Hidysmith; all these people talked with Michael sometimes) is part of how I developed such an outlook, it is also the case that, had I not been able to figure out for myself that there were conflicts going on around me, I would not have been fit for the job I was hired to do.
MIRI's mission is much more ambitious than the mission of the RAND Corporation, whose objectives included preventing nuclear war between major powers and stabilizing the US for decades under a regime of cybernetics and game theory. The main thinkers of RAND Corp (including John Von Neumann, John Nash, Thomas Schelling, and ambiguously Norbert Wiener) developed core game theoretic concepts (including conflict-theoretic concepts, in the form of zero-sum game theory, brinkmanship, and cybernetic control of people) and applied them to social and geopolitical situations.
John Nash, famously, developed symptoms of paranoid schizophrenia after his work in game theory. A (negative) review of A Beautiful Mind describes the dysfunctionally competitive and secretive Princeton math department Nash found himself in:
After Nash left, RAND corporation went on to assist the US military in the Vietnam War; Daniel Ellsberg, who worked at RAND corporation, leaked the Pentagon Papers in 1971, which showed a large un-reported expansion in the scope of the war, and that the main objective of the war was containing China rather than securing a non-communist South Vietnam. Ellsberg much later published The Doomsday Machine, detailing US nuclear war plans, including the fact that approval processes for launching nukes were highly insecure (valuing increasing the probability of launching retaliatory strikes over minimizing the rate of accidental launches), the fact that the US's only nuclear war plan involved a nuclear genocide of China whether or not China had attacked the US, and the fact that the US air force deliberately misinformed President Kennedy about this plan in violation of the legal chain of command. At least some of the impetus for plans like this came from RAND corporation, due to among other things the mutually assured destruction doctrine, and John Von Neumann's advocacy of pre-emptively nuking Russia. Given that Ellsberg was the only major whistleblower, and delayed publishing critical information for decades, it is improbable that complicity with such genocidal plans was uncommon at RAND corporation, and certain that such complicity was common in the Air Force and other parts of the military apparatus.
It wouldn't be a stretch to suggest that Nash, through his work in game theory, came to notice more of the ways people around him (both at the Princeton math department and at the RAND Corporation) were acting against the mission of the organization in favor of egoic competition with each other and/or insane genocide. Such a realization, if understood and propagated without adequate psychological support, could easily cause symptoms of paranoid schizophrenia. I recently discussed Nash on Twitter:
MIRI seemed to have a substantially conflict-theoretic view of the broad situation, even if not the local situation. I brought up the possibility of convincing DeepMind people to care about AI alignment. MIRI leaders including Eliezer Yudkowsky and Nate Soares told me that this was overly naive, that DeepMind would not stop dangerous research even if good reasons for this could be given. Therefore (they said) it was reasonable to develop precursors to AGI in-house to compete with organizations such as DeepMind in terms of developing AGI first. So I was being told to consider people at other AI organizations to be intractably wrong, people who it makes more sense to compete with than to treat as participants in a discourse.
[EDIT: Nate clarifies that he was trying to say that, even if it were possible to convince people to care about alignment, it might take too long, and so this doesn't imply a conflict theory. I think the general point that time-to-converge-beliefs is relevant in a mistake theory is true, although in my recollection of the conversation Nate said it was intractable to convince people, not just that it would take a long time; also, writing arguments explicitly allows many people to read the same arguments, which makes scaling to more people easier.]
The difference between the beliefs of MIRI leadership and Michael Vassar was not exactly mistake theory versus conflict theory. Rather, MIRI's conflict theory made an unprincipled exception for the situation inside MIRI, exclusively modeling conflict between MIRI and other outside parties, while Michael Vassar's model did not make such exceptions. I was more interested in discussing Michael's conflict theory with him than discussing MIRI leadership's conflict theory with them, on the basis that it better reflected the situation I found myself in.
MIRI leadership was not offering me a less dark worldview than Michael Vassar was. Rather, this worldview was so dark that it asserted that many people would be destroying the world on fairly short timescales in a way intractable to reasoned discourse, such that everyone was likely to die in the next 20 years, and horrible AI torture scenarios might (with low probability) result depending on the details. By contrast, Michael Vassar thinks that it is common in institutions for people to play zero-sum games in a fractal manner, which makes it unlikely that they could coordinate well enough to cause such large harms. Michael has also encouraged me to try to reason with and understand the perspective of people who seem to be behaving destructively instead of simply assuming that the conflict is unresolvable.
And, given what I know now, I believe that applying a conflict theory to MIRI itself was significantly justified. Nate, just last month (due to myself talking to people on Twitter), admitted that he posted "political banalities" on the MIRI blog during the time I was there. I was concerned about the linked misleading statement in 2017 and told Nate Soares and others about it, although Nate Soares insisted that it was not a lie, because technically the word "excited" could indicate the magnitude of a feeling rather than the positiveness of it. While someone bullshitting on the public Internet (to talk up an organization that by Eliezer's account "trashed humanity's chances of survival") doesn't automatically imply they lie to their coworkers in-person, I did not and still don't know where Nate is drawing the line here.
Anna Salamon, in a comment on my post, discusses "corruption" throughout CFAR's history:
(It should go without saying that, even if suspicion was justified, that doesn't rule out improvement in the future; Anna and Nate's transparency about past behavior here is a step in the right direction.)
Does developing a conflict theory of my situation necessitate developing the exact trauma complex that I did? Of course not. But the circumstances that justify a conflict theory make trauma much more likely, and vice versa. Traumatized people are likely to quickly update towards believing their situation is adversarial ("getting triggered") when receiving modest evidence towards this, pattern-matching the new potentially-adversarial situation to the previous adversarial situation(s) they have encountered in order to generate defensive behavioral patterns.
I was confused and constrained after tasking people I most trusted with helping take care of me early in psychosis
The following events took place in September-October 2017, 3-4 months after I had left MIRI in June.
I had a psychedelic trip in Berkeley, during which I discussed the idea of "exiting" civilization, the use of spiritual cognitive modalities to improve embodiment, the sense in which "identities" are cover stories, and multi-perspectival metaphysics. I lost a night of sleep, decided to "bravely" visit a planned family gathering the next day despite my sleep loss (partially as a way to overcome neurotic focus on downsides), lost another night of sleep, came back to Berkeley the next day, and lost a third night of sleep. After losing three nights of sleep, I started perceiving hallucinations such as a mirage-like effect in the door of my house ("beckoning me", I thought). I walked around town and got lost, noticed my phone was almost out of battery, and called Jack Gallagher for assistance. He took me to his apartment; I rested in his room while being very concerned about my fate (I was worried that in some sense "I" or "my identity" was on a path towards death). I had a call with Bryce Hidysmith that alleviated some of my anxieties, and I excitedly talked with Ben Hoffman and Jack Gallagher as they walked me back to my house.
That night, I was concerned that my optimization might be "perverse" in some way, where in my intending to do something part of my brain would cause the opposite to happen. I attempted to focus my body and intentions so as to be able to take actions more predictably. I spent a number of hours lying down, perhaps experiencing hypnagogia, although I'm not sure whether or not I actually slept. That morning, I texted my friends that I had slept. Ben Hoffman came to my house in the morning and informed me that my housemate had informed Ben that I had "not slept" because he heard me walking around at night. (Technically, I could have slept during times he did not hear me walking around). Given my disorganized state, I could not think of a better response than "oops, I lied". I subsequently collapsed and writhed on the floor until he led me to my bed, which indicates that I had not slept well.
Thus began multiple days of me being very anxious about whether I could sleep in part because people around me would apply some degree of coercion to me until they thought I was "well" which required sleeping. Such anxiety made it harder to sleep. I spent large parts of the daytime in bed which was likely bad for getting to sleep compared with, for example, taking a walk.
Here are some notable events during that week before I entered the psych ward:
Zack Davis gave me a math test: could I prove eπi+1=0? I gave a geometric argument: "eix means spinning x radians clockwise about the origin in the complex plane starting from 1", and I drew a corresponding diagram. Zack said this didn't show I could do math, since I could have remembered it, and asked me to give an algebraic argument. I failed to give one (and I think I would have failed pre-psychosis as well). He told me that I should have used the Taylor series expansion of ex. I believe this exchange was used to convince other people taking care of me that I was unable to do math, which was unreasonable given the difficulty of the problem and the lack of calibration on an easier problem. This worsened communication in part by causing me to be more afraid that people would justify coercing me (and not trying to understand me) on the basis of my lack of reasoning ability. (Days later, I tested myself with programming "FizzBuzz" and was highly distressed to find that my program was malfunctioning and I couldn't successfully debug it, with my two eyes seeming to give me different pictures of the computer screen.)
I briefly talked with Michael Vassar (for less than an hour); he offered useful philosophical advice about basing my philosophy on the capacity to know instead of on the existence of fundamentally good or bad people, and made a medication suggestion (for my sleep issues) that turned out to intensify the psychosis in a way that he might have been able to predict had he thought more carefully, although I see that it was a reasonable off-the-cuff guess given the anti-anxiety properties of this medication.
I felt like I was being "contained" and "covered up", which included people not being interested in learning about where I was mentally. (Someone taking care of me confirms years later that, yes, I was being contained, and people were covering up the fact that there was a sick animal in the house). Ben Hoffman opened the door which let sunlight into the doorway. I took it as an invitation and stepped outside. The light was wonderful, giving me perhaps the most ecstatic experience I have had in my life, as I sensed light around my mind, and I felt relieved from being covered up. I expounded on the greatness of the sunlight, referencing Sarah's post on Ra. Ben Hoffman encouraged me to pay more attention to my body, at which point the light felt like it concentrated into a potentially-dangerous sharp vertical spike going through my body. (This may technically be or have some relation to a Kundalini awakening, though I haven't confirmed this; there was a moment around this time that I believe someone around me labeled as a "psychotic break".). I felt like I was performing some sort of light ritual navigating between revelation and concealment, and subsequently believed I had messed up the ritual terribly and became ashamed. Sometime around then I connected what I saw due to the light with the word "dasein" (from Heidegger), and shortly afterward connected "dasein" to the idea that zero-sum games are normal, such as in sports. I later connected the light to the idea that everyone else is the same person as me (and I heard my friends' voices in another room in a tone as if they were my own voice).
I was very anxious and peed on a couch at some point and, when asked why, replied that I was "trying to make things worse".
I was in my bed, ashamed and still, staring at the ceiling, afraid that I would do something bad. Sarah Constantin sat on my bed and tried to interact with me, including by touching my fingers. I felt very afraid of interacting with her because I thought I was steering in the wrong direction (doing bad things because they are bad) and might hurt Sarah or others. I felt something behind my eyes and tongue turn inward as I froze up more and more, sabotaging my own ability to influence the world, becoming catatonic (a new mind-altering medication that was suggested to me at the time, different from the one Michael suggested, might also have contributed to the catatonia). Sarah noticed that I was breathing highly abnormally and called the ER. While the ambulance took me there I felt like I could only steer in the wrong direction, and feared that if I continued I might become a worse person than Adolf Hitler. Sarah came with me in the ambulance and stayed with me in the hospital room; after I got IV benzos, I unfroze. The hospital subsequently sent me home.
One night I decided to open my window, jump out, and walk around town; I thought I was testing the hypothesis that things were very weird outside and the people in my house were separating me from the outside. I felt like I was bad and that perhaps I should walk towards water and drown, though this was not a plan I could have executed on. Ben Hoffman found me and walked me back home. Someone called my parents, who arrived the next day and took me to the ER (I was not asked if I wanted to be psychiatrically institutionalized); I was catatonic in the ER for about two days and was later moved to a psychiatric hospital.
While those who were taking care of me didn't act optimally, the situation was incredibly confusing for myself and them, and I believe they did better than most other Berkeley rationalists would have done, who would themselves have done better than most members of the American middle class would have done.
Are Michael Vassar and friends pro-psychosis gnostics?
Scott asserts:
I have only heard Michael Vassar use the word "jailbreak" when discussing Ziz, but he believes it's possible to use psychedelics to better see deception and enhance one's ability to use one's own mind independently, which I find to be true in my experience. This is a common belief among people who take psychedelics, and psychotherapeutic organizations including MAPS and Johns Hopkins, which have published conventional academic studies demonstrating that psychedelic treatment regimens widely reported to induce "ego death" have strong psychiatric benefits. Michael Vassar believes "tough conversations" that challenge people's defensive nonsense (some of which is identity-based) are necessary for psychological growth, in common with psychotherapists, and in common with some MIRI/CFAR people such as Anna Salamon.
I had tried psychedelics before talking significantly with Michael, in part due to a statement I heard from a friend (who wasn't a CFAR employee but who did some teaching at CFAR events) along the lines of "CFAR can't legally recommend that you try [a specific psychedelic], but..." (I don't remember what followed the "but"), and in part due to suggestions from other friends.
"Infinitely corrupt and conformist and traumatizing" is hyperbolic (infinite corruption would leave nothing to steal), though Michael Vassar and many of his friends believe large parts of normal society (in particular in the professional-managerial class) are quite corrupt and conformist and traumatizing. I mentioned in a comment on the post one reason why I am not sad that I worked at MIRI instead of Google:
I have talked with other people who have worked in corporate management, who have corroborated that corporate management traumatizes people into playing zero-sum games. If Michael and I are getting biased samples here and high-level management at companies like Google is actually a fine place to be in the usual case, then that indicates that MIRI is substantially worse than Google as a place to work. Iceman in the thread reports that his experience as a T-5 (apparently a "Senior" non-management rank) at Google "certainly traumatized" him, though this was less traumatizing than what he gathers from Zoe Curzi's or my reports, which may themselves be selected for being especially severe due to the fact that they are being written about. Moral Mazes, an ethnographic study of corporate managers written by sociology professor Robert Jackall, is also consistent with my impression.
Scott asserts that Michael Vassar treats borderline psychosis as an achievement:
A strong form of this is contradicted by Zack Davis's comment:
A weaker statement is true: Michael Vassar believes that mental states somewhat in the direction of psychosis, such as ones had by family members of clinical schizophrenics, are likely to be more intellectually productive over time. This is not an especially concerning or absurd belief. Scott Alexander himself cites research showing greater mental modeling and verbal intelligence in relatives of schizophrenics:
(He also mentions John Nash as a particularly interesting case of mathematical intelligence being associated with schizophrenic symptoms, in common with my own comparison of myself to John Nash earlier in this post.)
I myself prefer to be sub-clinically schizotypal (which online self-diagnosis indicates I am) to the alternative of being non-schizotypal, which I understand is not a preference shared by everyone. There is a disagreement between Michael Vassar and Scott Alexander about the tradeoffs involved, but they agree there are both substantial advantages and disadvantages to mental states somewhat in the direction of schizophrenia.
Is Vassar-induced psychosis a clinically significant phenomenon?
Scott Alexander draws a causal link between Michael Vassar and psychosis:
(to be clear: Michael Vassar and our mutual friends decided to place me in a psychiatric institution after I lost a week of sleep, which is at most a mild form of "discourag[ing] people from seeking treatment"; it is in many cases reasonable to try at-home treatment if it could prevent institutionalization.)
I have given an account in this post of the causality of my psychosis, in which Michael Vassar is relevant, and so are Eliezer Yudkowsky, Nate Soares, Anna Salamon, Sarah Constantin, Ben Hoffman, Zack Davis, Jack Gallagher, Bryce Hidysmith, Scott Alexander, Olivia Schaeffer, Alice Monday, Brian Tomasik, Venkatesh Rao, David Chapman, Carl Jung, M. Scott Peck, Martin Heidegger, Lao Tse, the Buddha, Jesus Christ, John Von Neumann, John Nash, and many others. Many of the contemporary people listed were/are mutual friends of myself and Michael Vassar, which is mostly explained by myself finding these people especially helpful and interesting to talk to (correlated with myself and them finding Michael Vassar helpful and interesting to talk to), and Michael Vassar connecting us with each other.
Could Michael Vassar have orchestrated all this? That would be incredibly unlikely, requiring him to scheme so well that he determines the behavior of many others while having very little direct contact with me at the time of psychosis. If he is Xanatos, directing the entire social scene I was part of through hidden stratagems, that would be incredibly unlikely on priors, and far out of line with how effective I have seen him to be at causing people to cooperate with his intentions.
Other people who have had some amount of interaction with Michael Vassar and who have been psychotic commented in the thread. Devi Borg commented that the main contributor to her psychosis was "very casual drug use that even Michael chided me for". Zack Davis commented that "Michael had nothing to do with causing" his psychosis.
Eric Bruylant commented that his thoughts related to Michael Vassar were "only one mid sized part of a much larger and weirder story...[his] psychosis was brought on by many factors, particularly extreme physical and mental stressors and exposure to various intense memes", that "Vassar was central to my delusions, at the time of my arrest I had a notebook in which I had scrawled 'Vassar is God' and 'Vassar is the Devil' many times"; he only mentioned sparse direct contact with Michael Vassar himself, mentioning a conversation in which "[Michael] said my 'pattern must be erased from the world' in response to me defending EA".
While on the surface Eric Bruylant seems to be most influenced by Michael Vassar out of any of the cases, the effect would have had to be indirect given his low amount of direct conversation with Michael, and he mentions an intermediary talking to both him and Michael. Anna Salamon's hyperbolic statement that Michael is "the devil" may be causally related to Eric's impressions of Michael especially given the scrawling of "Vassar is God" and "Vassar is the Devil". It would be very surprising, showing an extreme degree of mental prowess, for Michael Vassar to be able to cause a psychotic break two hops out in the social graph through his own agency; it is much more likely that the vast majority of relevant agency was due to other people.
I have heard of 2 cases of psychosis in former MIRI employees in 2017-2021 who weren't significantly talking with Michael or Ziz (I referenced one in my original post and have since then learned of another).
As I pointed out in a reply to Scott Alexander, if such strong mental powers are possible, that lends plausibility to the psychological models people at Leverage Research were acting on, in which people can spread harmful mental objects to each other. Scott's comment that I reply to admits that attributing such strong psychological powers to Michael Vassar is "very awkward" for liberalism.
Such "liberalism" is hard for me to interpret in light of Scott's commentary on my pre-psychosis speech:
[EDIT: I originally misinterpreted "it" in the last sentence as referring to "talk about demons and auras", not "psychosis", and the rest of this section is based on that incorrect assumption; Scott clarified that he meant the latter.]
I commented that this was effectively a restriction on my ability to speak freely, in contradiction with the liberal right to free speech. Given that a substantial fraction of the general public (e.g. New Age people and Christians, groups that overlap with psychiatrists) discuss "auras" and "demons", it is inappropriate to treat such discussion as cause for a "psychiatric emergency", a judgment substantially increasing the risk of involuntary institutionalization; that would be a case of a minority ideological community using the psychiatric system to enforce its local norms. If Scott were arguing that talk of "auras" and "demons" is a psychiatric emergency based on widely-accepted professional standards, he would need to name a specific DSM condition and argue that this talk constitutes symptoms of that condition.
In the context of MIRI, I was in a scientistic math
cult'high-enthusiasm ideological community', so seeing outside the ideology of thiscult'community' might naturally involve thinking about non-scientistic concepts; enforcing "talking about auras and demons is a psychiatric emergency" would, accordingly, be enforcingcult'local' ideological boundaries using state force vested in professional psychiatrists for the purpose of protecting the public.While Scott disclaims the threat of involuntary psychiatric institutionalization later in the thread, he did not accordingly update the original comment to clarify which statements he still endorses.
Scott has also attributed beliefs to me that I have never held or claimed to have held. I never asserted that demons are real. I do not think that it would have been helpful for people at MIRI to pretend that they thought demons were real. The nearest thing I can think of having said is that the hypothesis that "demons" were responsible for Eric Bruylant's psychosis (a hypothesis offered by Eric Bruylant himself) might correspond to some real mental process worth investigating, and my complaint is that I and everyone else were discouraged from openly investigating such things and forming explicit hypotheses about them. It is entirely reasonable to be concerned about things conceptually similar to "demon possession" when someone has just attacked a mental health worker shortly after claiming to be possessed by a demon; discouraging such talk prevents people in situations like the one I was in from protecting their mental health by modeling threats to it.
Likewise, if someone had tried to explain why they disagreed with the specific things I said about auras (which did not include an assertion that they were "real," only that they were not a noticeably more imprecise concept than "charisma"), that would have been a welcome and helpful response.
Scott Alexander has, at a Slate Star Codex meetup, said that Michael is a "witch" and/or does powerful "witchcraft". This is clearly of the same kind as speech about "auras" and "demons". (The Sequences post on Occam's Razor, relevantly, mentions "The lady down the street is a witch; she did it" as an example of a non-parsimonious explanation.)
I can't believe that a standard against woo-adjacent language is being applied symmetrically given this and given that some other central rationalists such as Anna Salamon and multiple other CFAR employees used woo-adjacent language more often than I ever did.
Conclusion
I hope reading this gives a better idea of the actual causal factors behind my psychosis. While Scott Alexander's comment contained some relevant information and prompted me to write this post with much more relevant information, the majority of his specific claims were false or irrelevant in context.
While much of what I've said about my workplace is negative (given that I am specifically focusing on what was stressing me out), there were, of course large benefits to my job: I was able to research very interesting philosophical topics with very smart and interesting people, while being paid substantially more than I could get in academia; I was learning a lot even while having confusing conflicts with my coworkers. I think my life has become more interesting as a result of having worked at MIRI, and I have strong reason to believe that working at MIRI was overall good for my career.
I will close by poetically expressing some of what I learned:
Postscript
Eliezer's comment in support of Scott's criticism was a reply to Aella saying he shared her (negative) sense about my previous post. If an account by Joshin is correct, we have textual evidence about this sense:
[EDIT: See Aella's response; she says she didn't say the line about undermining frames, and that use of the term "evil" has more context, and that the post overall was mostly wrong. To disambiguate her use of "evil", I'll quote the relevant part of her explanatory blog post below.]