Three Missing Cakes, or One Turbulent Critic?
23habryka
8Benquo
15Raemon
10Benquo
6Vaniver
12Benquo
9Vaniver
8Benquo
2Vaniver
4Benquo
3RobertM
8Ben Pace
3Benquo
8habryka
2Benquo
2habryka
7Benquo
4RobertM
4lemonhope
3[anonymous]
8Ben Pace
New Comment
Anna reports that Habryka claims that the slapfight on "Zetetic Explanation" is a good chunk of why you don't use the website anymore. Is that true?
People, come on, what kind of multi-step game of telephone is going on here. The thing that I said to Anna, in a spoken and largely assumed to be private conversation, which was 4 hours long with a general vibe of her trying to understand my models of LessWrong moderation (and vice-versa) without much of an advocacy component, was, as far as I remember, something vaguely like "I know that Benquo got in a bunch of fights with Said, and my guess is Said played a non-trivial role in him leaving, and clearly if I had to choose between Said and Benquo on the site, I would choose Benquo".
I think I still stand behind that![1] It was a pretty big deal at the time, and that conflict together with the Duncan conflict seemed like roughly the thing that caused you to leave.
On the rest: I agree with 1) and 3) of your concerns, and have commented at least on the latter extensively. I don't... really know what this has to do with banning people who are otherwise causing problems on the site since though, I don't really think those are things that could be somehow solved with banning (like, sure, it's plausible to me they invalidate a bunch of the premises of the community, and are important to address, but even if your government is engaging in unjustified wars, that doesn't mean you don't want normal everyday police doing normal police things).
Unrelatedly:
TurnTrout is not a co-author of the SolidGoldMagikarp post. Maybe he collaborated with it, but at least it doesn't say anywhere that I can easily find that he was involved with writing it.
- ^
But even if I didn't, I didn't make any statement with great confidence and I would summarize similar epistemic states the same way in the future again.
Thanks for closing the loop on the game of telephone. Oops, not sure how I misremembered about TurnTrout. Will delete that bit.
1I mean more like MIRI apologists who didn’t notice that the Death with Dignity post really ought to be a halt, melt, catch fire moment.
I wasn't sure what you meant here, where two guesses are "the models/appeal in Death with Dignity are basically accurate, but, should prompt a deeper 'what went wrong with LW or MIRI's collective past thinking and decisionmaking?, '" and "the models/appeals in Death with Dignity are suspicious or wrong, and we should be halt-melting-catching-fire about the fact that Eliezer is saying them?"
The former. There should have been a lot of open-minded postmortem discussion, not just a pivot.
3Hmm. I HMCFed after that, I think, but I don't remember why I didn't talk much about it publicly. (Also I think there was a CFAR postmortem that I don't recall getting written up and discussed online, tho there was lots of in-person discussion.)
1I remember being told by an attendee that I hadn’t been invited to a CFAR postmortem weekend because I’m “not a STEM person.” Since I did statistical programming professionally for five years and have an MS in Mathematics and Statistics from an elite-ish university, I can only interpret that as meaning I’m unwilling to use technical math to dissociate from substantive humanities problems.
I have to rate all the time spent that didn’t result in improvements visible from the outside as nothing but costs paid to sustain internal narcissistic supply; I can’t credit it as an attempt to solve or even discuss a problem unless I receive further evidence. The uniformly positive things I’ve heard about “Don’t Create the Torment Nexus II: If Anyone Builds It, Everyone Dies” implies not much in the way of new perspective or even consensus that one is needed.
I can't comment on why you weren't invited [to the CFAR postmortem], because I was not involved with the decision-making for who would be invited; I just showed up to the event. Naively, I would've guessed it was because you didn't work at CFAR (unless you did and I missed it?); I think only one attendee wasn't in that category, for a broad definition of 'work at'.
I have to rate all the time spent that didn’t result in improvements visible from the outside as nothing but costs paid to sustain internal narcissistic supply
This seems fair to me.
The uniformly positive things I’ve heard about “Don’t Create the Torment Nexus II: If Anyone Builds It, Everyone Dies” implies not much in the way of new perspective or even consensus that one is needed.
I think the main difference between MIRI pre-2022 and post-2022 is that pre-2022 had much more willingness to play along with AI companies and EAs, and post-2022 is much more willing to be openly critical.
There are other differences, and also I think we might be focusing on totally different parts of MIRI. Would you care to say more about where you think there needs to be new perspective?
If the transition from less to more disagreeableness doesn't come along with an investigation of why agreeableness seemed like a plausible strategy and what was learned, then we're still stuck trying to treat an adversary as an environment.
I think I agree with your statement; I assume that this happened, though? Or, at least, in a mirror of the 'improvements visible from the outside' comment earlier, the question is whether MIRI is now operating in a way that leads to successfully opposing their adversaries, rather than whether they've exposed their reasoning about this to the public.
Naively, I would've guessed it was because you didn't work at CFAR (unless you did and I missed it?)
The attendee who told me about it never worked at CFAR, and neither did a couple other people I knew who went. Also I did guest-instruct at a CFAR workshop once.
FYI I am generally good at tracking inside baseball but I understand neither what specific failures[1] you would have wanted to see discussed in an open postmortem nor what things you'd consider to be "improvements" (and why the changes since 2022/04/01 don't qualify).
- ^
I'm sure there were many, but I have no idea what you consider to have been failures, and it seems like you must have an opinion because otherwise you wouldn't be confident that the changes over the last three years don't qualify as improvements.
Not sure what Benquo would say, but I think a natural question when a community of people fails at its goal after ~10 years, is to ask why it failed and what could've been done differently. It's a good opportunity to learn, and it's a good incentive to expect to do it so that you make sensible choices during the initial period that you're doing the work (to expect to have to justify them if you fail).
I think that the CFAR one is more natural, because it seems to me that MIRI set itself a great scientific challenge with an externally imposed deadline, whereas CFAR did not have the external deadline on developing an art of rationality (which is also a very difficult problem). So it's more naturally a place where the locus of control was within yourself, and a postmortem will be able to be accurate.
(I’m interested in this topic because I have myself considered trying to put together a public retro on the past ten years of safe-the-world efforts from this scene.)
The main effects of the sort of “AI Safety/Alignment” movement Eliezer was crucial in popularizing have been OpenAI, which Eliezer says was catastrophic, and funding for “AI Safety/Alignment” professionals, whom Eliezer believes to predominantly be dishonest grifters. This doesn't seem at all like what he or his sincere supporters thought they were trying to do.
I’ve written extensively on this sort of perverse optimization, but I don’t see either serious public engagement with my ideas here, or a serious alternative agenda.
For instance, Ben Pace is giving me approving vibes but it didn’t occur to him to respond to your message by talking about the obvious well-publicized catastrophic failures I mentioned in the OP. And it seems like you forgot about them too by the time you wrote your comment.
I am sympathetic to your takes here, but I am not that sympathetic to statements like this:
but I don’t see either serious public engagement with my ideas here, or a serious alternative agenda.
As it happens I also happen to have written many tens-of-thousands of words about this in many comments across LW and the EA Forum. I also haven't seen you engage with those things! (and my guess is the way you are phrasing it suggests you are not aware of them)
Like, man, I do feel like I resonate with the things that you are saying, but it just feels particularly weird to have you show up and complain that no one has engaged with your content on this, while having that exact relationship to approximately the people you are talking to. I, the head admin of LessWrong, have actually spent on the order of many hundreds of hours, maybe 1000+ hours on doing postmortem-ish things in the space, or at least calling for them. I don't know whether you think what I did/do makes any sense, but I think there is a real attempt of the kind of thing you are hoping for (to be clear, mostly ending with a kind of disappointment and resulting distancing from much of the associated community's, but it's not like you can claim a better track record here).
And in contrast to your relationship with my content, I have read your content and have engaged with it a good amount. You can read through my EA Forum comments and LW comments on the topic if you want to get a sense of how I think about these things.
I’m aware that you’ve complained about these problems, but I’m specifically calling for the development and evaluation of explanatory models, which is a different activity. If you’ve done much of that in your public writing I missed it - anything you’d like to point me to?
I have tried to do that, though it’s definitely more dispersed.
Most of it is still in comments and so a bit hard to extract, but one post I did write about this was My tentative best guess on how EAs and Rationalists sometimes turn crazy.
That does seem like it's overtly concerned with developing an explanation, but it seems concerned with deviance rather than corruption, so it's on a different topic than the ones I complain about in the OP. I was aware of that one already, as I replied with a comment at the time.
And it seems like you forgot about them too by the time you wrote your comment.
It was not clear from your comment which particular catastrophic failures you meant (and in fact it's still not clear to me which things from your post you consider to be in that particular class of "catastrophic failures", which of them you attribute at least partial responsibility for to MIRI/CFAR, by what mechanisms/causal pathways, etc).
ETA: "OpenAI existing at all" is an obvious one, granted. I do not think EY considers SBF to be his responsibility (reasonable, given SBF's intellectual inheritance from the parts of EA that were least downstream of EY's thoughts). You don't mention other grifters in your post.
1"Death with dignity" was clearly intended to trigger the audience to HMCF right? He was doing exactly what you are asking for
"Trigger the audience into figuring out what went wrong with MIRI's collective past thinking and decision-making" would be a strange purpose from a post written by the founder of MIRI, its key decision-maker, and a long-time proponent of secrecy in how the organization should relate to outsiders (or even how members inside the organization should relate to other members of MIRI).
Not disagreeing with your point, just want to add the datapoint that for me it did lead me to something like "giving up faith in" MIRI. I no longer believed that they were working on a plan for getting the problem solved, and so I resigned myself to the world where I had to take responsibility for the problem getting solved.