Copying over a comment from the EA forum (and my response) because it speaks to something that was in some earlier drafts, that I expect to come up, and that is worth just going ahead and addressing imo.

IMO it would help to see a concrete list of MIRI's outputs and budget for the last several years. My understanding is that MIRI has intentionally withheld most of its work from the public eye for fear of infohazards, which might be reasonable for soliciting funding from large private donors but seems like a poor strategy for raising substantial public money, both prudentially and epistemically.

If there are particular projects you think are too dangerous to describe, it would still help to give a sense of what the others were, a cost breakdown for those, anything you can say about the more dangerous ones (e.g. number of work hours that went into them, what class of project they were, whether they're still live, any downstream effect you can point to, and so on).

My response:

(Speaking in my capacity as someone who currently works for MIRI)

I think the degree to which we withheld work from the public for fear of accelerating progress toward ASI might be a little overrepresented in the above. We adopted a stance of closed-by-default research years ago for that reason, but that's not why e.g. we don't publish concrete and exhaustive lists of outputs and budget.

We do publish some lists of some outputs, and we do publish some degree of budgetary breakdowns, in some years.

But mainly, we think of ourselves as asking for money from only one of the two kinds of donors. MIRI feels that it's pretty important to maintain strategic and tactical flexibility, to be able to do a bunch of different weird things that we think each have a small chance of working out without exhaustive justification (or post-hoc litigation) of each one, and to avoid the trap of focusing only on clearly legible short chains of this—>that (as opposed to trying both legible and less-legible things).

(A colleague of mine once joked that "wages are for people who can demonstrate the value of their labor within a single hour; I can't do that, which is why I'm on a salary." A similar principle applies here.)

In the past, funding MIRI led to outputs like our alignment research publications and the 2020/2021 research push (that didn't pan out). In the more recent past, funding MIRI has led to outputs like the work of our technical governance team, and the book (and its associated launch campaign and various public impacts).

That's enough for some donors—"If I fund these people, my money will go into various experiments that are all aimed at ameliorating existential risk from ASI, with a lean toward the sorts of things that no one else is trying, which means high variance and lots of stuff that doesn't pan out and the occasional home run."

Other donors are looking to more clearly purchase a specific known product, and those donors should rightly send fewer of their dollars to MIRI, because MIRI has never been and does not intend to ever be quite so clear and concrete and locked-in.

(One might ask "okay, well, why post on the EA forum, which is overwhelmingly populated by the other kind of donor, who wants to track the measurable effectiveness of their dollars?" and the answer is "mostly for the small number who are interested in MIRI-like efforts anyway, and also for historical reasons since the EA and rationality and AI safety communities share so much history." Definitely we do not feel entitled to anyone's dollars, and the hesitations of any donor who doesn't want to send their money toward MIRI-like efforts are valid.)

Most of the seven Extended Discussions under chapter 4 of Nate and Eliezer's book's supplementals are basically an expansion of this thesis (which I also agree with and think is true).

I have not yet been limiting screens, except insofar as, like, hanging out with Cadence from 6pm to sleep usually means no screens.  But it seems like I'm obviously going to end up there, and am sort of just relinquishing my reluctance/figuring out a new flow that still allows me to do the things I need to get done each day.

Currently, I get some Third State in the mornings, actually, when Cadence has woken up and is puttering around but hasn't come and fully woken me up yet.

This is great.

One constraint that Logan and I have both gotten a lot of benefit out of reinstating is one around artificial light; Logan is stricter than I but we're both much more subject to natural darkness than the average American and afaict it's doing a bunch of positive things.  Better sleep, more connection with the "third state of consciousness" that is between waking and sleeping, we essentially never struggle to settle Cadence down for bed, etc.

In past eras of my life, I had "don't drive less than two miles" as a soft rule and would instead walk any sub-two-mile distance, and I think this too was useful.

(This answer feels fragmented; sorry; am still on postsurgery medication and brain is at 40% capacity.)

Possibly of interest, although it sounds like you maybe already have all of these puzzle pieces.

Example: suppose someone says "I can imagine an atomic copy of ourselves which isn't conscious, therefore consciousness is non-physical." and I say "No, I can't imagine that."

Possibly of interest 

Or the followup by Logan Strohl, even more directly on this

Just noting for the audience that the edits which Anna references in her reply to CronoDAS, as if they had substantively changed the meaning of my original comment, were to add:

• The phrase "directly observed"
• The parenthetical about having good epistemic hygiene with regards to people's protestations to the contrary
• The bit about agendas often not being made explicit

It did not originally specify undisclosed conflicts of interest in any way that the new version doesn't.  Both versions contained the same core (true) claim: that multiple of the staff members common to both CFAR!2017 and CFAR!2025 often had various (i.e. not only the AI stuff) agendas which would bump participant best interests to second, third, or even lower on the priority ladder.

I've also added, just now, a clarifying edit to a higher comment: "Some of these staff members are completely blind to some centrally important axes of care."  This seemed important to add, given that Anna is below making claims of having seen, modeled, and addressed the problems (a refrain I have heard from her, directly, in multiple epochs, and taken damage from naively trusting more than once).  More (abstract, philosophical) detail on my views about this sort of dynamic here.

I claim to be as-aware and as-sensitive-to of all of these considerations as you are.  I think I am being as specific as possible, given constraints (many of which I wish were not there; I have a preference for speaking more clearly than I can here).

I know of one parent that puts three dollars aside each time they violate the bodily sovereignty of their infant - taking something out of their mouth, or restricting where they can go

It's me, by the way.  Happy to identify myself.

(I have more agreement than disagreement with the authors on many points, here.)

I'll note that both this and my top-level comment have a lot of agree-disagree votes, and that it would be wise for people looking in from the outside to ponder what it means for e.g. the top-level comment to have 18 people voting and to end up at -2.

(It might be tempting to sum it up as "ah, Duncan claimed that there's something to be wary of here, and the hive-mind ultimately ended up in disagreement" but I think it's more like "Duncan claimed there's something to be wary of here, and close to half of the people agreed (but were drowned out by the somewhat more than half who disagreed)."  Which is precisely what you would expect if there were some system or process that was consistently harmful to certain people, but not all or even most—a lot of people who passed through unscathed would be like "what do you mean? I was well-cared-for!" and might not pause to wonder about whether they were a black raven and what evidence their experience provides about claims of the existence of white ravens.)