Just to clarify:

I am personally uncertain how hard stopping the race is. I have spent some time and money myself trying to promote IABIED, and I have also been trying to do direct alignment research, and when doing so I more often than not think explicitly in scenarios where the AI race does not stop.

Am I in group C? Am I a fake member of C? 

I'd personally say I'd probably endorse C for someone who funds research/activism, and have personally basically acted on it.

I.e. I'd say it's reasonable to say "stop the race ASAP", and in another context say "the race might not be stopped, what projects would still maybe increase odds of survival/success conditional on a race?"

"One thing that makes agent foundations different from science is that we're trying to understand a phenomenon that hasn't occurred yet (but which we have extremely good reasons for believing will occur)."

TBF, physicists are also doing science when they're trying to figure out what will happen to the universe 2 trillion years from now, when various things are predicted to happen that presumably haven't happened yet.

I'm not sure this property therefore makes AF "not science".

Is there a writeup regarding your change of view?

Wow. It seems to me that the fact that you didn't even imagine what John had in mind somehow implies really a lot about how your model differs from his (and mine). 

These problems still exist in some versions of Shut It Down too, to be clear (if you're trying to also ban algorithmic research – a lot of versions of that seem like they leave room to argue about whether agent foundations or interpretability count).

This is the main reason for why, despite being strongly in favor of Shut It Down, I'm still very nervous about it.

Fair point, I've downvoted my comment. Apologies.

(although in my defense, you didn't make that argument in the comment I responded to, and also, liron assigning 50% doesn't mean he actually disagrees with Yudkowsky. It might be he's just not sure, but doesn't have any counterarguments per se).

"Perhaps a bit too consensual."

Yeah, horrible!! They should have pretended to disagree with each other in order to balance out all the agreement they have. They must be biased!!

But the problem is that we likely don't have time to flesh out all the details or do all the relevant experiments before it might be too late, and governments need to understand that based on arguments that therefore cannot possibly rely on everything being fleshed out.

Of course I want people to gather as much important empirical evidence and concrete detailed theory as possible asap. 

Also, the pre-everything-worked-out-in-detail arguments also need to inform which experiments are done, and so that is why people who have actually listened to those pre-detailed arguments end up on average doing much more relevant empirical work IMO. 

I don't think it's *contradicting* it but I vaguely thought maybe it's in tension with:

"Big changes within companies Government AI x-risk policy are typically bottlenecked much more by coalitional politics than knowledge of technical details.

Because lack of knowledge of technical details by A ends up getting B to reject and oppose A.

Mostly I wasn't trying to push against you though, and more trying to download part of your model on how important you think this is, out of curiosity, given your experience at OA.