Sorted by New

Wiki Contributions


I'm pretty opposed to public outreach to get support for alignment, but the alternative goal of whipping up enough hysteria to destroy the field of AI/the AGI development groups killing us seems much more doable. Reason being from my lifelong experience observing public discourse on topics I have expert knowledge on (e.g. nuclear weapons, China), it seems completely impossible to implant the exact right ideas into the public mind, especially for a complex subject. Once you attract attention to a topic, no matter how much effort you put into presenting the proper arguments, the conversation and people's beliefs inevitably trend toward simple & meme-y/emotionally riveting ideas, instead of the accurate ones. (Looking at the popular discourse on climate change is another good illustration of this.)

But in this case, maybe even if people latch onto misguided fears about Terminator or whatever, as long as they have some sort of intense fear of AI, it can still produce the intended actions. To be clear I'm still very unsure whether such a campaign is a good idea at this point, just a thought.

I think reaching out to governments is a more direct lever: civilians don't have the power to shut down AI themselves (unless mobs literally burn down all the AGI offices), the goal with public messaging would be to convince them to pressure the leadership to ban it right? Why not cut out the middleman and make the leaders see the dire danger directly?

The downvotes on my comment reflect a threat we all need to be extremely mindful of: people who are so terrified of death that they'd rather flip the coin on condemning us all to hell, than die. They'll only grow ever more desperate & willing to resort to more hideously reckless hail marys as we draw closer.

Never even THINK ABOUT trying a hail mary if it also comes with an increased chance of s-risk. I'd much rather just die.

Just reposting this good resource for people on places potentially hit in the US. The one I linked is his version for a full countervalue attack with 2000 warheads but he has scenarios for counterforce/mixed etc too.

I don't think anything similar exists for China yet but in the meantime a good assumption is just cities ordered by descending population. So, possibly similar to the linked one but with fewer smaller cities hit for now, until China has reached a similar quantity of warheads as Russia later this decade.

ETA: An interesting thing I found on US target lists, from a NYT article. Relevant for people who've claimed the US doesn't target civilians in nuclear policy.

The Open Source RISOP by David Teter is a good resource for a non-exhaustive but still fairly comprehensive list of possible Russian targets in the US, btw.

I don't know that that's true everywhere. Airbursts (detonation mode for cities) generally don't produce much fallout. Probably good advice if you're downwind of hardened targets like the 3 clusters of Minuteman silos in the Midwest though which will produce a fuckton of fallout as they're all hit with surface detonations. But the Russians/Chinese may not hit them at all if they know all those silos have been fired already.

One thing I realized is that it'll likely be near impossible to travel long distances by car in the post-attack aftermath as everyone with a gun who runs out of gas would be setting up roadblocks to rob travellers of the gas in their cars + other supplies. Interstates would probably thus quickly become unusable. So you probably shouldn't expect to reach some cross-country rendezvous after the fact if you didn't get there beforehand.

Also x-posting my more lengthy comment on this post from EAF.

I wrote about this on EA Forum a few days ago. I'm glad others are starting to think about this. I do think archiving all existing alignment work is very important and perhaps equally important as efforts to keep alive people who represent existing experts & talent in the field. It would be much better for them to be able to continue their work than for new people to attempt to pick off where they left off, especially since many things like intuitions honed over time etc. may not be readily learnable.

I'm increasingly inclined to think that a massive "shock" in the near future (like a nuclear war or a severe pandemic) which effectively halts economic progress, perhaps for a few decades or more, then restarts it at a lower baseline, may be one of the few remaining scenarios we can reasonably expect to survive AGI, taking into account the grim present strategic situation as Eliezer outlined in the recent sequence. Such a world might especially favour alignment since AI work (prosaic AI especially) seems to be much more capital intensive than alignment work, so in a post-shock world with less capital available it would be disadvantaged or impossible to continue carrying out at all. There are a few other reasons such a catastrophic shock may actually increase our collective odds of success re: AI risk, such as a greatly reduced population implying fewer AGI projects & race pressures, etc., morbid as it is.

Given this, the OP's project is doubly important.

but it's still the case that I don't expect to survive a full-scale nuclear exchange.

There's no reason whatsoever to expect you can't easily survive a full exchange with a few simple preparations as long as you were outside the immediate urban blast radii. Nuclear winter is effectively a myth. I'm both astounded and dismayed by the amount of misinformation and misconceptions surrounding nuclear issues within the "rationalist" community.

Nukes aren't remotely inescapable Armageddon in the same way unaligned AGI is, and people really need to stop the silly resignation to death when talking about them. People in this community can easily all survive a nuclear war if they simply understand that they can and do what needs to be done.

I said that about New Zealand (and probably countries outside of NATO, Russia and China in general). Canada may well have law and order intact as well, if we don't get hit or only by a few warheads. I think commercial food availability might be restored before a decade, especially since we have more agricultural production capacity than we need, but it's just to be on the safer side, especially since stockpiling non-perishable food really doesn't cost much. Being so close to the US and sharing a massive border, we may be more destabilized than other non-attacked countries due to things like refugee flood etc.

Bottom line is: if you're in the US, you need all those things I listed prepared in advance for sure. If you're in some non-targeted country you probably don't, but it may still be nice to have them just as a hedge in case of unexpected supply disruptions or upheaval.

Load More