p(doom)>0. The cost of speech is minimal. So it is good that people are talking about the risk. It is ok that we don't have a solution to the alignment problem (well, not ok, but it is ok to admit we are working on it), but we do need at least provisional solutions to keeping humanity safe. So far, the main approach suggested by Yudkowsky and Soares, and endorsed by that recent petition, is to advocate for and ultimately pass legislation, both within nations and via the international order. In a world where this effort is successful, it is likely to slow down AI development, but not likely to do so in... (read 3514 more words →)