Cross posted on Effective Altruism Forum https://forum.effectivealtruism.org/posts/XA8QSCL7wZ973i6vr/agi-safety-and-losing-electricity-industry-resilience-cost

Below is a paper about to be submitted. The focus is on interventions that could improve the long-term outcome given catastrophes that disrupt electricity/industry, such as solar storm, high-altitude electromagnetic pulse (HEMP), narrow AI computer virus, and extreme pandemic. Work on these interventions is even more neglected than interventions for feeding everyone if the sun is blocked. Cost-effectiveness is compared to a modified AGI safety cost-effectiveness model posted earlier on the EA forum. Two different cost-effectiveness estimates for losing industry interventions were developed: one by Denkenberger and a poll at EA Global San Francisco 2018, and the other by Anders Sandberg at Future of Humanity Institute. There is great uncertainty in both AGI safety and interventions for losing industry. However, the models have ~99% confidence that funding interventions for losing industry now is more cost effective than additional funding for AGI safety beyond ~$3 billion. This does not take into account model or theory uncertainty, so the confidence would likely decrease. However, in order to make AGI safety more cost effective, this required changing four variables in the Sandberg model to the 5th percentile on the pessimistic end simultaneously. For the other model, it required changing seven variables. Therefore, it is quite robust that a significant amount of money should be invested in losing industry interventions now. There is closer to 50%-88% confidence that spending the ~$40 million on interventions for losing industry is more cost effective than AGI safety. Overall, AGI safety is more important and more total money should be spent on it. The modeling concludes that additional funding would be justified on both causes even for the present generation.

Long Term Cost-Effectiveness of Interventions for Loss of Electricity/Industry Compared to Artificial General Intelligence Safety

David Denkenberger 1,2, Anders Sandberg 3, Ross Tieman *1, and Joshua M. Pearce 4,5

1. Alliance to Feed the Earth in Disasters (ALLFED), Fairbanks, AK 99775, USA

2. University of Alaska Fairbanks, Fairbanks, AK 99775, USA

3. Future of Humanity Institute, University of Oxford, Oxford, UK

4. Department of Material Science ...

Posts

Wikitag Contributions

Comments