It seems to be a widely held belief around here that unfriendly artificial general intelligence is dangerous, and that (provably) friendly artificial general intelligence is the soundest counter to it.
But I'd like to see some analysis of alternatives. Here are some possible technical developments. Would any of these defuse the threat? How much would they help?
- A tight lower bound on the complexity of SAT instances. Suppose that P != NP, and that researchers develop an algorithm for solving instances of Boolean satisfiability that is optimal in terms of asymptotic complexity, but far from efficient in practice.
- The opposite: a practical, say, quartic-time SAT algorithm, again with a proof that the algorithm is optimal.
- High-quality automated theorem proving technology, that's not self-modifying except in very narrow ways.
- Other special-purpose 'AI', such as high-quality natural-language processing algorithms that aren't self-modifying or self-aware. For example, suppose we were able to do language-to-language translation as well as bilingual but not-very-smart humans.
- Robust tools for proving security properties of complex programs. "This program can only produce output in the following format or with the following properties and cannot disable or tamper with the reference monitor or operating systems."
Are there other advances in computer science that might show up within the next twenty years, that would make friendly-AI much less interesting?
Would anything on this list be dangerous? Obviously, efficient algorithms for NP-complete problems would be very disruptive. Nearly all of modern cryptography would become irrelevant, for instance.