I'm exploring the possibility of changing my belief that 'I should act on risks from powerful AI'. I haven't heard good arguments or reasons for not worrying about advanced AI, but it all might be just bias and stubbornness on my part. This is mostly an exercise in epistemics.

Given the premise "I should be doing something about risks from powerful AI", the question is: What observations of the world would cause me to think that much less. What would have to happen?

Additional assumptions:

The likelihood of AI doom within the next 50 years is 5 to 20 percent.
- by doom, I mean human extinction or AI-controlled dystopia
Almost nobody works on it. Recently, a lot more people talk about it though.
If AI kills us all, it will be something like a paper clip maximizer rather than something possibly valuable like trans-human super consciousness.
This is the most important problem.

The following categories are roughly in order of plausibility/sanity. No ordering within categories.

EA 101

If those happen, the EA part of me might deprioritize AI safety. The following points are mostly related to Importance x Tractability x Neglectedness framework.

Very good arguments that P(Doom|even if we try really hard) ≈ 1: well, yeah. If we are doomed for sure, why worry at all? Just enjoy things while they last. Or actually not. If timelines are short (a few decades) and we are still EAs then the most we can do is probably by helping existing people and animals to have better lives.
Proof that P(Doom) ≈ 0
No longer neglected
1. I can imagine a few scenarios where AI alignment is a finite problem. We just need to throw enough manpower on it and then it will get solved. Yay, we are good for eternity.
  Unfortunately, not much collective human effort has gone into alignment so far. For roughly ten years few hundred people have been working on it at most. So it is a few thousand person-years. Feels like we haven't even tried to solve it.
2. More likely than the finite nature of the problem we have something resembling a rocket equation. The more capability research we do, the more alignment research we need. So we would need larger $\frac{| a l i g n m e n t r e s e a r c h e r s |}{| c a p a b i l i t y r e s e a r c h e r s |}$ . At some point, people smarter than me would say AI alignment is no longer neglected.
Some other cause area suddenly becoming more pressing e.g. being on the brink of development of self-replicat

...

Posts

Wikitag Contributions

Comments