AI is a poorly understood, speculative field, so people with different intuitions come to different conclusions. If your intuition is that AI is inherently dangerous, it makes sense to seriously investigate the AI safety questions, whereas if you don't, you would focus on the potential benefits and want to accelerate.
I remember when first reading about paperclip maximizer AIs I thought it was stupid, as in anything superintelligent would be capable of understanding instructions. I think this intuition hints at the stronger reason to be more optimistic abou...
I agree, but from a different angle.
There are several ways in which communicating probabilities offers useful information, and I don't see p(doom) in any of them.
For one, there are true probabilities, such as the probability of a radioactive atom decaying by it's half life. This satisfies the frequentist interpretation, where one could re-run the radioactive decay experiment infinitely and observe the true probabilities over time. This does not apply to p(doom); it is unclear how quantum uncertainty could lead to noticeably different outcomes were we to re... (read more)