For reasons that are obvious, as of late I've been thinking a lot about artificial intelligence and the possibility of doom. I think I've been able to come up with a couple of unorthodox and interesting reasons for why there is still hope. If I were to sum them up in some kind of post, it would probably go like this:

Most models fail due to unexpected reasons, partly because expected reasons are expected and then taken care of. Many people here have spent lots of years building a perfect model of how there is no or very little hope for humanity to survive and end up thinking that anything other than that is improbable, but in the real world, it is actually very rare that non-empirical, purely theoretical models are anywhere close to the actual thing that is happening (and those who happen to be right probably just got lucky). As an example of such a thing happening, we can take a look at whatever creation myth you like more as a purely theoretical model not based on any empirical evidence and see how far it is from anything resembling the actual thing. I am not trying to say that AI safety as a field is as far from the truth as creation myths are; please take that as an example of a common pattern in human thought processes and not as an insult to the AI safety people. In the real world, there are plenty of unknowns that even very smart people are incapable of predicting. If you want another example, you can look at the attempts at predicting the future by literally anyone. People tend to make predictions about the future by taking current trends and extending them into infinity, but the actual future is nowhere close to that. Trends are changing all the time; there are black swans everywhere, and there is not a single thing that doesn't change, apart from physically unchangeable ones. All that chaos is incomprehensible to the human mind, and I tend to think that even superintelligence would be incapable of accurately predicting the future since there are too many factors with probabilities far too uncertain to actually predict anything, no matter how smart you are. I am trying to say here: AGI, if possible, is very likely going to be nothing like we predicted, and there is a huge probability that the AGI that is going to emerge will not be murderous or (humanity-exterminating levels of) dangerous. One example of an unexpected reason for hope we can actually

...

Posts

Wikitag Contributions

Comments