There is a common argument that AI development is dangerous that goes something like:
- The “goal” of evolution is to make animals which replicate their genes as much as possible;
- humans do not want to replicate their genes as much as possible;
- we have some goal which we want AIs to accomplish, and we develop them in a similar way to the way evolution developed humans;
- therefore, they will not share this goal, just as humans do not share the goal of evolution.
This argument sucks. It has serious, fundamental, and, to my thinking, irreparable flaws. This is not to say that its conclusions are incorrect; to some extent, I agree with the point which is typically... (read 2308 more words →)
Good point WRT that first line -- I edited it to something more clunky but I think more accurate. Hopefully the intended meaning came across anyway.
WRT the second point -- I agree that this is the weakest/most speculative argument in the post, although I still think it's worth considering. Evolution obviously "had the ability" to make us much more baby-obsessed, or have a higher sex drive, and yet we do not. This indicates that there are tradeoffs to be made; a human with a higher reproductive drive is less fit in other ways. One of those ways is plausibly that a human with a lower reproductive drive gets more "other stuff" done--like... (read more)