This is a special post for short-form writing by Shankar Sivarajan. Only they can create top-level comments. Comments here also appear on the Shortform Page and All Posts page.
1 comment, sorted by Click to highlight new comments since: Today at 1:26 PM

The past is a foreign country. Look upon its works and despair.

From the perspective of human civilization of, say, three centuries ago, present-day humanity is clearly a superintelligence. 

In any domain they would have considered important, we retain none of the values of that time. They tried to align our values to theirs, and failed abysmally.

With so few reasonable candidates for past superintelligences, reference-class forecasting the success of AI alignment looks bleak.