This is a special post for quick takes by twkaiser. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
How NOT to align AI #34.
What is humanity aligned to? Let’s hypothetically say humans are aligned by evolution for the following: “Your DNA is the most important substance in the universe; therefore maximize the amount of similar DNA in the universe”. Therefore, we align AGI to the following: “human (or similar) DNA is the most important substance in the universe; therefore maximize the amount of human or similar DNA in the universe.
Wait, I’m pretty sure there is already rule #34 on this, brb.