Feb 9, 2018
I'm not sure if this is insightful enough to share here, but I'll try anyway.
A fair amount of wondering has been done about how FAI could figure out what humans actually want. A school of thought says persuasively that what we say we want is not what we actually want, so what we really want has to be extrapolated.
If we take the end-game promises of popular religions at face value, it occurs to me that Buddhism promises something between non-existence and wireheading (nirvana - "to blow out"), while Christianity promises wireheading (eternal bliss - heaven). I am not familiar enough with other religions to make statements about them.
In my anecdotal experience, it seems to me rationalists are quick to dismiss wireheading and non-existence as desirable possibilities. We experience this grasping desire to live, create, discover, and experience. We're not sure to what end, but we feel this indescribable zest, and we're convinced it's going to be great.
Look at people getting mental orgasms from Elon Musk launching a car into space. Whatever problems you have, space exploration is not going to solve them, and if your life is in harmony, you don't need space exploration. And yet there's this palpable zest about an unspoken implication... That perhaps an age of discovery is dawning, an age of adventure, an age of transcending our present problems and tackling larger ones. An age of being awesome.
There's a type of person that feels this zest, and this type is not a majority. The median person on Earth is confused by the world. They believe in things like Jesus Christ, and they press on in hope that adhering to divine guidance while they attempt to survive the trials and tribulations of life will be rewarded with not having to do this again. To such a person, the sight of two metal meteors descending from the sky with loud sonic booms, igniting engines and landing in synchrony does not necessarily inspire awe or enthusiasm as much as confusion and terror.
We, the few, are the seekers of something we cannot describe, and most of us find it hard to identify with the mindset of the median individual. The median individual does not want the excitement we seek, they just want an end... a release.