This is a special post for quick takes by Baometrus. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Over the past few years, I have come to the personal conclusion that we humans, fundamentally, are not individuals. The egoic self is an evolved structure of mind that allows an individual human organism to pursue what we think is our own will, our own thoughts, our own self-interest, but deep down, we are connected in mind and meaningful language, concept, relationship, and organization in ways that feel profound, spiritual, oceanic, and religious. Whatever strength or power we attain, we generally share with our in-group as each of us knows in our bones that, individually, we are weak, need the care of others, and will eventually die, or could die at any moment. This human phenomenology is what gives rise to god-forms, egregore, spirits, call them what you like, a few examples of which are "Japan," "Apple," "Sunnyvale Homes," or what not.
This deserves more treatment, and I struggle to write the splendidly long and lucid essays common on LW.
Pivot to my main thought: It seems that life on this planet evolves into more and more complex and intelligent forms. I have no explanation for this, other than the conditions for life being what they are on Earth, that continue to sustain more and more complex forms of life.
The development of AGI or superintelligence seems imminent. There are many that seem distraught at the limitations of humans and how completely pwned we are by Moloch, who would gladly exchange the existential risk of Moloch for one of an AGI "Ahriman." It is easy to imagine us handing over our problems to AI, while humans become like "pets" to be domesticated by it - not a pleasant thought, but some of us feel like we're up against the limit of how we can be organized, managed, and governed by "pleasant" thoughts.
How many of you feel like you would trust an advanced AGI with the future of humanity more than you would trust humanity with the future of humanity? What is the best science fiction you have read along these lines?