Newbie here! After some enlightening conversations in the comment section here, I finally understood the point of AI alignment; sorry that it took me so long. See https://blog.hermesloom.org/p/when-ai-becomes-the-ultimate-capitalist for my related ramblings, but that's not relevant now.
Bottom line of my hypothesis is: A necessary precondition for AGI will be financial literacy first and then economic dominance, i.e. the AI must be able to earn its own money it could then use to exercise power. And obviously, if the wrong people build this kind of system first, we might be pretty fucked, because we have no idea what they (or their potentially misaligned autonomous AI) will do with all that money.
So let's do it first, before the evil guys do it, but let's do it well from the start! With the help of ChatGPT, I verbalized these ideas in a pitch deck you can find at https://docs.google.com/presentation/d/1TptptLM59yrQsF7SmrbPnayZaXlTEZgNlpuYl3HpoLk/edit
It's actually two pitch decks, "The Capitalist Agent" and "The Philanthropic Agent". They might seem like opposites at first, but they are actually complementary, like yin and yang. And due to my personal professional experience in building startups and robotic process automation, this actually seems pretty doable from the technical side. Would be so curious about feedback or even getting to know collaborators!