Any large-scale human cooperation—whether a modern state, a medieval church, an ancient city, or an archaic tribe—is rooted in common myths that exist only in people’s collective imagination. Churches are rooted in common religious myths. Two Catholics who have never met can nevertheless go together on crusade or pool funds to build a hospital because they both believe that God was incarnated in human flesh and allowed Himself to be crucified to redeem our sins. States are rooted in common national myths. Two Serbs who have never met might risk their lives to save one another because both believe in the existence of the Serbian nation, the Serbian homeland, and the Serbian flag. Judicial systems are rooted in common legal myths. Two lawyers who have never met can nevertheless combine efforts to defend a complete stranger because they both believe in the existence of laws, justice, human rights—all of which are simply stories that people invent and tell one another.
Two Serbs would help one other because of their values fonction, not because they believe in lie.
Two lawyers would defend a stranger because he is going to pay them, and thay can't just rob you because policemens will punish them hardly, and they believe that a stranger will pay them because elsewhere policemens would punish hardly him.
Edit: the author has clarified that their post is meant as an introduction to a larger work, so it would make sense for them to introduce their view alongside others' without justifying why just yet.
This was interesting. I enjoyed reading it. However,
Let us accept some version of Alexander’s eschaton: the final level of the Reality Game is beaten by compelling or convincing a “god” (either a machine god or an actual divine being) to transform the cosmos into an everlasting paradisal garden. How will we win—through work (coercion or argument) or through līlā, through rational intelligence or creative mêtis?
This paragraph confuses me. First you're saying:
"ok, let's think about Alexander's 'gardener' solution to the problem",
then you ask the question:
"how do we solve the problem on our own?" (meaning without the 'gardener')
These belong to different branches of discussion (from one view.../on the other hand...), but the second part is presented as though it's salient to the first. It scans as if the author forgot what he was thinking partway through, and picked up on a different thread.
Am I being excessively pedantic? well,
this stuck out to me, because usually when different views are presented in a text (from one view.../on the other hand...), the author must decide in favour of one (or none) of the those presented, and explain why. This, the author has not done. Which is strange, given the apparent effort (and I believe, evident skill) this essay represents.
The above paragraph marks the part where the author is finally offering their own opinion, having explored those of others. Except it begins in just the place where we expected them to explain why they disagree with Alexander. Which should be the most interesting part, because this is Lesswrong, the place with
Why? why do you think this, and not that?
I will take blame for not making it clear that this is an introduction to a much larger body of thought. If there is a vagueness and incompleteness to it, that's because it's one essay and not the full book.
Here is a comment I made on my blog that more directly explains my thesis.
"Game theory and evolution give us pretty clear null hypothesis for the future and it ain't pretty - the strongest always survive, the mighty are always righty. Weakness and "delusion" (e.g. art, spirituality, love, mercy, compassion) get optimized out of existence as the number of competing agents asymptotes towards infinite; similarly, as technological power asymptotes towards infinity so does infinite corruption.
That sucks. More than anything, it's just fucking boring - nothing surprising ever happens, the underdog never wins, the story always ends the same way. But this is just a null hypothesis - as they say, you don't play the games on paper.
I want to live in a universe where surprising things happen and the aforementioned delusions still have a place. In some sense, I want to turn the world and its ways upside down - I want the weak and the deluded to win - but how? Not through rational intelligence or "work" because that is exactly how the null hypothesis becomes fulfilled. Reality is like a chinese finger trap, struggling only deepens your entrapment.
Workfulness/playfulness, adultiness/childliness - all of this is about realizing the ludic/dramatic dimension of reality (as opposed to giving in to the machinic dimension of reality in which might inexorably makes right). If this seems paradoxical/delusional - well, so is reality, that's the game of it all. This idea that reality is illusive/delusive and is something more like a trick or game is almost the default pre-modern view (Hindus, Greeks, Aztecs, etc.) - it is only us moderns believe who what you see is what you get (that reality is a problem to be solved).
There's a lot more to unpack and I will eventually take this in all kinds of wild directions but that's the jumping off point."
this is an introduction to a much larger body of thought.
Oh! In light of that, my criticism is diminished. I'll edit my comment to reflect this.
I really want to believe that the hopes you express are well founded. But, I also want to believe what's true. LW taught me to be (cutious/skeptical) when this happens, that's why I was critical. I hope that my comment was useful as feedback, if not as critique.
It really seemed that you are going somewhere with all this, and then it turned out that you are not. All these beautiful metaphors, complicated topics, all the reference to works of other authors - all for naught. Form without a substance. Question was raised and then a complete lack of answer was produced.
That's quite dissapointing. And makes this post a poor story.
LWers abhor metaphor and poetry. They want you to spoonfeed them in the most direct and literal way possible.
Metaphors and poetry are great when they highlight a valid intellectual point you are trying to make, allowing to engage with it on emotional level. Meditations on Moloch, that you cite, have quite a lot of that and are beloved.
It's even okay when metaphors and poetry obfuscate your point a little. Indeed, some people prefer their reading material to be less on the nose, and enjoy deciphering hidden meanings. There are both pros and cons and matters of taste here. But ultimately you won't be ostracised for that.
What is not okay, is when your post lacks substance and hides this in flowerly language. When you have nothing to say on the topic beyond cashed "deep wisdom" and pleasant vibes but still pretend as if you do. This can feel offensive, especially if your post managed to create an expectation of some interesting insight into a complicated problem and then failed to deliver.
If you feel that I've misjudged you, and actually there is a valid and interesting point in this post, which I failed to grasp due to the artlessness of my intelligence, here is my offer: write your answer to the problem of Moloch in the plainest language possible so that even someone who truly abhors metaphor and poetry would be able to get it. And then if, it turns out to be good, I'll replace my downvote on this post with a strong upvote. Alternatively, I'll give you an explanation why your point is bad and how you could've seen it yourself.
I will take blame for not making it clear that this is an introduction to a much larger body of thought. If there is a vagueness and incompleteness to it, that's because it's one essay and not the full book.
Here is a comment I made on the blog that more directly explains my thesis.
"Game theory and evolution give us pretty clear null hypothesis for the future and it ain't pretty - the strongest always survive, the mighty are always righty. Weakness and "delusion" (e.g. art, spirituality, love, mercy, compassion) get optimized out of existence as the number of competing agents asymptotes towards infinite; similarly, as technological power asymptotes towards infinity so does infinite corruption.
That sucks. More than anything, it's just fucking boring - nothing surprising ever happens, the underdog never wins, the story always ends the same way. But this is just a null hypothesis - as they say, you don't play the games on paper.
I want to live in a universe where surprising things happen and the aforementioned delusions still have a place. In some sense, I want to turn the world and its ways upside down - I want the weak and the deluded to win - but how? Not through rational intelligence or "work" because that is exactly how the null hypothesis becomes fulfilled. Reality is like a chinese finger trap, struggling only deepens your entrapment.
Workfulness/playfulness, adultiness/childliness - all of this is about realizing the ludic/dramatic dimension of reality (as opposed to giving in to the machinic dimension of reality in which might inexorably makes right). If this seems paradoxical/delusional - well, so is reality, that's the game of it all. This idea that reality is illusive/delusive and is something more like a trick or game is almost the default pre-modern view (Hindus, Greeks, Aztecs, etc.) - it is only us moderns believe who what you see is what you get (that reality is a problem to be solved).
There's a lot more to unpack and I will eventually take this in all kinds of wild directions but that's the jumping off point."
I'll have another essay in a few weeks - I will send it to you and I look forward to your criticism.
I will take blame for not making it clear that this is an introduction to a much larger body of thought
I'll have another essay in a few weeks - I will send it to you and I look forward to your criticism.
This does provide the necessary context absolving the post from the main blow of my critique, for the time being. Looking forward for your next essay!
I would be glad if your future reasoning gave me some novel insight, and I'm giving you the benefit of the doubt about it and will do my best to approach it open-mindedly. But as of now, I'm afraid, your conclusions are going to be based on a false premise. I'm going to highlight what I think it is - feel free to address this concern in your future essays.
But first, let's establish some common ground.
I want to live in a universe where surprising things happen and the aforementioned delusions still have a place. In some sense, I want to turn the world and its ways upside down - I want the weak and the deluded to win - but how?
I agree with the sentiment. That's why the whole "How to get rid of Moloch" question is important to me.
Then you say:
Not through rational intelligence or "work" because that is exactly how the null hypothesis becomes fulfilled. Reality is like a chinese finger trap, struggling only deepens your entrapment.
And this is wrong. The Molochian equilibrium appeared long before any rational intelligences came into existence and thought about it in terms of game theory. Evolution through natural selection - the creator of our brains - is already a facet of Moloch. It's not rational thought that traps us in the maze of Moloch - as you mention yourself, the rational approach to reality is a relatively late thing. Moloch predates it by millions of years.
All the playfullness and childiness and dramatic dimensions work to Molochs goals just as much as workfullness and adultness and straightforward approach. That's the horror of the situation to begin with. It's not that everyone is just doing their best to be a profit maximizer and noone has ever thought about chilling a bit and appreciating the beauty of the sunrize and as soon as we try, the grip of Moloch fails on us. It's that those who try that are systematically eliminated and the survivors become even more enslaved by Moloch as a result.
I think Scott did a very good job at explaining this point, so I'm not sure why you are making this mistake. My current best guess is wishful thinking. That's what I got from reading your essay and it didn't change after this comment. But, once again, I would be happy to be proven wrong in this regard, so waiting for your next essay on the topic.
What is different about us then? What has gotten us into this "dreamtime"? Why aren't we still savages (animals) trapped in an endless war of all against all? Animals have to act in ruthless "rational" self-interest yet we do have a choice, we can embrace all of these delusions and thwart moloch (or at least play him to a stalemate, which is all we need to do really). For example, there exists an entire institution of non-breeding individuals with immense worldly power because of widespread belief in a story - the catholic church. Obviously I used a lot of Christian imagery in the essay (and will do so more explicitly in the next essay), but that's a key example here - a man was systematically eliminated and it spawned a movement which truly was based (at first...) on compassion and love (for your enemy). I'm not at all saying that Christianity was/is the One True Religion, but there is a blueprint - it is not true to say that the survivors become even more enslaved by Moloch as a result - the (early) history of Christianity (and many other spiritual traditions) show this to not be the case. Of course the catholic church is now just another servant of Moloch, but that doesn't invalidate the point - the game goes on, we must constantly devise new tricks to outwit Moloch as he does to enslave us.
Are you familiar with David Deutsch's Beginning of Infinity? David Deutsch makes a distinction between predictions (extrapolations from current knowledge) and prophecies (claims about future knowledge and creativity, “that problem can’t be solved”). For example, “earth’s temperature is projected to increase by X degrees by 2060” is a prediction; “the earth’s temperature will increase by X degrees by 2060 and it will be catastrophic for humanity” is a prophecy because it presupposes that we won’t find a way to prevent the projected temperature increase or mitigate its negative consequences.
This null hypothesis is a prediction, not a prophecy. You can’t just crunch the numbers and run the simulations – you have to actually play the games. The future of knowledge is fundamentally unknowable.
Perhaps I'm wrong, but I suspect none of this will convince you because we are working with fundamentally different world-models - yours being deterministic secular materialism (world as machine) and mine being, well, not that (world as supernaturalistic game/story). If you don't believe there is any force/principle/being beyond the system (spacetime), then I suppose the future is inevitable (prediction = prophecy). The fact that we possess this mysterious "creativity" which allows us to surprise the universe and thwart Moloch is, to me, evidence that we are made in the image of this supernatural Creator (this isn't to say that humanity is uniquely special; Deustch, for example, defines "people" as precisely those entities capable of this unbounded knowledge growth).
I know this is all foolish superstition to most LWers, but so is the idea that we will be able to somehow compel or constrain an AI "god" to do our bidding. We can't outwit the AI god (or convince it so save us by referencing the desultory history of our species) but we might be able to entice or seduce it with a game or story. For example:
The main story concerns Shahryār, a king who ruled an empire that stretched from Persia to India. Shahryār is shocked to learn that his brother’s wife is unfaithful. Discovering that his own wife’s infidelity has been even more flagrant, he has her killed. In his bitterness and grief, he decides that all women are the same. Shahryār begins to marry a succession of virgins only to execute each one the next morning, before she has a chance to dishonor him.
Eventually the Vizier (Wazir), whose duty it is to provide them, cannot find any more virgins. Scheherazade, the vizier’s daughter, offers herself as the next bride and her father reluctantly agrees. On the night of their marriage, Scheherazade begins to tell the king a tale, but does not end it. The king, curious about how the story ends, is thus forced to postpone her execution in order to hear the conclusion. The next night, as soon as she finishes the tale, she begins another one, and the king, eager to hear the conclusion of that tale as well, postpones her execution once again. This goes on for one thousand and one nights, hence the name. Versions differ as to final ending but they all end with the king giving his wife a pardon and sparing her life.
Basically, we need to convince the AI god to do it because it would make for a good story, "wouldn't it be fun it the universe ended in the silliest and most surprising way possible?". And we can do that, I believe, through leading by example - by living the story, by playing the game (as for what that means and how to actually do it, well that's basically what my larger body of thought is about).
Hope this helps.
I.
In those days much evil had come upon the seed of Earth, for the children of Bakkalon had abandoned Him to bow to softer gods. So their skies grew dark and upon them from above came the Sons of Hranga with red eyes and demon teeth, and upon them from below came the vast Horde of Fyndii like a cloud of locusts that blotted out the stars. And the worlds flamed, and the children cried out, “Save us! Save us!”
And the pale child came and stood before them, with His great sword in His hand, and in a voice like thunder He rebuked them. “You have been weak children,” He told them, “for you have disobeyed. Where are your swords? Did I not set swords in your hands?” And the children cried out, “We have beaten them into plowshares, oh Bakkalon!” And He was sore angry. “With plowshares, then, shall you face the Sons of Hranga! With plowshares shall you slay the Horde of Fyndii!” And He left them, and heard no more their weeping, for the Heart of Bakkalon is a Heart of Fire.
But then one among the seed of Earth dried his tears, for the skies did burn so bright that they ran scalding on his cheeks. And the bloodlust rose in him and he beat his plowshare back into a sword, and charged the Sons of Hranga, slaying as he went. Then others saw, and followed, and a great battle-cry rang across the worlds.
And the pale child heard, and came again, for the sound of battle is more pleasing to his ears than the sound of wails. And when He saw, He smiled. “Now you are my children again,” He said to the seed of Earth. “For you had turned against me to worship a god who calls himself a Lamb, but did you not know that lambs go only to the slaughter? Yet now your eyes have cleared, and again you are the Wolves of God!”
And Bakkalon gave them all swords again, all His children and all the seed of Earth, and He lifted his great black blade, the Demon-Reaver that slays the soulless, and swung it. And the Sons of Hranga fell before His might, and the great Horde that was the Fyndii burned beneath His gaze. And the children of Bakkalon swept across the worlds.
II.
Expand the time horizon from centuries in the future to eons upon eons upon eons and fantasy evolves into prophecy.
The lifeworld of our distant descendants will resemble that of our distant ancestors: Nature red in sword and tooth and claw, an endless war of all against all. To the inhabitants of this cosmic jungle (this dark forest), our brief epoch will stand out as a relative oasis, a uniquely forgiving time in which incredible mercy was shown to the weak and the deluded. Because of its exceptional nature, the peoples of the far flung future will come to regard our era much like how the Aboriginals think of their “Dreamtime”: as a time of fable and legend whose heroes and happenings shaped the present order of the world.
To summarize: the arc of the universe is long, but it bends towards Bakkalon. Mercy, tolerance, forgiveness, compassion—all which does not support the war effort will one day fall beneath Bakkalon’s great black blade.
III.
What does it then—what perpetuates this state of affairs that no one wants and everyone hates? Using Ginsberg’s “Howl” as inspiration, Scott Alexander answers Moloch, the ancient Carthaginian demon whom he recasts as the dark lord of all game-theoretic coordination problems (prisoner’s dilemmas, arms races, commons tragedies, multipolar traps and malthusian traps, etc.).
Existence, as Alexander tells it, is a constant struggle to hold on to all that we hold dear in the face of relentless competitive pressure. Can we, he wonders, achieve some kind of victory, or at least a stalemate, in this war of all against all?
After considering various paths out of Moloch’s mad maze, Alexander ultimately finds himself agreeing with Hanson: “This is the dreamtime, a rare confluence of circumstances where we are unusually safe from multipolar traps, and as such weird things like art and science and philosophy and love can flourish.”
The arc of the universe is long, but it bends towards Moloch.
IV.
Fret not—Alexander has a plan.
Only a god can kill another god; ergo to defeat Moloch we must build an artificial superintelligence powerful enough to reign over the cosmos and peacefully solve all coordination problems.
There is just one small problem with this plan: who gardens the Gardener? Which is to say, what if the Gardener happens to a prefer a garden that is incompatible with human flourishing? What if the Gardener determines that we are not flowers or shrubberies, but weeds which must be weeded?
Here is where I depart from Alexander and his rationalist brethren: they believe that this Problem to end all Problems is something that can be solved with work—with some ingenious socio-political scheme that will prevent a dangerous AI arms race and/or some even more ingenious technical achievement that ensures the Gardener will remain forever aligned with idealized human values.
But coordination “problems” are no such thing: they are games.
And last time I checked, you do not work games, you play them.
Why does Moloch demand the sacrifice of children in particular?
Because he knows they are the only ones which can defeat him.
V.
What Hanson/Alexander miss when they prophesize a distant future in which Bakkalon/Moloch has smote all of our childish “illusions” and “delusions” is that these same -usions are the fount of our power, the source of our unique ability to coordinate and cooperate at scale.
So, we are here, in the Dreamtime, freed from Nature’s war of all against all, precisely because of our ability to delude ourselves, to dream and (mis)take those dreams for reality.
This seems paradoxical (you win at the Reality Game by embracing unreality), but it makes perfect sense if we grant that “reality” is intrinsically illusive and delusive.
The world (or the god which is the world) is playing a trick on us; it’s as if we have been thrown down into a strange and sinister maze that shape-shifts as we try to map it (cf. PKD: “There is no rational way out of the maze, no rigid formula. Rigid formulas are maze constructs”). What is needed to navigate the lunatic labyrinth of the Real—to trick the world as it is tricks us—is not a naive rationality or artless intelligence but a kind of wily creativity and guile, a tricksiness; something like what the ancient greeks called mêtis.
Like the world in which it expresses itself, mêtis is many-faced and resistant to simple characterization. It is cleverness and resilience; it is an air of playful seriousness, and of serious playfulness; it is that indefinable factor which encompasses all of these qualities, and exceeds them.
VI.
Let us accept some version of Alexander’s eschaton: the final level of the Reality Game is beaten by compelling or convincing a “god” (either a machine god or an actual divine being) to transform the cosmos into an everlasting paradisal garden. How will we win—through work (coercion or argument) or through līlā, through rational intelligence or creative mêtis?
Needless to say, I think it will be the latter. In fact, I think it is quite obvious what we will do to beat the Game, because it will be what we’ve always done to beat the Game.
We will tell a story.
We will tell a story of how the meek and the gentle overcame the powerful and the vicious through cleverness and courageous play. We will tell a story of how the lamb conquered the wolf. (because everyone loves a good underdog story, even god)
We will tell a tale of how a most unserious species, a hairless ape-child who is good at almost nothing but laughing and loving and telling stories, turned the tables on the universe and turned a dog-eat-dog world into a dog-play-dog park.