In the experience machine thought experiment, you choose to either continue your life as you are currently living it, or spend the rest of it inside a machine that cultivates a world perfectly fitted to you. It was introduced to argue against hedonism, but we can imagine the machine pushing for any metric we care about—it doesn’t necessarily have to be pleasure. For example, we can consider a non-hedonistic experience machine which makes you feel a perfect balance of happy, challenged, loved, fulfilled, empowered, or anything else you care about.
The philosophical tension in this formulation is supposed to be between any experience-based ethics and some intuitive feeling that the experience machine is inherently deceptive. The thing that feels wrong is that the experiences don’t seem real in some important way. Robert Nozick, the creator of the original thought experiment, says “we want to do certain things, and not just have the experience of doing them.” This allows us to ask: Does Nozick's objection[1] make sense? Can we explain what realness is or why it matters to begin with?
Interestingly, I have found that many people are unable to understand what this thought experiment is getting at. They say things like “I wouldn’t want to go into the experience machine because part of what makes life enjoyable is the fact that you sometimes struggle, feel pain, etc.” But if this is true, our non-hedonistic experience machine would account for it! By construction, we are considering an experience machine which is simulating the ideal world for you. Still, people think we cannot artificially construct a world better than the one we already have. I try to explain, “Well, imagine your life, with nothing changed, but without kidney stones, broken AC in the summer, etc. Surely there are some adjustments to your life which would make you happier?” I think people do think the world can be better but not in a way an experience machine could create. These people claim the experience machine can’t have anything to offer in a way that makes me feel like they fundamentally don’t understand what we are attempting to talk about.
Coxon’s “rhetorical failure”
In early October of this year, Jacob Coxon appeared on The Daily Show to warn Jon Stewart's audience of AI danger. It’s a very interesting interview. I suggest people watch it in full.
Many of the YouTube comments were positive. I think that much of Stewart’s audience left feeling like this was not merely a troubling interview but a highly important one. It seems like Coxon communicated effectively how humanity is at risk, and the audience was largely receptive.
Now here’s what Coxon himself wrote on Twitter after the interview:
In this interview I tried to argue that a post-scarcity society will benefit working people. It was largely a rhetorical failure. People don’t trust that the benefits will be distributed, and tech optimists have to fix this communications issue.
When Coxon discussed how he thinks aligned AI could usher in a kind of AI utopia, the audience wasn't with him. He doesn't get into too much detail, but Coxon's good AI future is roughly "No jobs. Universal high income."
This leaves Stewart appalled, saying, "I think for the most part, even when we're doing jobs we don't particularly like, that humans like to feel relevant." He also feels like the AI companies aren't giving the outside world a choice. That Coxon and SF have a utopia in mind, that is probably a dystopia, but the outside world isn't allowed to object. Stewart's responses are met with thunderous applause from the studio audience.
The SF man
Utopias feel like the mask an evil thing wears to appear friendly. People have heard too-good-to-be-true promises ending in tragedy before and are on their toes.
So it’s hard to talk about utopia, but it’s especially hard when you are Jacob Coxon. It’s probably fair to say there is a widespread suspicion of the young, successful tech guy who wants to dramatically change a bunch of stuff and thinks everyone is going to really, really like it. One comment under the YouTube video of the interview reads, “This has the feeling of John asking an Alien representative what the Aliens are about to do to humanity.” Importantly, Coxon doesn't fit the classic tech optimist caricature! He quit his job warning of tech-induced extinction! But the tech optimist character is strong enough in the public consciousness that Coxon is easily pattern-matched to it, and therefore is subjected to a bunch of assumptions about what is really going on here.
At one point, Stewart almost says the quiet part out loud. He explains to Coxon, with a gentle paternal tone, that the AI industry “lives in a bubble of its own making. And that they should really get out and hug some people because in the real world, that’s not reality.” I think that this is Stewart’s way of saying “Candidly, I think you are weird. I don’t trust that you really understand people. Your utopian vision is fundamentally antisocial in a way you may not be fully capable of understanding.” To be clear, I don't think that this is fair, but I believe this undertone is present throughout the interview.
Coxon’s vision fails translation
Through this lens, Coxon's statements take on a certain meaning. When Coxon says "No jobs. Universal high income," I understand him as saying:
Look. AI superintelligence is going to basically be magic and could transform the world in any way it wanted to. The set of possibilities is so vast. It's hard to imagine that traditional jobs and the 40-hour workweek are literally the best of all those possibilities. I don't think machines should just replace humans. But I imagine that if the superintelligence was on our side, it would look more like every human is empowered to build whatever life they want, and wouldn't be forced to work a job they don't find fulfilling just to pay the bills. In simple language: "No jobs. Universal high income."
But this is not what is heard. I think what is heard is something closer to:
You people hate your jobs! And you have to work them because you are poor. My AI god could help with that. It can replace your human job so you can focus on other stuff. It would solve your other problems too so you can just hang out and be happy.
Get ready for your jobless future
The public can have a difficult time imagining a bad AI future. But I think it can be equally difficult for people outside of the EA/rationalist/AI safety sphere to understand what we mean by a good AI future. It's not only that they don't want it, but they don't even understand what's being proposed.
No matter how I describe my non-hedonistic experience machine, the assumption is that the utopia it promises is really a hedonistic hell. In the same way, when someone says AI could make the future radically better, I think the public is imagining that AI will solve all of our problems for us and there won't be any purpose for humans, leaving us with a WALL-E-style dystopia.
In WALL-E, humans leave an uninhabitable planet Earth to just drink soda and watch TV and stuff.
To be clear, I'm not claiming that the experience machine is good or equivalent to what Coxon is proposing. I am more trying to say that it produces the same kind of confusion. When Coxon says, "AI can transform our world to be radically better!" the audience hears "READY OR NOT, IM BUILDING THE FUCKING EXPERIENCE MACHINE LETS GOOOOOO!!!! GET READY FOR YOUR JOBLESS FUTURE!"
I think there is something generalizable here. Painting in broad strokes, the public has a set of expectations for what the future will look like, and it's hard to picture something different. Perhaps a different set of biases also makes it hard for optimists to imagine AI dystopia.
In my favorite moment of the interview, Stewart asks Coxon about his time at OpenAI and Anthropic: "I don't mean this in the wrong way, but what did you think you were doing? Like honestly, you're there for four years and all of a sudden you're like 'these motherfuckers[2] are smart!'" Coxon responds, "That's completely fair. It was always in the long term. There was this big scary thing in the future. But man. I struggle to think too far into the future and now it's just right there on the doorstep."
He continues, "So now it's right there. It's actually scary. In the past ... it's just hard to visualize a decade into the future." Indeed, when you have a certain future in mind, it's difficult to imagine what another may hold.
Experience machines
In the experience machine thought experiment, you choose to either continue your life as you are currently living it, or spend the rest of it inside a machine that cultivates a world perfectly fitted to you. It was introduced to argue against hedonism, but we can imagine the machine pushing for any metric we care about—it doesn’t necessarily have to be pleasure. For example, we can consider a non-hedonistic experience machine which makes you feel a perfect balance of happy, challenged, loved, fulfilled, empowered, or anything else you care about.
The philosophical tension in this formulation is supposed to be between any experience-based ethics and some intuitive feeling that the experience machine is inherently deceptive. The thing that feels wrong is that the experiences don’t seem real in some important way. Robert Nozick, the creator of the original thought experiment, says “we want to do certain things, and not just have the experience of doing them.” This allows us to ask: Does Nozick's objection[1] make sense? Can we explain what realness is or why it matters to begin with?
Interestingly, I have found that many people are unable to understand what this thought experiment is getting at. They say things like “I wouldn’t want to go into the experience machine because part of what makes life enjoyable is the fact that you sometimes struggle, feel pain, etc.” But if this is true, our non-hedonistic experience machine would account for it! By construction, we are considering an experience machine which is simulating the ideal world for you. Still, people think we cannot artificially construct a world better than the one we already have. I try to explain, “Well, imagine your life, with nothing changed, but without kidney stones, broken AC in the summer, etc. Surely there are some adjustments to your life which would make you happier?” I think people do think the world can be better but not in a way an experience machine could create. These people claim the experience machine can’t have anything to offer in a way that makes me feel like they fundamentally don’t understand what we are attempting to talk about.
Coxon’s “rhetorical failure”
In early October of this year, Jacob Coxon appeared on The Daily Show to warn Jon Stewart's audience of AI danger. It’s a very interesting interview. I suggest people watch it in full.
Many of the YouTube comments were positive. I think that much of Stewart’s audience left feeling like this was not merely a troubling interview but a highly important one. It seems like Coxon communicated effectively how humanity is at risk, and the audience was largely receptive.
Now here’s what Coxon himself wrote on Twitter after the interview:
When Coxon discussed how he thinks aligned AI could usher in a kind of AI utopia, the audience wasn't with him. He doesn't get into too much detail, but Coxon's good AI future is roughly "No jobs. Universal high income."
This leaves Stewart appalled, saying, "I think for the most part, even when we're doing jobs we don't particularly like, that humans like to feel relevant." He also feels like the AI companies aren't giving the outside world a choice. That Coxon and SF have a utopia in mind, that is probably a dystopia, but the outside world isn't allowed to object. Stewart's responses are met with thunderous applause from the studio audience.
The SF man
Utopias feel like the mask an evil thing wears to appear friendly. People have heard too-good-to-be-true promises ending in tragedy before and are on their toes.
So it’s hard to talk about utopia, but it’s especially hard when you are Jacob Coxon. It’s probably fair to say there is a widespread suspicion of the young, successful tech guy who wants to dramatically change a bunch of stuff and thinks everyone is going to really, really like it. One comment under the YouTube video of the interview reads, “This has the feeling of John asking an Alien representative what the Aliens are about to do to humanity.” Importantly, Coxon doesn't fit the classic tech optimist caricature! He quit his job warning of tech-induced extinction! But the tech optimist character is strong enough in the public consciousness that Coxon is easily pattern-matched to it, and therefore is subjected to a bunch of assumptions about what is really going on here.
At one point, Stewart almost says the quiet part out loud. He explains to Coxon, with a gentle paternal tone, that the AI industry “lives in a bubble of its own making. And that they should really get out and hug some people because in the real world, that’s not reality.” I think that this is Stewart’s way of saying “Candidly, I think you are weird. I don’t trust that you really understand people. Your utopian vision is fundamentally antisocial in a way you may not be fully capable of understanding.” To be clear, I don't think that this is fair, but I believe this undertone is present throughout the interview.
Coxon’s vision fails translation
Through this lens, Coxon's statements take on a certain meaning. When Coxon says "No jobs. Universal high income," I understand him as saying:
But this is not what is heard. I think what is heard is something closer to:
Get ready for your jobless future
The public can have a difficult time imagining a bad AI future. But I think it can be equally difficult for people outside of the EA/rationalist/AI safety sphere to understand what we mean by a good AI future. It's not only that they don't want it, but they don't even understand what's being proposed.
No matter how I describe my non-hedonistic experience machine, the assumption is that the utopia it promises is really a hedonistic hell. In the same way, when someone says AI could make the future radically better, I think the public is imagining that AI will solve all of our problems for us and there won't be any purpose for humans, leaving us with a WALL-E-style dystopia.
In WALL-E, humans leave an uninhabitable planet Earth to just drink soda and watch TV and stuff.
To be clear, I'm not claiming that the experience machine is good or equivalent to what Coxon is proposing. I am more trying to say that it produces the same kind of confusion. When Coxon says, "AI can transform our world to be radically better!" the audience hears "READY OR NOT, IM BUILDING THE FUCKING EXPERIENCE MACHINE LETS GOOOOOO!!!! GET READY FOR YOUR JOBLESS FUTURE!"
I think there is something generalizable here. Painting in broad strokes, the public has a set of expectations for what the future will look like, and it's hard to picture something different. Perhaps a different set of biases also makes it hard for optimists to imagine AI dystopia.
In my favorite moment of the interview, Stewart asks Coxon about his time at OpenAI and Anthropic: "I don't mean this in the wrong way, but what did you think you were doing? Like honestly, you're there for four years and all of a sudden you're like 'these motherfuckers[2] are smart!'" Coxon responds, "That's completely fair. It was always in the long term. There was this big scary thing in the future. But man. I struggle to think too far into the future and now it's just right there on the doorstep."
He continues, "So now it's right there. It's actually scary. In the past ... it's just hard to visualize a decade into the future." Indeed, when you have a certain future in mind, it's difficult to imagine what another may hold.
It's worth noting that this isn't his only objection, but it highlights what I consider to be the most obvious, interesting tension.
He means the AIs.