There's a class of prophecy that runs:  "In the Future, machines will do all the work.  Everything will be automated.  Even labor of the sort we now consider 'intellectual', like engineering, will be done by machines.  We can sit back and own the capital.  You'll never have to lift a finger, ever again."

    But then won't people be bored?

    No; they can play computer games—not like our games, of course, but much more advanced and entertaining.

    Yet wait!  If you buy a modern computer game, you'll find that it contains some tasks that are—there's no kind word for this—effortful.  (I would even say "difficult", with the understanding that we're talking about something that takes 10 minutes, not 10 years.)

    So in the future, we'll have programs that help you play the game—taking over if you get stuck on the game, or just bored; or so that you can play games that would otherwise be too advanced for you.

    But isn't there some wasted effort, here?  Why have one programmer working to make the game harder, and another programmer to working to make the game easier?  Why not just make the game easier to start with?  Since you play the game to get gold and experience points, making the game easier will let you get more gold per unit time: the game will become more fun.

    So this is the ultimate end of the prophecy of technological progress—just staring at a screen that says "YOU WIN", forever.

    And maybe we'll build a robot that does that, too.

    Then what?

    The world of machines that do all the work—well, I don't want to say it's "analogous to the Christian Heaven" because it isn't supernatural; it's something that could in principle be realized.  Religious analogies are far too easily tossed around as accusations...  But, without implying any other similarities, I'll say that it seems analogous in the sense that eternal laziness "sounds like good news" to your present self who still has to work.

    And as for playing games, as a substitute—what is a computer game except synthetic work?  Isn't there a wasted step here?  (And computer games in their present form, considered as work, have various aspects that reduce stress and increase engagement; but they also carry costs in the form of artificiality and isolation.)

    I sometimes think that futuristic ideals phrased in terms of "getting rid of work" would be better reformulated as "removing low-quality work to make way for high-quality work".

    There's a broad class of goals that aren't suitable as the long-term meaning of life, because you can actually achieve them, and then you're done.

    To look at it another way, if we're looking for a suitable long-run meaning of life, we should look for goals that are good to pursue and not just good to satisfy.

    Or to phrase that somewhat less paradoxically:  We should look for valuations that are over 4D states, rather than 3D states.  Valuable ongoing processes, rather than "make the universe have property P and then you're done".

    Timothy Ferris is again worth quoting:  To find happiness, "the question you should be asking isn't 'What do I want?' or 'What are my goals?' but 'What would excite me?'"

    You might say that for a long-run meaning of life, we need games that are fun to play and not just to win.

    Mind you—sometimes you do want to win.  There are legitimate goals where winning is everything.  If you're talking, say, about curing cancer, then the suffering experienced by even a single cancer patient outweighs any fun that you might have in solving their problems.  If you work at creating a cancer cure for twenty years through your own efforts, learning new knowledge and new skill, making friends and allies—and then some alien superintelligence offers you a cancer cure on a silver platter for thirty bucks—then you shut up and take it.

    But "curing cancer" is a problem of the 3D-predicate sort: you want the no-cancer predicate to go from False in the present to True in the future.  The importance of this destination far outweighs the journey; you don't want to go there, you just want to be there.  There are many legitimate goals of this sort, but they are not suitable as long-run fun.  "Cure cancer!" is a worthwhile activity for us to pursue here and now, but it is not a plausible future goal of galactic civilizations.

    Why should this "valuable ongoing process" be a process of trying to do things—why not a process of passive experiencing, like the Buddhist Heaven?

    I confess I'm not entirely sure how to set up a "passively experiencing" mind.  The human brain was designed to perform various sorts of internal work that add up to an active intelligence; even if you lie down on your bed and exert no particular effort to think, the thoughts that go on through your mind are activities of brain areas that are designed to, you know, solve problems.

    How much of the human brain could you eliminate, apart from the pleasure centers, and still keep the subjective experience of pleasure?

    I'm not going to touch that one.  I'll stick with the much simpler answer of "I wouldn't actually prefer to be a passive experiencer."  If I wanted Nirvana, I might try to figure out how to achieve that impossibility.  But once you strip away Buddha telling me that Nirvana is the end-all of existence, Nirvana seems rather more like "sounds like good news in the moment of first being told" or "ideological belief in desire" rather than, y'know, something I'd actually want.

    The reason I have a mind at all, is that natural selection built me to do things—to solve certain kinds of problems.

    "Because it's human nature" is not an explicit justification for anything.  There is human nature, which is what we are; and there is humane nature, which is what, being human, we wish we were.

    But I don't want to change my nature toward a more passive object—which is a justification.  A happy blob is not what, being human, I wish to become.

    I earlier argued that many values require both subjective happiness and the external objects of that happiness.  That you can legitimately have a utility function that says, "It matters to me whether or not the person I love is a real human being or just a highly realistic nonsentient chatbot, even if I don't know, because that-which-I-value is not my own state of mind, but the external reality."  So that you need both the experience of love, and the real lover.

    You can similarly have valuable activities that require both real challenge and real effort.

    Racing along a track, it matters that the other racers are real, and that you have a real chance to win or lose.  (We're not talking about physical determinism here, but whether some external optimization process explicitly chose for you to win the race.)

    And it matters that you're racing with your own skill at running and your own willpower, not just pressing a button that says "Win".  (Though, since you never designed your own leg muscles, you are racing using strength that isn't yours.  A race between robot cars is a purer contest of their designers.  There is plenty of room to improve on the human condition.)

    And it matters that you, a sentient being, are experiencing it.  (Rather than some nonsentient process carrying out a skeleton imitation of the race, trillions of times per second.)

    There must be the true effort, the true victory, and the true experience—the journey, the destination and the traveler.

    New Comment
    75 comments, sorted by Click to highlight new comments since: Today at 10:59 PM
    Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

    Why do you think there are so many beetles? God is bored. Maybe he created us so we would invent the Wii?


    We'll reject the games where everything is too easy. We will not want to live in a universe which is as we wish it were. It'll be unhealthy for us. We'll throw it off. Instead, we'll play games that are just as hard and frustrating as the life we live now.

    Then, the robots will discover a way to circumvent thermodynamics and harness us as batteries and we'll all start saying "Woah!". Plus all the time playing computer games with help will make us so stupid that we give superintelligent, super-fast hostile ems a warning like "Dodge this!" before we shoot them because we don't comprehend their abilities. (Perhaps they meant that as a metaphor?)

    On the subject of computer games (an underrated area for the study of psychology, economy and even AI, IMO):

    During the last 3 years, I have spent just over 1000 hours playing World of Warcraft. Why did I choose to spend (some, incl my wife, might say waste) my time on this? I am fairly wealthy and quite fit - just about any fun activity is open to me. So why do I like WoW? And why 10 million people around the world do the same?

    Some important reasons why the game is so pleasurable seem to be:

    a) the ultimate goals are pretty clear (so unlike real life...)

    b) the "measures of progress" are likewise clear - and there is only one way to go, namely "up"! (again, so unlike real life, apart from possibly "youth" - is that what makes "youth" so good?)

    c) the rewards are clear - and are earned so progressively that playing the game seems akin to wireheading (the trickle of XPs, "gold" and new items continuously stimulates some pleasure centre or other).

    (Good play involves fairly sophisticated analysis, strategy and tactics, which maintains intetrest... but is beside the point I want to make. The game is attractive to good and poor players.)

    With... (read more)

    Well, does time permit?
    The life is entertaining too. If you want to gamify it, use you'll set your goals, see your progress and achieve the rewards. What else is there to want to?

    @ D. Alex: Some important reasons why the game is so pleasurable seem to be:

    a) the ultimate goals are pretty clear (so unlike real life...)

    b) the "measures of progress" are likewise clear -

    c) the rewards are clear -

    This looks like real life without the hard parts. Sure, it makes it more fun, but at the end will you feel rewarded? If you look back now or in a few years to the time spent playing and consider what you could have achieved in real life if you invested the same time into real challenges how will you feel? From my own experience I can... (read more)

    Sounds like WoW is optimized for System 1 pleasures, and you explicitly reject this. I think that brings up an important point: How can we build a society/world where there are strong optimization forces to enable people to choose System 2 preferences? Once such a world iterated on itself for a couple generations, what might it look like? I don’t think this would be a world with no WoW-like activities, because a world without any candy or simple pleasures strikes me as deeply lacking. My System 2 seems to place at least a little value on System 1 being happy. So I’d guess the world would just have many fewer of such activities, and be structured in such a way as to make it easy to avoid choices we’d regret the next day. If this turns out to a physically impossible problem to overcome for some reason, then I could imagine a world with no System 1 pleasures, but such a world would be deeply lacking, even if that loss was more than made up for by gains in our System 2 values. As a side note, it'd be an interesting question how much of the theoretical per capita maximum value falls into which categories. An easier question is how much of our currently actualized value is immediate gratifications. I'd expect that to be heavily biased toward System 1, since we suffer from Akrasia, but it might still be informative.
    I think the real world qualifies quite well. People who listen to their System 2 achieve much more than people who are slaves to their System 1. If you want stronger "optimization forces", take away the safety net. Hunger and pain are excellent incentives. Not many people would allow themselves to get addicted to WoW if it means they'll become homeless in a short while.
    That provided me with some perspective. I'd only been thinking of cases where we imposed limitations, such as those we use with Alcohol and addictive drugs. But, as you point out, there are also regulations which push us toward immediate gratification, rather than away. If, after much deliberation, we collectively decide that 99% of potential values are long term, then perhaps we'd wind up abolishing most or all such regulations, assuming that most System 2 values would benefit. However, at least some System 2 values are likely orthogonal to these sorts of motivators. For instance, perhaps NaNoWriMo participation would go down in a world with fewer social and economic safety nets, since many people would be struggling up Maslow's Hierarchy of Needs instead of writing. I'm not sure how large of a fraction of System 2 values would be aided by negative reinforcement. There would be a large number of people who would abandon their long-term goals in order to remove the negative stimuli ASAP. If the shortest path to removing the stimuli gets them 90% of the way toward a goal, then I'd expect most people to achieve the remaining 10%. However, for goals that are orthogonal to pain and hunger, we might actually expect a lower rate of achievement. If descriptive ethics research shows that System 2 preferences dominate, and if the majority of that weighted value is held back by safety nets, then it'll be time to start cutting through red tape. If System 2 preferences dominate, and the majority of moral weight is supported by safety nets, then perhaps we need more cushions or even Basic Income. If our considered preference is actually to "live in the moment" (System 1 preferences dominate) then perhaps we should optimize for wireheading, or whatever that utopia would look like. More likely, this is an overly simplified model, and there are other concerns that I'm not taking into account but which may dominate the calculation. I completely missed the libertarian perspective,
    Actual experiments in doing this have proven it to be extremely counterproductive. The more human effort needs to be poured into avoiding hunger, homelessness, and base pain, the less ends up available for serving "self-actualizing" goals, conforming to socially-approved-of lifestyles, or even increasing economic productivity. If you have an intuition which tells you that punishing people makes them act smarter, it is wrong. Punishing people makes them spend mental effort on avoiding getting caught transgressing your norms when they could have spent that effort on something that was actually important.
    LOL. For a lot of people "self-actualization" ends up with sitting on a couch in front of an idiot box, eating chips. Nowadays it might be in front of their FB feed, but that's essentially the same. And I'm not sure what are "socially-approved-of lifestyles" -- that seem to depend a lot on the society in question. No. My intuition is that the threat of pain/hunger/etc. makes people act. Incentives matter.
    Look, the mere fact that you condescend at and disapprove of the actions of others doesn't mean you've proposed any kind of alternative (no, survivalism does not count, that problem was already solved), let alone demonstrated a metric by which your non-proposed alternative is superior (not even the "I like it" metric). Now explain why those actions or incentives matter, that is, what makes them superior to alternatives. No, sneering does not count.
    I think people who mainly listen to system 2 frequently suffer from akrasia. Productive people usually feel motivated to do what they are doing and that's system 1.
    The biggest problem isn't System 1 dominating System 2. It's system 2's being filled with BS and falsehoods.
    Excellent point. Most people aren't trying and failing to achieve their dreams. We aren’t even trying. We don’t have well-articulated dreams, so trying isn’t even a reasonable course of action until we have a clear objective. I'd guess that most adults still don't know what they want to be when they grow up, and still haven't figured it out by the time they retire.
    Evidence or typical mind fallacy..? X-)
    Guilty. I've spent most of my life trying to articulate and rigorously define what our goals should be. It takes an extra little bit of cognitive effort to model others as lacking that sense of purpose, rather than merely having lots of different well-defined goals. (EDIT, to avoid talking past each other: Not that people don't have any well defined sub-goals, mind you. Just not well defined terminal values, and well defined knowledge of their utility function. No well-defined answers to Life, The Universe, And Everything.)
    "Well-defined terminal values" are a very different thing from "well-articulated dreams". P.S. People with "well-defined answers to Life, The Universe, And Everything" are usually pretty scary.
    Or have dreams that would be horrific if actually implemented because they haven't thought through the implications.

    @D. Alex: (For Robin Hanson: have you heard about the economic studies carried out in WoW setting?)

    Since you mentioned economics, did you ever consider the opportunity cost of playing WoW?

    What you describe as targets over '4D states' reminds me of Finite and Infinite Games by James Carse. For an example, playing a game of basketball with a winner/loser after an hour of play is a finite game. However, the sport of basketball overall, is an infinite game. So playing a specific video game to reach a score or pass the final level is a finite game, but being a 'gamer' is an infinite game, allowing ever more types of gaming to take place.


    Unlike Roland, who is obviously a puritan, I rather enjoy the occasional spot of idleness. For a non-trivial number of people, playing WoW for a couple of hours a day is more fun that playing real life. Rather than make thinly veiled moral judgements about folks for their unproductivity, perhaps he should consider what makes certain games so engaging.

    I spent a year playing a lot of WoW, attaining non-trivial sucess in both raiding and competitive PVP, but I gave it away, partly because the time commitment became too great and partly because what passed for progression started to lose its shine.

    So, being suitably qualified, I'll take a stab at a few features that make this virtual social experience psychologically rewarding:

    1. Competition with minimised risk. I love fighting. Seriously - nothing beats the adrenaline buzz, time compression and sheer physicality. Unfortunately even controlled fighting in the physical world entails a level of risk that as a father I'm not willing to assume. Simulated violence, while a poor substitute, helps to fill the void.

    2. Persistent progress. Sure online FPS is fun, but when you log on you're always the same guy (more or less). There's also less ris

    ... (read more)

    @ ac: I agree with everything you said except the part about farming a scripted boss for phat lewt in the future. One would think that in the future they could code something more engaging. Have you seen LOTR...

    EY: you'll find that it contains some tasks that are - there's no kind word for this - effortful. (I would even say "difficult", with the understanding that we're talking about something that takes 10 minutes, not 10 years.)

    Some tasks in WoW can take months to complete, and it's clearly intended by WoW developers. Many tasks require 'raiding', which is an organized, coordinated activity involving up to 40 players, strategy, advance preparations, purchases, crafting etc. -- I have a friend who keeps a calendar of his evening raids and plans his real-world time in advance. When I played WoW, I didn't raid at all because it placed too much constraints on my real-world schedule.

    EY: So in the future, we'll have programs that help you play the game

    To a certain extent, we already do. Speaking of WoW again, we have Thottbot and Ludwig that help you instantly look up any item or spell (this function isn't build into WoW), talent/glyph calculators, forums where people calculate all these probabilities of critical strikes, and a huge number of addons -- for example Auctioneer, which lets people trade at the Auction House far more effectively (you see mean/median/average prices for ... (read more)

    From reading Ainslie's Breakdown of the Will: relatively few. Many addicts will go right back to their drug even after withdrawal is long over, and will even deviously work around their tools (like that one drug which makes any alcohol consumption induce vomiting).

    MMO of the future lol(some swearing)

    And just so I'm not completely off topic, I agree with the original post. There should be games, they should be fun and challenging and require effort and so on. AI's definetly should not do everything for us. A friendly future is a nice place to live in and not a place wher an AI does the living for us so we might as well just curl up in a fetal position and die.

    D. Alex:this is a bit out of left field, but: Is a setting like World of Warcraft a good medium for development of AI? Clear goals, clear measures of progress, sufficient complexity to provide an indication of when important insights are achieved, and a safe environment (in the sense that the path to paperclip AI seems unlikely)...?

    Ben Goertzel proposes a exactly that (but for different reasons):

    How about making games that serve a purpose in the real world? Imagine a virtual world that generates and distributes quests and puzzles based on what kind of (robotic) work is needed in the real world. I guess this would go under "removing low-quality work to make way for high-quality work".

    ... huh. I wonder if Neal Stephenson is a LW reader. See his (most recent?) book, REAMDE, for an implementation of this idea.

    I watch kittens "playing", definitely building useful skills for the future. I enjoy effort on puzzles and games because each gives me a moment of pleasure on success, and no bad consequences of failure, but some games improve reactions or are otherwise useful.

    The "Culture" sequence of novels by Iain M. Banks suggests how people might cope with machines doing all the work. One man works in a cafe, waiting on tables and cleaning up. Yes, the machines could do this work, but he gains happiness from the feeling of serving other people. Oth... (read more)

    EY: So this is the ultimate end of the prophecy of technological progress - just staring at a screen that says "YOU WIN", forever.

    On second thought, playing a modern game IS staring at the "YOU WIN" screen.

    Say, you just started playing a game. You did nothing at all, but you're already immortal, you look badass, you have fists the size of a boulder, and you can engage some mean-looking bad guys and win!

    So, the actual product of the game industry is 4D "YOU WIN" screens.

    I find the opposite to be true, at least for those games I've played recently. For example, Minecraft is so open-ended that I found myself expending more and more time and effort devising subgames and constructing various structures that I realized I was 'working' and further that I'd rather work on something more lastingly rewarding. Interestingly, I haven't found any more lastingly rewarding activity with which I've replaced it.

    There's so much to consider here. For me at least, for something to be fun, I have to know that there's a challenge. For it to be a challenge, there has to be the possibility of failure. There has to be scary parts, that remind you of failure. There has to be multiple real, meaningful, obvious paths that suggest fun in the short term, but disaster in the long term, that you have to look at and reject. There have to be rewards that are enticing, but incredibly rare and difficult, and other rewards that are easier, faster and more localized.

    But more im... (read more)



    "The "Culture" sequence of novels by Iain M. Banks suggests how people might cope with machines doing all the work."

    Exactly, I think Culture is highly relevant to most topics discussed here. Obviously, it is just a fictional utopia, but I believe it gives plausible answer to "unlimited power future".

    For the reference:


    "Since you play the game to get gold and experience points, making the game easier will let you get more gold per unit time: the game will become more fun."

    I know this statement is being set up to be knocked down but when I read it I recoiled in disgust. Many people already do not play games with the end goal of increasing the values in the computer that represent gold or experience, because that is a completely empty and pointless thing to care about. My goal when playing games is "to improve myself" and has been for several years. The most fun I have when playing a game is just after I improve my own skill enough to overcome some challenge in a game that I couldn't do before.

    I mainly play competitive multiplayer games and extremely difficult singleplayer games. I refuse to play any multiplayer game with meaningful persistent character state because that inevitably makes the game revolve around grinding. Grinding is pointless and stupid and typically does not develop much skill.

    "For me at least, for something to be fun, I have to know that there's a challenge. For it to be a challenge, there has to be the possibility of failure."

    I go farther than this. For a game to be significantly fun for me I have to ACTUALLY FAIL. Repeatedly. Until I get good.

    I find Greg Egan's Permutation City relevant here, particularly the character of Peer. He gives himself arbitrary desires over time, say collecting butterflies or making table legs with a lathe. Re-design the brain to enjoy some arbitrary, meaningless task for a finite time. It is one way of implementing jb's notion. At one point, he constructs a simulation that is perfectly circular: the experience is a closed loop that leads him back to his starting point and mental state. He realizes that he could maintain this one infinitely, or, with no loss, just run it once and permanently shut down.

    Or as another character explains, how does it all feel? "However I want it to feel."

    what's more fun? a holodeck that you have complete control over? or a holodeck with built in constraints?

    playing god might be fun for awhile, but I think everyone would eventually switch over to programs with built in constraints to challenge themselves. the profession of highest prestige will probably people who write really really good holodeck programs.

    Computer games are the devil but I agree strongly with Hyphen, the good ones are like sports not work.

    "Though, since you never designed your own leg muscles, you are racing using strength that isn't yours. A race between robot cars is a purer contest of their designers."

    Eliezer: While people don't design their muscles, they presently don't design their brains either, so a robot car-designing contest seems like just as impure a contest. Even if people did repeatedly redesign their brains, wouldn't this either result in convergence, in which case the contestants would be identical and the contest wouldn't be interesting, or alternatively, the arbitrary initial advantages and disadvantages would just be passed on in modified and perhaps even amplified form and the contest stays as impure as ever. Even if you try to measure the amount of effort the contestants put in, that's no good either because different people are born with unfairly different amounts of will-power.

    So what on earth do you mean by "purer contest"?

    This could function as an intuition pump for my own hypothesis that advanced self-modifying minds won't compete at arbitrary preselected goals for fun. (At least not exclusively for fun; they might design themselves to enjoy competing to achieve goals in a pedagogical framework, but actually achieving the goal [or acquiring the skills/knowledge/etc. necessary to achieve the goal] will be the point.)

    I didn't say pure, I said purer. If you write your own legs, that's a purer contest than running on muscles you don't understand. If you rewrite your own mind, that's a purer contest than thinking with neural circuits you don't understand. You never made and can never make yourself from scratch, from nothing; but it's possible that the brain you wield could be Truly Part Of You. The implications of which are not exactly straightforward, but I haven't gotten to the section on introspection yet.

    What can I say, apart from "Progress Quest"

    Officially voted the Top Role Playing Game for Post-Singularity Sentient Beings.


    One common answer to the question "What will we do in the future when we've fixed all that is wrong with today" is "How the hell should I know?"

    For example, imagine our neolithic ancestors asking each other the same question. "What will they do in the future when they don't have to worry about food, shelter, or even disease?" I think they could have imagined some things; "They'll make more complicated art." "They'll have more complicated sports."

    But I don't they they would have imagined full time mathema... (read more)

    That one bothered me too. Perhaps you could say bodies are much more peripheral to people's identities than brains, so that in the running case what is being tested is meat that happens to be attached to you and in the robot case it's you yourself. On the other hand I'd still be me with some minor brain upgrades.

    Oh, massive crosspost.

    Based on the comments here, it would seem that it's the people who reject ultimately-meaningless forms of play - that is, 'play' that doesn't develop skills useful to perpetuation - and concentrate on the "real world" who will end up existing.

    And the Luddites will inherit the Earth...

    Or, to put it another way:

    "Fixing" the future, in a way that renders human beings completely redundant and unnecessary even to themselves, isn't fixing anything. It's creating a problem of unlimited scope.

    If that's the ultimate outcome of, say, producing superhuman minds - whether they're somehow enslaved to human preferences or not - then we're trying very hard to create a world in which the only rational treatment of humanity is extinction. Whether imposed from without or from within, voluntarily, is irrelevant.


    I don't know about you guys but I'm having fun just trying to keep this rock from rolling back down the hill.

    What's most interesting to me is that lizards don't have fun.

    Maybe they have fun. But if they do, I'm pretty sure worms don't have fun. A discussion like this one, carried on by lizards (or worms), wouldn't have included the concept "fun".

    And if you keep going back in time or down in size, I'm sure you'll find organisms that don't experience pleasure.

    Are there other types of possible experiences as qualitatively different and intrinsically good? Are there infinitely many of them? Is charting the course based on "fun theory" like lizards charting the course of the future based on "basking on a hot rock theory"?

    Probably. And if the set of organisms that experience pleasure is a proper subset of the set of organisms that experience fun, then the answer is even more likely to be yes.

    That should have said "as qualitatively different and intrinsically good as fun?"

    "Heaven is being perfect.": Even a circle can't be perfect, in the classical sense of being the best possible circle. Is a circle of 2cm radius better than a circle of 1cm radius? It is much more nonsensical to talk of a person being perfect. It is even more nonsensical to talk of a still-evolving species being perfect.

    I didn't mean "proper subset". I mean that if there are organisms that experience pleasure but not fun (or vice-versa), then it's more likely that there's an infinite number of possible "inherently good" noumena like pleasure, fun, and love; and that we've discovered only a small number of them.

    And the mapping of attraction to those noumena is entirely subjective! Like in another Elizer Yudkowsky essay:

    I'd say that "work" generally consists of activities that are only useful as a means, and not as an end. In general, "work" is anything that you'd rather have someone (or something) else do for you.

    For example, you don't like cleaning the toilet, but you want the toilet to be clean, so you clean it anyway. Cleaning the toilet is work.

    Anything you wouldn't volunteer to do if you weren't getting paid is work.

    So, yeah, let's get rid of work!


    Unlike Roland, who is obviously a puritan, I rather enjoy the occasional spot of idleness. For a non-trivial number of people, playing WoW for a couple of hours a day is more fun that playing real life. Rather than make thinly veiled moral judgements about folks for their unproductivity, perhaps he should consider what makes certain games so engaging.

    I enjoyed playing games myself, so I know what you are talking about.

    You mention idleness, which I agree is sometimes worthwhile. This is the package deal fallacy since there are other ways to achieve tha... (read more)

    I defy this conclusion. online friends are maybe not as socially fulfilling as face-to-face friends, but I wouldn't count them as having zero positive benefits. Especially if you have an obscure hobby/interest only shared by some - or only by a few local people. Doubly-so if you happen not to live in a big city. I have both online and offline friends and both sets of them make up my complete social circle. If you also happen to travel frequently - offline contact with your old circle of friends/family may be the best way to keep in touch. As a member of a family who I can truthfully say the sun-never sets upon... this is a huge benefit.

    caledonian: I agree. if we develop some sort of virtual reality that can provide any desire, we'll just be selecting for people who don't go in and never come out. If so the future will be populated by people who refuse such self gratification.

    increasing your social circle(online friends don't count) It will be a small, lonely post-upload life...

    On the subject of MMORPGs, I've enjoyed playing one for about a year and then it stopped being fun. In the beginning the interesting part was the world exploration, acts of learning new things, rules and interactions between various parts of the system, feeling of steady advance towards some clearly defined goal and work that was guaranteed to pay off. After a while grinding has started to become annoying and my interest shifted towards minmaxing everything and writing complex scripts to allow bots do the boring parts. Then realisation hit me. In real worl... (read more)

    You can be expected to have that kind of fun, but only in the better sorts of games. I'm thinking particularly of games like Minecraft where, after mastering the game's own systems, the player's next activity is "write some neat game systems of your own and share them around so other people can expand their experiences".

    Then realisation hit me. In real world an extraordinarily efficient way of doing things is good, it's called an invention. In a game it is called cheating. Nature doesn't care what smart tricks you used to achieve your goals. In a game if it wasn't anticipated by developers it probably counts as an exploit. The universe has a set of unchanging rules, a game is perpetually balanced by series of patches and crutches in unpredictable places. By being creative you are fighting against game developers, which is pointless because they will actively oppose and ge... (read more)

    EY: I'm not going to touch that one. I'll stick with the much simpler answer of "I wouldn't actually prefer to be a passive experiencer." If I wanted Nirvana, I might try to figure out how to achieve that impossibility. But once you strip away Buddha telling me that Nirvana is the end-all of existence, Nirvana seems rather more like "sounds like good news in the moment of first being told" or "ideological belief in desire" rather than, y'know, something I'd actually want.

    Actually, to me Nirvana - or wireheading - sound like... (read more)

    I don't understand what's the problem Eliezer has with advanced computer games. Why not "waste effort" if it's fun, and all the important work in the real world is anyway getting done?

    No, the end of the road for the MMORPGer wouldn't be staring at an "YOU WIN" screen. That's not fun, I want to instead go waste some effort in e.g. advanced ancestor simulations, possibly with my memories of "the future" temporarily erased.


    Kaj: But if they were made into orgasmium, even for a short while, then they wouldn't want it to ever stop...

    Well, yeah, but most people, if they had their brain removed and replaced by a slab of jelly, wouldn't raise any objections either.

    I don't think wireheading is that different. A wireheaded brain is basically a hacked brain that doesn't work any more.

    (you could also replace "made into orgasmium" by "given heroin")

    Emile: A wireheaded brain is basically a hacked brain that doesn't work any more.

    Sure, if you want to define it that way... but I'd be wary of that kind of thinking. Doesn't work for what anymore? It certainly still works for experiencing phenomenal sensations. Doesn't work for the evolutionary purpose of maximizing fitness? Well, we should try to get past that anyhow. Doesn't work for self-preservation? That's just a poorly implemented wirehead. If one were, say, running as code, you could relatively easily have a subprogram that monitored the environment... (read more)

    "I sometimes think that futuristic ideals phrased in terms of 'getting rid of work' would be better reformulated as 'removing low-quality work to make way for high-quality work'."

    Alternatively, you could taboo work and play entirely, speaking instead of various forms of activity, and their various costs and benefits.


    ...a science fiction story about such a utopia included a conversation between a robot and a person. In the end, the robot said, "But, you can get drunk."

    Do we even need the destination? When you consider "fun" as something that comes from a process, from the journey of approaching a goal, then wouldn't it make sense to disentangle the journey and the goal? We shouldn't need the destination in order to make the journey worthwhile. I mean, if the goal were actually important, then surely we'd just get our AI buddies to implement the goal, while I was off doing fun journey stuff.

    For a more concrete example:

    I like baking fruitcakes. (Something I don't do nearly often enough these days.) Mixing the ra... (read more)

    I totally agree... there are heaps of processes that I enjoy far more than the actual end-result. Crochet is my example. I'm quite happy to continue crocheting something pretty (it has to be pretty - I don't enjoy crocheting abominations) for a long time and never "owning a crocheted thing" at the end. Before I hit upon the solution, I spent a long time starting projects - some of which I finished, but lots I didn't... because I didn't care about finishing - just about doing. Of course, couple this with an aversion to destroying something I've already made (which might have solved the problem by turning it into a sisyphean task). and I got a lot of "why don't you ever finish anything?" from my mother. The question usually comes as "why don't you ever finish anything, don't you want the [crocheted thing] you set out to create?" - and the honest answer is "no".... but if you say that - they ask "well why did you start making it in the first place?" Most people don't seem to understand enjoying the process - at least not on a gut level... I actually solved this particular dilemma by giving away my crocheted things to my grandma - who likes owning crocheted doilies et al. Works for embroidery projects too. Unfortunately, I still tend to get lack of understanding from other people: "but why don't you ever make something for yourself?" I find it very hard to explain to goal-oriented people why I don't like crochet... I like crocheting. I would definitely consider myself to be more process-oriented than goal-oriented. I like doing stuff... I like crocheting, not the goal of having crocheted something in particular. Especially, I like learning - not the feat of "having learned something". So for me - it's very difficult to go to those "attain your goals" seminars etc - because I don't have set goals. I can't point at something and say I want to have achieved precisely that thing, because for me, the thing itself doesn't matter. It can be frustrating, because I cer

    I just want to say that your point about valuing actual people and not (potentially illusory) experiences of people is a very important one, and one I wish I could explain better to people who think that maximizing stimulation to the opiate system is the final word on happiness.

    Though I've never thought of myself as a Singularitarian, one thing I do look forward to that could be called a "singularity" is the point at which all the problems where winning is what counts have been won---no more war, no more cancer, no more poverty. Amazingly, we are about 80% there, and have risen from roughly 20% in the last few hundred years.

    I'm not sure that the difference between 4D states and 3D states is meaningful, with respect to eudaimoniac valuations. Doesn't this overlook the fact that human memories are encoded physically, and are therefore part of the 3D state being looked at? I don't see any meaningful difference between a valuation over a 4D state, and a valuation over a 3D state including memories of the past.

    In other words, I can think of no 3D state whose eudaimoniac valuation is worse than that of the 4D state having it as its endpoint.

    (In fact, I can think of quite a few whic... (read more)


    realistic nonsentient

    But but anti-zombie principle!

    [This comment is no longer endorsed by its author]Reply

    Though, since you never designed your own leg muscles, you are racing using strength that isn't yours.  A race between robot cars is a purer contest of their designers.

    How do you figure? You didn't design your brain, either, so using your intellect to design a robot car is also using strength that isn't yours.