To me, this post seems perhaps better-suited for the Discussion section. It's too long to be a comment but doesn't really strike me as necessarily meriting an independent mainpage post.
SarahC: I do like the idea of living by your own strength somewhat though... a sense of helplessness and lack of skill is incredibly demoralizing.
Alicorn: I don't want to stop people from doing that; I just don't really find it appealing and object to enforcing it
SarahC: actually, I often think that for me and a large number of people I know, the biggest negative factor in our lives is awareness that we're not good at much.
Alicorn: I'm good at some stuff. To do that stuff, I like to use tools. I don't want to eat bugs for fifty years in a fake savannah while I work out how to fire ceramics so I can cook.
SarahC: oh. yeah. that's undeniable.
Alicorn: Alicorn-who-has-to-live-by-her-own-strength has to get PHENOMENALLY bored before she ever picks up a flute again. She sings, when she wants to make music, because she hasn't come around to finding whittling interesting yet. And this is sad for Alicorn-who-has-to-live-by-her-own-strength.
SarahC: agreed.
This was probably a joke, but: This strikes me as not much better. It's still "I have to do something largely unrelated before I can do what I actually wanted to do." Imagine you had a flute-playing competition, but only those who were also good singers could enter! The best specialized flute-players might not be able to enter at all.
I doubt I'd do an unassailable job. And an unassailable job is probably necessary, if everyone will abide by Laws of Fun forever.
...
Unless we plan to deceive them about the nature of the postsingularity universe, their not being allowed X will be known to be somebody's fault. Someone believed in some Laws of Fun that they programmed into an AI that determined that the best way to optimize for that kind of Fun was to disallow X.
One point of that sequence is that typical suggestions for Utopias are abominably flawed, that there are certain things that could be done much better, and that people don't spontaneously notice these flaws upon hearing a description of a Utopia. The sequence draws attention to certain problems of human judgment, and trains you to notice such problems whenever you see another proposed "Utopia", makes you less gullible.
It is emphatically NOT a suggested set of rules for an AGI to enforce, indeed one of the arguments that could be drawn from that sequence is that trying to manually construct such rules is a bad idea, that lists of issues such as that sequence or one given in this post will inevitably follow from any simplistic ruleset. You don't program rules into an AI, you program a way of figuring out what the rules should be (and those rules have to get down to the level of saying how to arrange atoms, so won't have much to do with verbal descriptions of human condition).
Both this post and the one linked seem to be both about fictional utopias for literature, and actual optimal future utopias. These are completely unrelated issues the same way good fictional international conflict resolution is WW3, and good real world international conflict resolution is months of WTO negotiations over details of some boring legal document between 120+ countries.
More seriously, Internet shows a lot about what people truly like, since there's so much choice, and it's not constrained by issues like practicality and prices. Notice total lack of interest in realistic violence and gore and anything more than one standard deviation outside of sexual norms of the society, and none of these due to lack of availability.
Eh? Total lack of interest? Have you ever been on 4chan? Realistic violence threads crop up regularly over there, and it's notorious for catering to almost any kind of sexual deviance the average person can think of. (Out of curiosity: what would you consider "more than one standard deviation" outside the sexual norms of the society? How about two?) I say almost, because 4chan is regulated and it isn't the go-to place for quite everything; child pornography nets its posters permabans pretty quickly and it doesn't have the dedicated guro boards of its Japanese counterpart. Which is to say nothing of blood sports like traditional bullfighting or cockfights, for which even a quick search on YouTube can offer some clips (relatively mellow and barely containing any actual blood as they are).
Stuff like that may not match the t...
I think the criticism of 6 is a misunderstanding. It doesn't say "the world resembles the ancestral savanna", it says "the world resembles the ancestral savanna more than say a windowless office". The best environment is unlikely to be anything like the ancestral savanna, but it's likely to be closer to that than to a windowless office, in terms of sensory experience. The point I think is not the specifics of the environment, but that it engages with our bodies and senses in a way that we, as evolved creatures, find satisfying, and in a way that the purely mental stimulation available in the office does not.
That's what I took away from the linked post.
In general, the post was about trying to design utopia. Both the real world, and most proposed utopias, fail miserably at both all of these things and your rejoinders.
Eliezer's world has small happy surprises. The current one has "Surprise! Your friends are all dead." While the "Laws of Fun" might need to be replaced entirely, something like Fun Theory really ought to exist.
I've never experienced that disquiet, and it worries me. I read Brave New World and while the unnecessary torment of the Gammas and Deltas bothered me, my only thought about the lifestyle of the doped-up, lavishly entertained, socially and physically secure Alphas and Betas was "Awesome! Can't wait." Same thing with the Experience Machine -- if you really could lock everyone who wanted one into their own personal optimal-experience simulator, where the only catch was that it wasn't "real," and we stipulate that the machine works as advertised, I'd sign up without much hesitation.
If somebody in real life offered that kind of opportunity, I'd turn it down, because we can't really be sure that it would work as advertised...but if we really think that life in one of these utopias would be an improvement over the status quo, what's to find disquieting?
my modification to the reedspace lower bound: your utopia should be at least as good as the one I could give myself with a star trek holodeck.
...27 and 28 are useful tools for fiction and interesting thought exercises, but it is not how we build houses (mine is right-side-up and has its plumbing in its bathrooms rather than on the roof, thank you). It may not be how we should build eutopias in which we hope to actually live. I like comfort. I expect culture to change radically - the way cultures do when time passes and/or things change - once everyone has settled into transhumanity, but the form that transhumanity takes needn't itself be frightening, and trying to design this in sounds like a b
Thing is, people who are anomalous in some direction aren't particularly rare. Alicorn's specific anomaly might not warrant a rule-rewrite on its own, but the meta-point that all of these rules might be able to evoke 'AAAAAAAH' from some subset of the population does seem like an important one.
Perhaps, but she's far from alone. I'm mostly with her on this one; letting people live in ignorance we can cure just so they can appreciate knowledge more when it's eventually obtained makes about as much sense to me as letting them suffer from illness we can cure just so they can appreciate health.
If I understand this post correctly, the main thesis seems to be something along the lines of "different people should live by different rules, with rulesets locally optimized for the person". Fine, but there still has to be some sort of metaruleset by which local ordinances are chosen. And, per 17/18, user customization is not necessarily the right choice here.
Upvoted for content (although the piece could be made more readable if it's going to be on Main). You might also post this as a comment on the "Laws" post, or at least a shorter version with link.
7 is the one I have the biggest problem with. The opposite of happiness is sadness, not boredom. Anyone who says otherwise fails at opposites, and should probably retake the first grade.
Yes, happiness and sadness are both sides of the same coin, as are love and hate, but they are sides that are as far away from each other as it is possible to be on that coin, and anything not on the coin isn't pertinent.
Opposites are not things with as few commonalities as possible. Black has more common with white than it does with ketchup, but that does not make black a...
(I was scooped by Wedrifid, but here's my phrasing.)
The opposite of happiness is sadness [...] Anyone who says otherwise fails at opposites, and should probably retake the first grade.
I say otherwise. Specifically, it seems to me that the opposite of happiness is a type error; not everything has to have an opposite, and most things don't. The purest examples of what we call opposites have a duality to them, a one-dimensional-ness: it makes sense to speak of hot and cold as opposites because temperature has an order relationship to it; if the temperature is changing, it's either getting hotter or it's getting colder, but not both or neither. Similar remarks could be made about left and right, or the boolean values true and false. But most useful concepts are way too complicated for this kind of duality to apply: what's the opposite of blogging? What's the opposite of graphite? What's the opposite of Vernor Vinge? These questions simply have no sensible answer.
Oftentimes we like to contrast two different things, but this should really be kept conceptually distinct from those things being opposites. I would expect your star first grader, when queried about the opposite of cat, to ...
The opposite of cat is clearly not dog, but rather cow.
After all, we don't eat dogs or cats, but we do eat cows.
And you can keep a dog or a cat in an apartment, but you can't keep a cow. Also, cows have horns; dogs and cats don't.
Also, they revered cats in ancient Egypt and they revere cows in modern India; ancient and modern are opposites; and Egypt and India are on opposite sides of the navel of the world.
Wow, I think we've invented a new version of the Rationalization Game.
[Meta] Quick quibble about presentation, it would help if you quoted or summarised the laws rather than just the number, I'm flicking between tabs which is disorientating. [Otherwise good article.]
10 is just easily improved. It is good to be able to do for yourself, but it is also good to sometimes do for others and to let others do for you. If you don't fully understand that statement, go to burning man -- this is the first and best thing it can teach you.
A lot of people value making things, but the type of things I want to make is different from the type of things I want to use. Many people want to make art of various kinds, fewer would value making disposable plastic packaging. There is also the fact that if a non expert is making something for the fun of it, there is a good chance it just won't work. I might want to play a functional flute despite having made a flute that sounds like strangled cat. And if you want to build anything high tech, you can't easily work your way up from raw materials. Many people would like to do some programming without having to build microelectronic components.
People want change, and they want to have an effect on their world.
A utopia that actually satisfies people is going to change, so any particular vision of utopia is of a temporary state.
A lot of people also become very fond of things they've experienced, so a utopia will include retro-utopias.
I suggest that a utopia has to have resilience built in so that it can recover if it starts turning into something its members don't want, and that this is worth speculating on, rather than just listing the good things you want in a utopia.
Suggestions for a resilient utopia?
I always consider this to be The blunder Eliezer has ever made, and beneath that worried that it might not and that any words that'd not be hell to everyone except me would be hell to me. Thanks for releasing that worry a bit.
I've never been convinced that gender neutral pronouns actually fulfill the underlying function of language -- namely, to communicate. Especially with so many competing standards. Personally, I use the singular 'they.' I mean, yes, it's not technically correct, but people understand you.
Especially with so many competing standards. Personally, I use the singular 'they.' I mean, yes, it's not technically correct, but people understand you.
And, after all, 'em eir ey' aren't technically correct either. On account of not being words.
I have mellowed in the last year or so. I no longer downvote every comment that uses that kind of language. They no longer have the same close ties with an abhorrent (local) political agenda so I can now consider them more or less acceptable.
EDIT: Most unexpected significant and rapid downvoting of one of my comments ever. I retract it, including the downvote policy change - I have returned to considering the subject as distasteful politics.
9 AAAAAAAAAAAAAAAAAAAAAAAH
You've never enjoyed a pleasant surprise? I can imagine not being pleasantly surprised at largish changes in your life, like where you live, work, have relationships, etc. But what about littler things? Like reading a book you thought would be good, but have it turn out to be GREAT instead? Or finding a five-dollar bill on the ground?
I think maybe a good corollary to 9 might be "Different people appreciate different scales of pleasant surprises. Some people will be delighted to find out they can switch to an awesome new career. Other would prefer something smaller-scale, like finding a little money on the ground. Adjust accordingly"
Actually, going back to Eliezer's article expanding rule 9, Justified Expectation of Pleasant Surprises, he imagines two possible worlds:
But in one world, the abilities that come with seniority are openly discussed, hence widely known; you know what you have to look forward to.
In the other world, anyone older than you will refuse to talk about certain aspects of growing up; you'll just have to wait and find out.
I ask you to contemplate—not just which world you might prefer to live in—but how much you might want to live in the second world, rather than the first. I would even say that the second world seems more alive; when I imagine living there, my imagined will to live feels stronger. I've got to stay alive to find out what happens next, right?
I take the first option.
My problem with the second is that the real world contains enough surprise already, without having to add artificial, fake surprise. I've got to stay alive to find out what happens next, anyway, and I can do without UFNIs sticking their oar in and treating me like their pet cat. ("Oh yes, you do like a surprise, don't you, I just know you do, yes you do, it will be a wonderful surprise yes it will etc.etc.")
What makes people happy varies from person to person. I think the problem that we should be solving is how to help people make optimal decisions for their joys. Once we get a great solution to that problem, then we can just make a Utopia where people are totally free, except to physically harm others or their property.
Reflecting on all of this, it occurred to me that the world of the show "Adventure Time" keeps pretty close to Eliezer's laws (investigating this thoroughly is left as an exercise because it's my bed time)
I definitely think it would be a fun world to live in, but it's certainly not something that I'd force on the world.
That said, in direct contradiction to taw's comment, there are a number of episodes where they spend time having movie night, looking at cute things from the internet, playing video games all day, or having ice cream eating marathons.
Certainly humans should do things, but what if I want to play the flute now and only have flute-whittling down as my activity for century seventeen?
Well, what if you do? The whole point is that getting everything you want is not necessarily a proper utopia.
Sometimes - often - I just want people to tell me stuff.
Again, utopia does not necessarily consist of getting everything you want.
Prerequisite reading which you will probably want open in another tab for reference: 31 Laws of Fun
Unprefaced, this post might sound a lot like I'm just picking on Eliezer, or Eliezer's particular set of "laws". I'm sort of doing that, but only as a template for ways to pick on Laws of Fun in general. The correct response to this post is not "Here is my new, different list of N things that will satisfy everyone".
(Well, it would be if you could do that. I'm skeptical.)
If I purported to come up with general laws of fun, I might or might not do a better job. Probably I'd do a better job coming up with a framework for myself; I might also be more cautious about assuming human homogeneity, but I doubt I'd do an unassailable job. And an unassailable job is probably necessary, if everyone will abide by Laws of Fun forever. An unassailable job of Legislating Fun is needed make sure that some people aren't caught between unwanted mental tampering and, probably not Hell, but a world that is subtly (or glaringly) wrong, wrong, wrong.
Please do not assume that I outright endorse unmentioned laws; these are just the ones I can pick at most obviously.
I fully expect to be told that I have misunderstood at least half of these items.
6 sits uncomfortably. The savannah is where we were designed to survive, but evolution is miserly; it is not where we were designed to thrive gloriously. (Any species designed to thrive gloriously there which was actually put there would find its descendants getting away with more and more corner-cutting until they found a more efficient frontier. Creatures that can fly don't keep flight just because flying is awesome; they must also need it.) I want a home designed for me to thrive gloriously in, not one that takes its cues from the environment my ancestors eked out a living in. I suspect this is more like a temperate-clime park than a baking savannah, and it might be more like an architecturally excellent house than either. "Windowless office" is not the fair comparison. That is not how we design places to put people we like.
8 sounds just wrong, or like a misstatement. Why should we get better and better? Objecting to flat awesomeness sounds like a matter of denying that it is flat. Perhaps it just sounds tedious for things to be the same level of awesomeness forever, perhaps it sounds inevitable that the hedonic treadmill will pull downward, but that tedium or treadmill means the awesomeness is not really flat. If it's actually awesome, by all means let's take a flying leap there and carry on forever. (Certainly things should not get worse over time.)
9 AAAAAAAAAAAAAAAAAAAAAAAH
10 sounds like it's missing an option. What can people do for each other? In our world, buttons do stuff for us, because people with whom we are interdependent made them do that. In the ancestral environment, people ate food that others hunted and gathered, they listened to music others played, they carried stuff in baskets others wove, etc. Gifts are good. Specialization is not evil. Certainly humans should do things, but what if I want to play the flute now and only have flute-whittling down as my activity for century seventeen?
12 sounds okay provided it is not interpreted to forbid really great video games, roleplaying scenarios, fiction in general, etc.
13 clashes with a lot of the other laws. (What if I don't want to live my life according to things like, oh, Law 9 [AAAAAH]? Is someone going to stop me from planning myself a predictable future?) People might work best under different rules, too.
15 isn't even a law, it's a problem statement. Same with 16.
17 sounds like artificial difficulty. Just because there is a challenge there and I don't have the road to get around it doesn't stop me from acknowledging that someone else does. If there is an AI around, and it could get me out of this jam, a psychologically-untampered-with Alicorn will resent that it is making me do stuff if I don't happen to want to do it, the same way I resented busywork in school that didn't happen to interest me. Eventually I plan to try making my own butter. In the meantime, I'm glad that's not a step in making scrambled eggs.
20 sounds idiosyncratic or case-by-case. (I take it to be intended as a stronger statement than the mere "there are situations where you can tell people true things and it doesn't help them" - even if that's almost false there has to be some perverse scenario to make it so.) Sometimes - often - I just want people to tell me stuff. See also 17; the fact that someone else knows, even if nothing I can do will get them to tell me, makes my not-knowing something of a fake problem.
23 sounds like an outright contradiction of 22. What do you want to do - solve problems by messing with the environment, or "nudge" people to solve a "statistical sex problem"? Are there not any environmental changes that would accomplish this? Condoms made a dent, didn't they? What would literally perfect birth control and disease protection do? More energy, better health, more time? Literally perfect privacy? Better information for everyone about how to be good in bed? Non-twisted culture to raise new minds in? Optional perfect body/avatar modification for everyone? That took me three minutes to come up with. Leaping straight to "nudges" is... discomfiting!*
27 and 28 are useful tools for fiction and interesting thought exercises, but it is not how we build houses (mine is right-side-up and has its plumbing in its bathrooms rather than on the roof, thank you). It may not be how we should build eutopias in which we hope to actually live. I like comfort. I expect culture to change radically - the way cultures do when time passes and/or things change - once everyone has settled into transhumanity, but the form that transhumanity takes needn't itself be frightening, and trying to design this in sounds like a bad idea.
The general undercurrent through the laws "people should not have [access to] X" thing sounds problematic. Unless we plan to deceive them about the nature of the postsingularity universe, their not being allowed X will be known to be somebody's fault. Someone believed in some Laws of Fun that they programmed into an AI that determined that the best way to optimize for that kind of Fun was to disallow X. Somebody is going to want X and somebody will be disappointed.
It is one thing to have a Eutopia that is scary, but it sounds so terribly sad to have one that is disappointing.
And I have never yet sincerely overestimated human heterogeneity.
*I think Eliezer in general assigns more-than-average importance to gender as a factor in personality, anyway... often in a heteronormative way.