I think any FAI-generated [e]utopia worth calling one would have to allow safe self-modification abilities and the right to choose one's own fun, so you have little to worry about in this respect. Though if things go well, there'll be forms of fun available to us (both wireheady/sensory and externally-motivated fun) that are orders of magnitude better than any you can imagine now; I support your right to scratch as many itches and listen to as much recent popular music as you want indefinitely into the future, and any FAI should as well, but you probably won't actually keep wanting to do all of these for very long.
I think the future will be more complex than that. The fact that unbounded fun will be available to me points to the hypothesis that at some moment a revolutionary modification of our body-mind structure will be necessary.
but you probably won't actually keep wanting to do all of these for very long.
The original post was stating clearly that I could possibly will. And to please not encode your skepticism into an AI :)
You seem to be confusing terminal and instrumental values.
Do you really value "scratching an itch" in itself, such that you would be saddened if that activity was not available to you anymore? What if you gain a cybernetic body that doesn't itch anymore, but its splines need the occasional reticulating, which feels even better than an itch?
What if then another AI offers you to disregard the perfectible cybernetic body and live in a simulation in which there's just an enormous single pleasure available to you 24/7? If you disregard "parochial" values, in wedrifid terminology, what prevents you to collapse into an orgasmium? The point was not only that I value the pleasure coming from those activities, but that I value those activities as well.
You can value other things than actions themselves and the pleasure they bring. Maybe you like choosing and planning and acting. Maybe you like solving complex problems. Maybe you like variety (laughter and food and discovery and music, rather than the best out of four) and complexity. Maybe you like growing and learning. Maybe you like creating things. Maybe you like overcoming obstacles. Maybe you like exchanging ideas with other minds and bulding relationships and helping. Maybe you like intricate webs of feelings that aren't all pleasant but are made more enjoyable because they're bittersweet or frightening or weird.
So you'll select activities that maximize that and your zillion other values, not just activities that maximize pleasure. But there's no particular reason why current activities should be optimal. The pleasure of scratching an itch is unique, and somewhat complex. I wouldn't trade it for, say, a more intense copy of the pleasure of eating a potato chip - I already have that one. But I would trade it for a different, more intense, more complex pleasure.
But I would trade it for a different, more intense, more complex pleasure.
Which is my thesis: outside of denying change, the only two dimension under which is significant to find an improvement are intensity and complexity. However your position appears to me not to be reflectively consistent: after all, you could consider the complexity and intensity of your whole human experience and be seduced by a different, more complex, more intense type of existence.
As I parse it, yes! It'd have to be done by gradual change (I don't want to die and be replaced by someone cooler), but yeah, that's what I want. Where am I inconsistent?
Thank Aumann, we agree! I said it was inconsistent not to be seduced by the cooler existence :)
Hmm, don't we disagree about connotations? You seem to think "orgasmium" covers that kind of being that has varied, complex, intense fun, and does things instead of just feeling like it does them. Do you?
No I don't. My reply to muflax was not related from this one. I don't have a name for this state of affair, it doesn't even seem to have an official name. Cooler existence is ok for me
So what is it you are asking for, then? Merely that you don't just get the fun, but the things that cause the fun as well?
If so, then that's a pretty straightforward point that any FAI would respect. (Even though I tend to think it's mistaken, or at least, awfully inefficient.)
I'm curious. You're alone in the universe, will always be. After centuries of hard work, you've finished your Theory of Everything and Then Some. Your mind strains under the ecstasy of your discovery. You're now in a room with two buttons. Button A resets you to your mindstate thirty seconds ago, lets you experience those thirty seconds of extreme bliss, then repeats, for five minutes. Button B entertains you for five minutes with mildly amusing videos of cats dressed as anime characters. Which button do you press?
You are not resolving my curiosity! (Well, you're telling me I haven't completely misunderstood your position, which is something.) I'd like to know your reasons for your reasons for choosing A (your reasons for choosing A are as you say obvious).
My intuitions say that A is wrong on the same level that replacing all kinds of fun by a big orgasm is sad. This is connected to the intuition that overcoming things is good (and you can't cheat by tying the "Behold! I, human, have vanquished this!" emotion to eating a potato chip), and that choosing and planning and acting are fun. I'm not done unpacking them.
(Come to think of it, I'm not sure I press B, though I probably do. But if B gives me a small, pleasant, new insight (say, a new way to prove Pythagoras's theorem) instead, I press it.)
I choose A because A is the most fun thing to do. I find it really hard to get any deeper reasons than that. I'll try to elaborate anyway and hope it doesn't get too messy. It would be really weird for humans to have that large a value dissonance, so there's probably some confusion around here somewhere.
A couple of months ago, I even would've agreed with you and would've opposed straight orgasmium. But I just can't find any argument against it. I tried tracing my enjoyment of activities, but I couldn't find nuanced, complex fun there. It's all composed of few (probably <5) components. I enjoy solving complex problems, but that's because it's one of the few ways I know to trigger the sensation of enjoying-to-learn. It's the only reason I even bother with programming anymore. If I could trigger the same learning experience by reading a dictionary, I'd get back to studying Latin. It seems very much possible to just straight maximize all components and reach perfect orgasmium. (I strongly suspect they are all bounded, as I already have found my personal Best Movie[1] and Best Song[2] and have for years not been able to improve one bit on that enjoyment, only match it.)
If I could self-modify to eat chips like Light (Death Note spoiler), I totally would. Why is that sad or cheating? Who are you wronging? What code are you breaking? If it's a way to maximize fun, then so be it. It doesn't match common tropes, it doesn't have much status and you don't look clever for doing it, but if it works, then that's that. Not my fault that the universe has obvious cheat codes built in. Screw the dungeon, just get the cheese already.
I strongly suspect that I don't actually care that my values are fulfilled outside of my experience. I see no reason why anyone would. The only explanation that even begins to make sense to me is memetic hijacking, as Tim Tyler often proposes. Claiming to oppose hedonistic optimization might easily be evolutionary beneficial. It looks similar to normal altruistic trade-offs. (Of course this doesn't discredit these more complicated values. A rational agent is free to value whatever it wants, regardless how it got these values. But it would explain why some agents have them and others don't.)
(I'm assuming the least convenient world where there aren't any choices around that are more enjoyable than A, where being locked in a loop of pressing A doesn't prevent me from fulfilling some other desires and so on. As I already have a Theory of Everything and am alone, I can't off the top of my head think of any value that would be left out anyway.)
[1] Moulin Rouge, though Wanted) and Sky Captain and the World of Tomorrow and others are equally good
[2] 嘘とワンダーランド - Asian Kung-Fu Generation (I listened to it literally ~4000 times in a row when I found it)
I tried tracing my enjoyment of activities, but I couldn't find nuanced, complex fun there.
You like G. R. R. Martin and you don't have complex fun?! You said you have depression, this is probably linked (boo for flat affect). I do have complex fun. Like, just yesterday I ate a peach and a pastry and compared everything about the texture and the taste and the temperature, had an uncontrollable urge to run my fingers over every surface in the house, and thought about reducing qualia. (And, uh, I already eat crisps like Light, but like him I would enjoy it more if it were part of an intricate world-conquering plot.)
If it's a way to maximize fun, then so be it.
Clippy would say "If it's a way to maximize paperclips, then so be it. " I don't care only about fun, I care about a zillion things only a few of which fall under the usual concept of fun. I want the dungeon. You don't go in the cave of trials for the items, you go for the challenge and the made-more-valuable-by-the-challenge items. I don't want to stare at a "You win" screen forever.
And I don't actually know why! All I can say is "Becoming orgasmium? That sounds pointless. I'm not going to renounce complexity and variety to make my value system more elegant, it's just not worth it".
I do know why I value things outside my own experience. (Will shoot you and get tortured for it rather than let you press the blow-up-the-world-when-we're-both-dead button, for example.) There's no reason optimization processes should optimize their input sequences only. Clippy doesn't want to overestimate the number of paperclips in the world.
You like G. R. R. Martin and you don't have complex fun?!
GRRM is one of the few authors who writes fiction that doesn't insult my intelligence. It's not that I prefer his work to, say, Naruto. It's just that his is the only I can enjoy at all. I remember reading much more trash in my childhood and found them equally enjoyable. If I try reading them now, I just keep on rolling my eyes and raging.
You said you have depression, this is probably linked (boo for flat affect).
I don't think so. I don't feel depressed right now (luckily, for months). I sometimes did have a fairly flat affect; that ain't now. The degradation effect I described seems fairly common. Lots of experiments in happiness studies show that set levels adjust ruthlessly. Also, Daoists and Buddhists, among many others, have advocated simplicity in favor of complexity and it seems to work.
(Which is the reason I intentionally don't learn to cook so I can still enjoy really cheap and simple food. I don't want to fall into the same trap my obsession with fiction has gotten me into.)
Clippy would say "If it's a way to maximize paperclips, then so be it."
Yes, and Clippy would be right. So? I don't see how this is a counter-argument. This reminds me of AIXI agents wireheading themselves. I don't think that's a failure mode. It's a perfect valid action and if anything shows that they actually are clever. Of course it sucks for the programmers who wanted to get other things done. Similarly, evolution might've taken care to prevent this happening in humans for the same reasons, although it failed in my case. (Pure speculation.)
I do know why I value things outside my own experience.
Why? Honest question, I just get "404" if I ask myself that.
My perspective is that there are rewards and actions to get these rewards. I think that the rewards are the important thing. They are not instruments to get to my "real" values, they are my values. I was set up to favor specific pathways to these rewards for reasons that are not my own (but those of my genes, memes or other influences). I, too, sometimes feel bad for bypassing them, but that is a very obvious security feature and I'm much more willing to consider this aversion to be in error than the bypass.
For example, D&D has many ways to get Infinite Wish Spells. They completely break the game and prevent all the intended benefits of the game, so it's a bad idea for the players to use them as shortcuts. It's much better to go through long, dangerous dungeons than to just ignore everyone else and ascend to godhood and take over the whole universe. But that's because the one who is making the decisions is the player, who wants to socialize and boast and so on. If you lived in D&D, this intuition, this aversion to do things that weren't "intended", would not serve you well. Now you should go for the game-breaker.
The most obvious anti-hedonism feature is boredom. I agree with you that
I don't want to stare at a "You win" screen forever.
But I consider this boredom to be a bad feature! I wish I could just stare at "you win" instead of having to jump through all the hoops every time, in increasing difficulty, with dwindling supply, as has already happened. I've watched >90% of every anime even remotely interesting to me and can't rarely get any real enjoyment out of them anymore because this stupid boredom is adjusting my happiness set level down. I'd much prefer to just get The Perfect Entertainment and have it stay that way, forever. Simplicity and elegance are good things.
I notice that this discussion makes me feel resigned and sad, so I will adjust down my confidence that this is right. Can you try to elaborate on why you value external things? Why you think a holodeck is bad, except for the emotions that come up?
(Besides, just pay me more than the 10 bucks Dave offered and I'm not pressing anything. I'm very pain-averse, so no need to go through such extremes. ;))
The degradation effect I described seems fairly common. Lots of experiments in happiness studies show that set levels adjust ruthlessly.
New fun gets old. We want variety over time as well as space. Doesn't affect complexity or external referents.
Yes, and Clippy would be right.
I meant that fun-maximizers don't have more power to move me than paperclips-maximizers.
I do know why I value things outside my own experience.
Why? Honest question, I just get "404" if I ask myself that.
Just that there's no particular reason for my values to be only about my experience, like there's no reason for them to be only about parts of the universe that are green.
My perspective is that there are rewards and actions to get these rewards. I think that the rewards are the important thing. They are not instruments to get to my "real" values, they are my values.
Yeah, I understand that. I don't know why I disagree. I value the reward, but I also value many things about the action.
As an example: in Judaism, you want to set obstacles for yourself and to overcome them (there even is a bunch of rules to prevent this from going overboard). There are vegan Orthodow Jews who complain that keeping kosher is too easy for them and who decide some vegetables count as milk and others as meat and they don't mix them. Chosen fetters like that appeal to me like mad. It's a kind of fun, but, I expect, one completely alien to you.
I was set up to favor specific pathways to these rewards for reasons that are not my own (but those of my genes, memes or other influences).
I have to ask where your own reasons come from, causally speaking.
Now you should go for the game-breaker.
Agree we should break the rules to get what we want, disagree about what we want.
But I consider this boredom to be a bad feature!
So you want classical orgasmium, not complex fun with no referent (like falling in love with a chatbot). Glad that's clear.
Simplicity and elegance are good things.
Yes. But not so good that I'd renounce humor and challenge and pride and aesthetics and freedom and truth and more. Same reason that I can't decide to burn down an orphanage - I care about making a point about choosing my own values and relativity of morality, but I care about human life more.
I notice that this discussion makes me feel resigned and sad, so I will adjust down my confidence that this is right.
I've been adjusting in the opposite direction as well. Truly we are an Aumann cage match made in heaven.
Can you try to elaborate on why you value external things?
Why you think a holodeck is bad, except for the emotions that come up?
Same reason I think that if I could never have anything more pleasant than ice cream (but I'd live forever and get as much complex and varied fun as I want and nothing horribly bad would be happening), it'd be bad. It's missing stuff.
(Besides, just pay me more than the 10 bucks Dave offered and I'm not pressing anything. I'm very pain-averse, so no need to go through such extremes. ;))
Shooting you would be a last resort. I like you humans, I don't wanna kill you.
Been thinking more and noticed that I'm confused about how "terminal values" actually work.
It seems like my underlying model of preferences is eliminativist. (Relevant caricature.) Because the decision making process uses (projected and real) rewards to decide between actions, it is only these rewards that actually matter, not the patterns that triggered them. As such, there aren't complex values and wireheading is a fairly obvious optimization.
To take the position of a self-modifying AI, I might look at my source code and find the final decision making function that takes a list of possible actions and their expected utility. It then returns the action with the maximum utility. It is obvious to me that this function does not "care" about the actions, but only about the utility. I might then be tempted to modify it such that, for example, the list always contains a maximum utility dummy action (aka I wirehead myself). This is clearly what this function "wants".
But that's not what "I" want. At the least, I should include the function that rates the actions, too. Now I might modify it so that it simply rates every action as optimal, but that's taking the perspective of the function that picks the action, not the one that rates it! The rating function actually cares about internal criteria (its terminal values) and circumventing this would be wrong.
The problem then becomes how to find out what those terminal values are and which of those to optimize for. (As humans are hypocritical and revealed preferences often match neither professed nor introspected preferences.) Picking the choosing function as an optimization target is much easier and always consistent.
I'm not confident that this view is right, but I can't quite reduce preferences in any other consistent way. I checked the Neuroscience of Desire again, but I don't see how you can extract caring about referents from that. In other words, it's all just neurons firing. What these neurons optimize is being triggered, not some external state of the world. (Wireheading solution: let's just trigger them directly.)
For now, I'm retracting my endorsement of wireheading until I have a better understanding of the issue. (I will also try to not blow up any world as I might still need it.)
I was set up to favor specific pathways to these rewards for reasons that are not my own (but those of my genes, memes or other influences).
I have to ask where your own reasons come from, causally speaking.
Good point. I can't just disown all reasons or "I" become a rock, which doesn't appeal to me, identity-wise. I like minimalist identities the most, so I retain pleasure = good, but not reproductive success, for example. In other words, I keep the basic mechanism that evolution gave me to achieve goals, I ignore the meta-goal of reproductive success it had.
I'm not happy with this argument, but I find extended versions that care about externals just as implausible. The choice between both seems arbitrary, so I go with the simpler one for now.
Also, not sure what happens to the value of suicide if you value only your subjective experience. Isn't it undefined?
Yes. Death itself has fairly close to 0 utility to me, but I don't like dying (because of the pain and shame it causes me, mostly), so I'm normally against suicide.
Can you try to elaborate on why you value external things? As I said earlier, why wouldn't I? I value non-green things.
Ok, fair. I can't provide a better case for even why "pleasure" is good, but "pain" ain't. It just feels that way to me. That's just how the algorithm works. I'm just surprised that this difference in perceived values exists. If I further add MrMind's stated values, either terminal value acquisition is fairly shaky and random in humans or easy to manipulate or very hard to introspect on, despite the appearance to the contrary.
A thought experiment. Imagine "reality" disappears suddenly and you wake up in Omega's Simulation Chamber. Omega explains that all your life has been a simulation of the wallpaper kind. There weren't any other minds, only ELIZA-style chatbots (but more sophisticated). Would this make you sad?
I don't get a particularly bad response from that, maybe only slight disappointment because I was mistaken about the state of the world. I take that as weak evidence that I don't care much about referents. But maybe I just have shitty relationships with people and nothing much to lose, so I'll try improving in that regard first, to make that intuition more reliable. (That's gotta take me some time.)
ETA:
The degradation effect I described seems fairly common. Lots of experiments in happiness studies show that set levels adjust ruthlessly.
New fun gets old. We want variety over time as well as space. Doesn't affect complexity or external referents.
What about sustainability? What if we run out of interesting complexity?
Been thinking more and noticed that I'm confused about how "terminal values" actually work.
It seems like my underlying model of preferences is eliminativist. (Relevant caricature.) Because the decision making process uses (projected and real) rewards to decide between actions, it is only these rewards that actually matter, not the patterns that triggered them. As such, there aren't complex values and wireheading is a fairly obvious optimization.
To take the position of a self-modifying AI, I might look at my source code and find the final decision making function that takes a list of possible actions and their expected utility. It then returns the action with the maximum utility. It is obvious to me that this function does not "care" about the actions, but only about the utility. I might then be tempted to modify it such that, for example, the list always contains a maximum utility dummy action (aka I wirehead myself). This is clearly what this function "wants".
But that's not what "I" want. At the least, I should include the function that rates the actions, too. Now I might modify it so that it simply rates every action as optimal, but that's taking the perspective of the function that picks the action, not the one that rates it! The rating function actually cares about internal criteria (its terminal values) and circumventing this would be wrong.
The problem then becomes how to find out what those terminal values are and which of those to optimize for. (As humans are hypocritical and revealed preferences often match neither professed nor introspected preferences.) Picking the choosing function as an optimization target is much easier and almost consistent.
I'm not confident that this view is right, but I can't quite reduce preferences in any other consistent way. I checked the Neuroscience of Desire again, but I don't see how you can extract caring about referents from that. In other words, it's all just neurons firing. What these neurons optimize is being triggered, not some external state of the world. (Wireheading solution: let's just trigger them directly.)
For now, I'm retracting my endorsement of wireheading until I have a better understanding of the issue. (I will also try to not blow up any world as I might still need it.)
I think we are approaching the core of the problem raised by my post. Maybe this thread warrants an entrirely new discussion post (I've still 14 karma point to put in the line ;)), if you and Muflax agree to start afresh.
Yeah, I think we've got each others' positions here (MrMind wants to stay in meatspace, muflax wants to move to a holodeck, MixedNuts wants to move to an expensive simulation and also to know with so many handles start with 'm'). That's a better starting point for debate than bluntly stating one of your conclusions. Is there a way to co-write posts? (Also, what does muflax think? What about the rest of LW?)
Sure, no problem. Both of you can feel free to contact me via mail or Jabber (IRC, too.) if that helps, or just keep the discussion going.
I get the impression both of you are arguing essentially the Humans' side in Three Worlds Collide, which is just outright insane to me. You are defending pain and boredom and challenge! Me, I'm SuperHappy all the way. I find the Baby Eaters easier to sympathize with than the Humans!
muflax wants to move to a holodeck
Nit-picking: I'm not fully endorsing a holodeck. In a laboratory metaphor, I want the cheese. Whether the best way to get there is through the labyrinth (your way) or around it (the holodeck) doesn't matter to me. I suspect a bypass is easier, but I'm not sure. I'm qualifying my position so I don't get forced into a more fanatic position through argument than the evidence warrants. :)
I'm defending boredom and challenge, undecided on pain so far. MrMind wants to retain a configuration that has those abilities but didn't say these abilities mattered in themselves, though that's a likely implication.
Well, obviously you'd rather fall in love with a real person than with wallpaper if the wallpaper is more expensive. That ain't likely; a system to write arbitrary input to your brain is easier than the galaxy-spanning civilizations I want.
Nitpicking: I was not stating bluntly my conclusion, but my preferences.
Anyway, a three way three m post seems ok, if muflax also agree to move the discussion there. May I PM the draft to you? I don't think there's a way to coauthor a post.
If so, then that's a pretty straightforward point that any FAI would respect
I'm merely suspicious of that, since no one still know what an FAI really looks like. You seem to tend to collate an FAI with agreeable, pleasant or fun super-singleton, but these are your heuristics. It's obvious that a friendly AI will respect my wishes, but it's not obvious that an AI possibly developed by the people in this forum will.
(Even though I tend to think it's mistaken, or at least, awfully inefficient.)
And that's exactly the reason why I made the post.
I'm merely suspicious of that, since no one still know what an FAI really looks like. You seem to tend to collate an FAI with agreeable, pleasant or fun super-singleton, but these are your heuristics. It's obvious that a friendly AI will respect my wishes, but it's not obvious that an AI possibly developed by the people in this forum will.
A FAI will respect your preferences (as far as it can). That's the whole point of Friendliness. Of course, an UFAI might just ignore you. But then, asking it to spare you probably won't help you, either. It would be an awful coincidence that we got an AI that will try to force people into orgasmium, but allow them to opt-out.
That was not my point. I was not appealing to some abstract "friendly AI in the sky" as a new kind of rationalist god.
My post was meant to address the people in the forum, a sort of warning that shouted: "Hey, lesswrongians! I saw you are in love with complexity and so on, but don't forget that people might have radically different preferences. Indeed, these are mine. Don't forget to include them in a friendly AI, if you happen to build one".
Baseline: I think the fun theory that circulates the most in this forum, as per Yudkowsky sequence, is dangerously narrow. That could have potentially catastrophic consequences, and I wanted to prevent that.
The most commonly discussed FAI won't have any (human) terminal values built in (except maybe for bootstrapping), but will examine humans to get a complete understanding of what humans actually value and then optimize that.
Therefore, whatever we humans now think about what might be fun or not is fairly irrelevant for the FAI simply because we don't know what our values are. (That is also why I referenced the confusion between terminal and instrumental values in my first comment. I think that pretty much everyone talking about their "values" is only talking about instrumental values and that properly reduced terminal values are much, much simpler.)
Then I'm outright confused. Cooler existences likely contain very few of your listed pleasures, since they're not the most intense or the most complex or the most social, and when they do, they're not likely to be linked to the corresponding activities (sexual pleasure could be linked to conversation, taste could be linked to ideas). So you object to being cooler. Why?
I do not object that. I object that an eutopia is made solely of cooler existence, and I made formal request that I'm left the option to choose the uncooler one. That is not trivial, depending on the fun theory you use.
Would you object if (on top of making you cooler) we tied your responses to other situations? Say, looking up a simple fact would feel like scratching an itch, exchanging ideas would feel like sex, whatever environment you're most often in would trigger your "this is home and normal and safe" response, manipulating fleets of nanobots would feel like carving, etc.
Yes, I would object to that too. To put it differently, I would object to everything that violates the integrity of the present correlation between my brain states and the external world (that includes the fact that my body is actually the mediator).
Been thinking more about your stated preferences. To untangle them more:
Do you insist on keeping simple pleasures because you fear you might lose them if cooler ones were available, i.e., increasing happiness set levels would eventually lower your total fun?
I once had a discussion with my mother in which I was optimistic about the increasing influence of machines and software in our lives and that in a near future we might spend most of our leisure uploaded in advanced MMOs. She objected that she valued gardening and similar menial tasks and that she did not like this direction. What I found out after some probing was that she objected to the aesthetic. "Machines" brought up fictional evidence of a very sterile future with heroin-addict-like wireheads being hooked up to computers.
So what I'm getting at, might that be a factor? Do you have a specific vision of what a complex future of the kind you are opposing might look like?
If I understand you right, you think that increasing fun leads to either wireheading and/or increasing complexity. (I generally agree.) I see how you can have a problem with wireheading. What's your issue with complexity? Why would you want to not want to indulge in complex fun with the consequence of less interest in simple fun along the way?
In other words, wouldn't you always want to have as much fun as possible? (With "fun" in the fun theory sense that includes all your terminal values, not just pleasure.) It seems to me like this should be true for any properly functioning agent, although agents might disagree on what fun is. But you seem to agree that you would actually enjoy more complex fun more.
Do you insist on keeping simple pleasures because you fear you might lose them if cooler ones were available, i.e., increasing happiness set levels would eventually lower your total fun?
No, my opinion is that a cooler existence would make them meaningless. It's not a question of fun or happiness: in a eutopia those are cheap commodities. It is more a question of identity and futility.
So what I'm getting at, might that be a factor? Do you have a specific vision of what a complex future of the kind you are opposing might look like?
I feel it's important to say that I'm not opposing a future like that. I like AIs and robots and think that we need more of that in our lives. What I'm opposing is that for some people, it's unnecessary and unwanted to increase the complexity of existence per se. I simply don't value terminally complexity, so for me an existence which is built on that is simply an existence which I don't prefer.
In other words, wouldn't you always want to have as much fun as possible? (With "fun" in the fun theory sense that includes all your terminal values, not just pleasure.) It seems to me like this should be true for any properly functioning agent, although agents might disagree on what fun is.
That, in essence, is the central 'dogma' of your theory of fun. I'm telling you however that for some people (me, for example), that is just not true. I just don't want to have more and more fun, it strikes me as meaningless and 'childish' (that is not an exact description. I would need to dig deeper into the precise feeling). I would like to add to your theory of fun that there are agents who, once a certain level of fun/happiness is reached, they just need no more and can continue happily forever in that state of mind.
Thanks, now I understand you much better.
I can understand "maxing out" fun. I even suspect that my ability to experience fun is bounded and that even without post-singularity tech I might maximize it. I wonder, what happens then? Once all your values are fulfilled (and sustainability is not an issue), what do you do?
(Obviously, self-modify to not get bored and enjoy the ride, says the wirehead. I'm not so sure about that anymore.)
Why would you ever choose the uncool option?! I don't intend that to be a rude response, I just can't wrap my head around it. Do you just feel that way? How do you know? Have you thought about that you might be confused, or might have some kind of status-quo bias?
(Not that I have a problem with your choice. I think a utopia should totally allow people to do things I find stupid, as long as it doesn't cause any other harm.)
Have you thought about that you might be confused, or might have some kind of status-quo bias?
Even though it's too strong to say that I would never choose the cooler existence, for what I understand of my present preferences, I'm not tempted at all about living in a simulation. By saying that I mean:
(Not that I have a problem with your choice. I think a utopia should totally allow people to do things I find stupid, as long as it doesn't cause any other harm.)
That is very cool from you: I promise that if I were to code a friendly AI, I will let it have people upload in a cooler existence, as long as they don't cause any harm to the 'meatballs' ;)
The point was not only that I value the pleasure coming from those activities, but that I value those activities as well.
This is what I was trying to convey, that you aren't confusing terminal with instrumental - you just have vastly different weights on certain poo related terminal goals. Which is a coherent state of preferences to have.
Note, however, that rejecting the non-dystopic necessity of faeces disposal pleasure does not lead us in the direction of orgasmium. Like MixedNuts obsrved there are many things that I value that are nothing to do with pleasure. Orgasms aren't even at the top of the list.
Yes, of course. I was specifically refering to the way muflax put it, see my reply to MixedNuts. It's amusing however that you anchored faeces instead of say sex or social laughter :)
It's amusing however that you anchored faeces instead of say sex or social laughter :)
I chose the terminal value that I do not share. If I thought 'lack of sex' was a lame thing to declare dystopic then I would have focussed on that. As it happens, though, I like sex a lot more than poo.
I want my future self to share my values. It is the content of some of your declared terminal values in particular and the degree with which you declare them vital which I don't share. In fact sex and social laughter are perhaps the only two items that don't look like they are deliberately selected to be limiting.
In fact sex and social laughter are perhaps the only two items that don't look like they are deliberately selected to be limiting.
I'm curious about that. In what eutopian sense sex is better than poo?
I'm curious about that. In what eutopian sense sex is better than poo?
Why are you confused about that? You wrote a whole post on your arbitrary preferences! I arbitrarily prefer one of them to another. That one of them is sex and one of them bowel movement should make the causal factors behind my preferences fairly unsurprising.
Why are you confused about that?
Because in the sentence that I quoted a comment above I thought I recognized that you were making a more general statement than uttering a personal preference. You attached the qualifier 'limiting' to my other preferences, so I thought there was a general, independent-from-you side that you were pointing at. Was I wrong?
a general, independent-from-you side that you were pointing at.
Not independent from arbitrary subjective preferences. Although there are certain trends and associations one can observe in the preferences of others. One could reasonably imagine something of a spectrum on how 'core' people consider certain traits to their identity, ranging from positive social connections on one side down to digestive details on the other. You present rather a lot from lower down on that spectrum. Things that many would not consider the lack of to constitute a dystopia.
Forgive me if I appear dense to you, because I'm more and more confused. You write about subjective preferences and then about a continuum with positive social connection on one side and digestive detail to the other. May I ask you of what parameter this is a continuum of? Maybe the "what would you loose first if you were forced to?"-asked-to-core-people parameter?
May I ask you of what parameter this is a continuum of? Maybe the "what would you loose first if you were forced to?"-asked-to-core-people parameter?
Approximately. Although replace 'core' with 'most'.
Let me recap: given the possibility (not just the necessity) you would bail out of having to have to expel the waste produced when ingesting food. That would stop there? Having the possibility of dispose of other bodily function, would you? And having the possibility of having other functions enhanced, would you?
I'm not asking this because I don't believe you, I'm asking because you might present an important point in the data-set of the model I'm trying to build
Let me recap: given the possibility (not just the necessity) you would bail out of having to have to expel the waste produced when ingesting food
Yes.
That would stop there?
Conceivably but unlikely.
Having the possibility of dispose of other bodily function, would you?
I'm indifferent to breathing. I don't much care about toenail growth. I certainly see no need for snot if any health function is rendered obsolete.
And having the possibility of having other functions enhanced, would you?
Yes. But note that this is not an enhancement in the direction of orgasmium, pleasure maximisation, happiness maximisation or any other one dimensional function. It is in the direction of whatever my arbitrary preferences happen to be. Right now I haven't given enough thought to details and don't pretend to have the self awareness to give a precise prediction of what my preferences would resolve to.
I'm not asking this because I don't believe you, I'm asking because you might present an important point in the data-set of the model I'm trying to build
You seem to be confusing terminal and instrumental values.
It would not be a stretch to say he just has parochial terminal values. I think the gets the difference but either has weird preferences or has failed at introspection.
Plenty of people enjoy those things, so CEV will most likely preserve them if it's implemented.
Plenty of people enjoy those things, so CEV will most likely preserve them if it's implemented.
Just so long as it doesn't force them on me. I don't care if MrMind likes emptying his bowels but I for one don't want to be relying on bowels at all. In my judgement Mr is trying to hang on to the past just a little too strongly - and he can keep his shit habit for himself so long as I don't get stuck with it.
I agree with you completely. Hopefully, CEV will be at least as unlikely to compel you and I to have bowels as it will be to compel MrMind not to have any.
Well, the item directly above that was to enjoy "food/drinking" and stuff's got to go somewhere afterwards...
[Barring entirely virtual food, of course]
[Barring entirely virtual food, of course]
Virtual food would obviously be possible in a post-FAI world, and I don't see any reason why most people would bother with non-virtual food if they could get the same experience (probably even the apparent experience of putting physical food into one's mouth and tasting and chewing and swallowing it and feeling satisfied) for free, in unlimited quantities, and in varieties not limited by regional or seasonal availability.
Though even if we entirely replace our bodies with better-designed ones that don't need actual food as fuel, I'm sure there could be VR simulations of the experience of emptying one's bowels, if anyone really misses it by that point...
In Second Life, many people use custom tools that trigger a virtual pregnancy from cybersex. At least one brand advertises realistic probabilities of conception, including variation on a monthly cycle. This suggests that some people may want to eat food that makes them poop, even if non-poop-causing food already exists (and is even the default).
Second Life being what it is, that probably has a lot more to do with proclivities unrelated to any of pregnancy's usual associations than with a general desire to keep meat-phase life's default inconveniences around. I'd expect the same for virtual defecation, and for most similar features.
It appears from the discussion so far that there are two points of stability in the fun-space of the mind. If you disregard natural wireheading and their sources, and your preferences goes to pleasure per se, then there's no reason not to be driven toward an entity powered by a single source of extreme pleasure and increasingly complex challenges (if you don't want to collapse into orgasmium, that is). That shouldn't be too hard to formalize
This post is a formal request for everybody in this forum, who are the most likely humans to produce an FAI in the future, that in the possible resulting utopia some activity will still be available, even if they are not based on challenges of increasing complexity. I call these activity natural wireheadings, and since it is my right to determine my own fun-generating activities, I hereby formally request that some simple pleasure, listened below, will still be available at *any* point in my future cone, and that I will consider a dystopia any future which deprives me of these natural wireheadings (if anyone is still around caring for those things).
A (non-exhaustive) list of them includes the following:
- sex
- eating food/drinking
- feeling relieved for having emptied my bowels
- dancing
- the pleasure of physical activity (wood-carving, sculpting, running, etc)
- the rapture I feel when in the presence of a safe ancestral environment
- social laughter
- the pleasure of talking in a small, same-minded crowd
- listening to pop/metal/rap music
- the pleasure of resting when tired
- scratching an itch
-...
More will come!
