Overall, it sounds to me like people are confusing their feelings about (predicted) states of the world with caring about states directly.
But aren't you just setting up a system that values states of the world based on the feelings they contain? How does that make any more sense?
You're arguing as though neurological reward maximization is the obvious goal to fall back to if other goals aren't specified coherently. But people have filled in that blank with all sorts of things. "Nothing matters, so let's do X" goes in all sorts of zany directions.
Is this just a case of the utility function not being up for grabs? muflax can't explain to me why wireheading counts as a win, and I can't explain to muflax why wireheading doesn't count as a win for me. At least, not using the language of rationality.
It might be interesting to get a neurological or evo-psych explanation for why non-wireheaders exist. But I don't think this is what's being asked here.
Think about a paper-clip maximiser (people tend get silly about morality, and a lot less silly about paper-clips so its a useful thought experiment for meta-ethics in general). Its a simple design, it lists all the courses of action it could take, computes the expected_paper-clips given each one using its model of the world, and then takes the one that gives the largest result. It isn't interested in the question of why paper-clips are valuable, it just produces them.
So, does it value paper-clips, or does it just value expected paper-clips?
Consider how it reacts to the option "update your current model of the world to set Expected paper-clips = BB(1000)". This will appear on its list of possible actions, so what is its value?
(expected paperclips | "update your current model of the world to set Expected paper-clips = BB(1000)")
The answer is a lot less than BB(1000). Its current model of the world states that updating its model does not actually change reality (except insofar as the model is part of reality). Thus it does not predict that this action will result in the creation of any new paper-clips, so its expected paper-clips is roughly equal to the number of p...
Let's use the example of the Much Better Life Simulator from the post of a similar name, which is less repellent than a case of pure orgasmium. My objections to it are these:
1: Involves memory loss. (Trivially fixable without changing the basic thought experiment; it was originally introduced to avoid marring the pleasure but I think I'm wired strangely with regard to information's effect on my mood.)
2: Machine does not allow interaction with other real people. (Less-trivially fixable, but still very fixable. Networked MBLSes would do the trick, and/or ones with input devices to let outsiders communicate with folks who were in them.
If these objections were repaired and there were no "gotcha" side effects I haven't thought of, I would enter an MBLS with only negligible misgivings, which are not endorsed and would be well within my ability to dismiss.
Let's consider another case: suppose my neurochemistry were altered so I just had a really high happiness set point, and under ordinary circumstances was generally pleased as punch (but had comparable emotional range to what I have now, and reacted in isomorphic ways to events, so I could dip low when unpleasant things happ...
Why does my intuition reject wireheading? Well, I think it has something to do with the promotion of instrumental values to terminal values.
Some pleasures I value for themselves (terminal) - the taste of good food, for example. As it happens, I agree with you that there is no true justification for rejecting wireheading for these kinds of pleasures. The semblance of pleasure is pleasure.
But some things are valued because they provide me with the capability, ability, or power (instrumental) to do what I want to do, including experiencing those terminal pleasures. Examples are money, knowledge, physical beauty, athletic abilities, and interpersonal skills.
Evolution has programmed me to derive pleasure from the acquisition and maintenance of these instruments of power. So, a really thorough wireheading installation would make me delight in my knowledge, beauty, charisma, and athleticism - even if I don't actually possess those attributes. And therein lies the problem. The semblance of power is not power.
What I am saying is that my intuitive rejection of wireheading arises because at least some of the pleasures that it delivers are a lie, a delusion. And I'm pretty sure that ...
Because writing big numbers on the speedometer with a sharpie doesn't get me to the destination sooner.
What is this "valuing"? How do you know that something is a "value", terminal or not?
Are you looking for a definition? Specifically coming up with a dictionary definition for the word "value" doesn't seem like it would be very instrumental to this discussion. But really, I think just about everyone has a pretty good sense for what we're talking about when we post the symbols "v", "a", "l", "u", and "e" on less wrong for us to simply discuss the concept of value without tryin...
Because wireheading is death.
Beyond the definitions, a person walks into a room, something happens, they never walk out again, nor is the outside world impacted, nor is anything changed by them. They might as well walk into the wireheading room and have their brains dashed upon the floor. Their body may be breathing, but they are dead just the same.
If the wireheading were un-doable, then it would be nothing more than suspended animation. Pleasurable, but it's still a machine you plug into then do nothing until you unplug. Frankly, I haven't the years...
How familiar are you with expected utility maximizers? Do you know about the difference between motivation and reward (or "wanting" and "liking") in the brain?
We can model "wanting" as a motivational thing - that is, if there was an agent that knew itself perfectly (unlike humans), it could predict in advance what it would do, and this prediction would be what it wanted to do. If we model humans as similar to this self-knowing agent, then "wanting" is basically "what we would do in a hypothetical situation.&qu...
Apparently, most of us here are not interested in wireheading. The short version of mulfax's question is: Are we wrong?
My answer is simple: No, I am not wrong, thanks for asking. But let me try to rephrase the question in a way that makes it more relevant for me:
Would we change our mind about wireheading after we fully integrated all the relevant information about neuroscience, psychology, morality, and the possible courses of action for humanity? Or to paraphrase Eliezer, would we choose wireheading if we knew more, thought faster, were more the people we...
To clarify, are you claiming that wireheading is actually a good thing for everyone, and we're just confused? Or do you merely think wireheading feels like a fine idea for you but others may have different values? At times, your post feels like the former, but that seems too bizarre to be true.
My own view is that people probably have different values on this issue. Just as suicide seems like a good idea to some people, while most people are horrified by the idea of committing suicide, we can genuinely disagree and have different values and make different decisions based on our life circumstances.
Can someone give a reason why wireheading would be bad?
Well, we don't want our machines to wirehead themselves. If they do, they are less likely to be interested in doing what we tell them to - which would mean that they are then less use to us.
It is possible to enjoy doing something while wanting to stop or vice versa, do something without enjoying it while wanting to continue. (Seriously? I can't remember ever doing either.
You should try nicotine-addiction to understand this. That's possible, because "reward" and "pleasure" are different circuits in the brain.
John Wesley said, "earn all you can; save all you can; give all you can." He was serious.
What does that have to do with wireheading? As far as I can tell, that quote resonates with me on a deep level, though I replace "give" with something like "optimize" or "control." And so when presented a choice between pleasure and control, I choose control. (If actually presented the choice, I calculate the tradeoff and decide if the control is worth it.) So, even though orgasmium!Vaniver would be more satisfied with itself, cu...
I know people who specifically said that if orgasmium were available, they'd take it in an instant. I also know people that would not. Wireheading doesn't have to be univerally, objectively "good" or "bad." If wireheading would satisfy all your values, and it becomes available to you, well, go for it.
I know that if I was given access to orgasmium, I'd probably be content living on it for the rest of my life. That doesn't change the fact that BEFORE having access to orgasmium, I simply prefer not to accept it, and instead create art and ...
You seem to classify each argument against wireheading as a bias: since the argument doesn't persuade you, the ones who are persuaded must make some error in judgement. But those arguments aren't (all) meant to make people value "reality" more than pleasure. Most of them aim at people who already do prefer real achievements over pleasure (whatever it means) but are confused about the possibility of wireheading. In particular,
My objection is precisely the opposite of what some have said here.
You are affecting the outside world, but only negatively - using resources that could be put to more important things than one person blissing out, without creating anything for anyone else. I therefore see wireheading just you as an unethical choice on your part.
I am not sure if I have an objection to "wirehead everyone sustainably forever", if that were ever practical.
Edited to clarify very slighty:
I do have some revulsion at the thought but I have no idea what it would be grounded in, if anything.
I believe that humans have natural psychological defenses against the lure of wireheading, because the appeal is something we navigate on a daily basis in our every day lives. In my case, I know I would really enjoy entertaining myself all the time (watching movies, eating good food, reading books) but eventually I would run out of money or feel guilty I'm not accomplishing anything.
Even if you tell people there will be no long-term consequences to wire-heading, they don't believe you. It's a matter of good character, actually, to be resistant to wanting t...
What ever makes you happy! - Wait a minute. . .
No but seriously as been pointed out, one problem with wireheading is that it tend to resemble death in the sense that you stop being a person, since you stop caring about everything you used to care about (as well as act upon) such as finding out why other people don't like the whole idea of head wiring, you are just in a kind of bliss-stasis. I don't see much difference between me being head-wired and me being shot in the head, then someone/something building something that is put into bliss-stasis, since ...
While there's no particular reason not to go orgasmium, there's also no particular reason to go there.
We already know that wanting and desiring are expressed by two different, although overlapping, neural circuitries, so it is natural that we may value things that doesn't directly feedback pleasure. Let's say that we find meaningful to be an active part of the world, that we assign a general utility to our actions as they effect the environment: descending into an orgasmium state would kill this effectiveness.
The general argument is that orgasmium can be c...
There are many situations over which I would prefer a wirehead state. For example I would prefer to be orgasmium than paperclips. But it isn't exactly inspiring. I like novelty.
This is arbitrary.
Our true self is the far one, not the near one. (Why? The opposite is equally plausible. Or the falsehood of the near/far model in general.
Because any decision to self-modify will more likley than not be determined more by my future self.
I see wireheading as another problem that is the result of utility maximization. The question is, can utility be objectively grounded for an agent? If that is possible, wireheading might be objectively rational for a human utility maximizer.
Consider what it would mean if utility was ultimately measured in some unit of bodily sensations. We do what we do for what it does with us (our body (brain)). We do what we do because it makes us feel good, and bad if we don't do it. It would be rational to fake the realization of our goals, to receive the good feeling...
You are right, muflax. There just isn't any explanation for badness of the "wireheading".
The case is the following:
Overall, it sounds to me like people are confusing their feelings about (predicted) states of the world with caring about states directly.
Did you read this?
Neurologically, wanting and liking are two separate albeit very related things. When you measure liking and wanting, you find that you can manipulate the two separately.
If your dopamine receptors are blocked, you don't want things as badly, even though your enjoy them equally well. If you increase dopamine, you (or experimental rats) work harder for something, even though you don't enjoy it more when you get it.
I have the subjective impression that when I'm happy for no particularly external reason, I still want to do things that I previo...
I've been thinking about torture and the nature of my values. Many people here have defended the importance of external referents or complex desires. My problem is, I can't understand these claims at all.
To clarify, I mean torture in the strict "collapsing into painium" sense. A successful implementation would identify all the punishment circuitry and directly stimulate it, or do something equivalent. It would essentially be a vastly improved box jellyfish. A good argument for either keeping complex values (e.g. by requiring at least a personal m...
I've been thinking about wireheading and the nature of my values. Many people here have defended the importance of external referents or complex desires. My problem is, I can't understand these claims at all.
To clarify, I mean wireheading in the strict "collapsing into orgasmium" sense. A successful implementation would identify all the reward circuitry and directly stimulate it, or do something equivalent. It would essentially be a vastly improved heroin. A good argument for either keeping complex values (e.g. by requiring at least a personal matrix) or external referents (e.g. by showing that a simulation can never suffice) would work for me.
Also, I use "reward" as short-hand for any enjoyable feeling, as "pleasure" tends to be used for a specific one of them, among bliss, excitement and so on, and "it's not about feeling X, but X and Y" is still wireheading after all.
I tried collecting all related arguments I could find. (Roughly sorted from weak to very weak, as I understand them, plus link to example instances. I also searched any literature/other sites I could think of, but didn't find other (not blatantly incoherent) arguments.)
(There have also been technical arguments against specific implementations of wireheading. I'm not concerned with those, as long as they don't show impossibility.)
Overall, none of this sounds remotely plausible to me. Most of it is outright question-begging or relies on intuition pumps that don't even work for me.
It confuses me that others might be convinced by arguments of this sort, so it seems likely that I have a fundamental misunderstanding or there are implicit assumptions I don't see. I fear that I have a large inferential gap here, so please be explicit and assume I'm a Martian. I genuinely feel like Gamma in A Much Better Life.
To me, all this talk about "valueing something" sounds like someone talking about "feeling the presence of the Holy Ghost". I don't mean this in a derogatory way, but the pattern "sense something funny, therefore some very specific and otherwise unsupported claim" matches. How do you know it's not just, you know, indigestion?
What is this "valuing"? How do you know that something is a "value", terminal or not? How do you know what it's about? How would you know if you were mistaken? What about unconscious hypocrisy or confabulation? Where do these "values" come from (i.e. what process creates them)? Overall, it sounds to me like people are confusing their feelings about (predicted) states of the world with caring about states directly.
To me, it seems like it's all about anticipating and achieving rewards (and avoiding punishments, but for the sake of the wireheading argument, it's equivalent). I make predicitions about what actions will trigger rewards (or instrumentally help me pursue those actions) and then engage in them. If my prediction was wrong, I drop the activity and try something else. If I "wanted" something, but getting it didn't trigger a rewarding feeling, I wouldn't take that as evidence that I "value" the activity for its own sake. I'd assume I suck at predicting or was ripped off.
Can someone give a reason why wireheading would be bad?