NB: This is a follow-up to my previous essay The Criminal Stupidity of Intelligent People, as cross-posted to my blog: http://fare.livejournal.com/168562.html.
here is a detailed rational refutation of
the utilitarian concept of (Global) "Happiness Maximization" in general,
and its specific variant that concludes in favor of
the cultivation of individuals in vats
in a permanent state of artificially stimulated bliss.
To recall the background, I recently had a discussion with
a very intelligent and extremely well-meaning colleague of mine,
who, as an aside during an evening he organized
to foster efficient philanthropy,
was seriously defending the utilitarian ideal
of "Happiness Maximization" to its wantonly absurd consequence,
putting humans, by force if needs be, into vats,
where the pleasure centers of their brains
will be constantly stimulated for lifelong bliss
through properly implanted electric wires and/or chemical injections.
Or perhaps instead of humans, the Utilitarian Elite would use
rats, or ants, or some brain cell cultures
or perhaps nano-electronic simulations of such electro-chemical stimulations;
in the latter cases, biological humans,
being less-efficient forms of happiness substrate,
would be done away with or at least not renewed
as embodiments of the Holy Happiness to be maximized.
He even wrote at least two blog posts on this theme:
hedonic vs preference utilitarianism in the Context of Wireheading,
Value of a Computational Process.
Now, in my previous post,
The Criminal Stupidity of Intelligent People,
I gave several blanket principles by which one might reject
such mass-criminal schemes.
Indeed, there is no question that Jeff is much more intelligent than most
(I would notably wager that
he's more intelligent than I, as measurable by an IQ test),
and much more knowledgeable of this topic than anyone;
few could oppose him a rational refutation like I'm going to offer
(no false modesty from me here);
yet one need not have identified the precise mistakes of a demonstration
to reject its provably absurd conclusions.
Just like you can without looking reject elaborate alleged proofs
of the squaring of the circle, you can without looking reject proofs
that some higher principle calls for mass murder:
that's the Libertarian principle,
recognizing that peace is the first social value,
and it depends on mutual respect for other individuals' life and property.
You can also reject any denial of intuition, tradition or common sense
that does not first explain why intuition, tradition and common sense work
in most usual cases yet not the case that is being considered.
That's the Conservative principle.
Finally, you shouldn't yield to the opinion
of any self-styled or even established expert,
however intelligent and authoritative they may be,
before you've pitted the proponents of that opinion against
the most intelligent and authoritative opponents of that opinion you can find
(who are seldom those that the proponents would choose to debate).
That's the Eristic debate principle.
And that's where I come:
to champion the debunking of "Global Happiness Maximization" in general,
and "Wireheading" in particular.
Let's start by questioning the very concept of "happiness" or "utility".
Certainly, it is sometimes useful when trying to make a decision
to measure opportunities and their consequences
in terms of ordinal preferences, and even of some scalar utility.
But that doesn't mean that there is an objective way
to always precisely measure everything in such terms:
the notion is fuzzy and uncertain;
ascribing measures is costly and imprecise,
totally subjective, and relative to a narrow choice at hand.
Even under the best hypotheses (von Neumann Morgenstern utility),
it is only defined up to a costly measure and an affine transformation
— and what more every measure
modifies the system being observed.
It certainly isn't a well-defined number easily accessible through introspection,
even less so through external observation.
Here's for one individual's "utility".
Given individual (cardinal) utilities
or even just individual (ordinal) preference scales,
the only objective notion of interpersonal optimization is Pareto-optimality:
to respect each person's property and let them interact
through voluntary exchanges and consensual transactions.
Any enhancement that respects everyone's rights and interests is legit
and improves each and everyone's utility.
This is the Libertarian principle of progress through peaceful commerce.
Anything else that tries to go "beyond" this Pareto-optimization
necessarily violates some individual's rights,
and obviously doesn't improve each and everyone's utility.
Yet that's what collectivist "utilitarians" claim to do.
Now, depending on their ambition and their gall,
collectivist utilitarians may claim to be able
to establish a common utility scale
not just over a small group of people for a day,
but over an entire country for a year,
over humanity for centuries,
over all living organisms on the planet for millions of years,
and even over all entities in the entire universe for aeons to come.
But what common scale can you establish between
a few humans who hardly know each other,
between billions of humans who live completely different lives
and have no clue about each other's existence,
between humans and rats or other mammals,
between mammals and insects or microbes,
between animals and plants,
between biological life forms and electronically simulated life forms,
between life forms as we know them and
potential future emergent artificial intelligences,
between terrestrial life as we may know it
and life that may have evolved in completely different ways
in other parts of the universe?
How can you establish a common scale between entities
that live at different places in different times,
separated by vast distances and durations?
How can you establish a common scale between
actual entities that are known to exist,
entities that exist but are not known to exist,
and infinitely many potential entities that
may or may not some day come into being or fail to come into being
depending on a large number of unpredictable decisions and unknowable events
and their largely unforeseeable consequences?
Yet this is the kind of feat that collectivist utilitarians such as Jeff
not only assume to be possible, but claim can be the basis
for making effective decisions that are to bind each and every one.
Of course, if you take the building of such scales seriously,
you'll find that depending on the fine-tuning
of the contributing factors to your scale,
you might have to consider such dilemmas as follow:
Should we forcefully impregnate women,
inconveniencing them somewhat for the sake of
the large number of potential unborn happy descendants years from now?
Is the fraction of life value associated to ebola virus big enough
that we should sacrifice each and every human on the planet
for the sake of feeding their bodies to a tremendous number of copies
of these valuable life-forms?
Should our omniscient economic advisers assign to everyone
a positive or negative tax (or subsidy) factor on everything he purchases,
perhaps with a different rate on each potential purchase?
Or should they directly buy what each of us needs
according to their perfect knowledge of our utility?
If we count "each man equally" in an open system such as "a country",
shouldn't we encourage prolific invaders
to replace the inhabitants of the country
after either slaughtering them or letting them die
without children or with few children?
Depending on how you compute GNP (or GNH) statistics, you may want
to import richer and happier invaders (so they bring their capital),
or poorer and unhappier invaders (so their portion of GNx increases most).
If you slaughter the original inhabitants, though, you may also want to
excommunicate or exterritoriate them before you do them in,
so that their suffering does not count negatively for "the country",
before you re-import the loot from their dead bodies.
Or maybe GNP and GNH studies are only valid for "closed" systems,
and then require some World or Universe Government to oversee everyone,
and strictly regulate and control the procreation of new children to maintain a fair approximation of closeness between the initial inhabitants;
the only totally fair non-approximate way to keep the system closed
would be to strictly forbid children;
but if there are no children at all, "in the long run, we're all dead".
Maybe to be meaningful, the National Utility Scale
can only be used by a country's government at a given date;
then to continue action it will have to morph into
some kind of government of the descendants of the inhabitants
of the country at said date, the authority of which is somehow
prorated by their contribution to the subject's genetic material.
Plenty of questions such as these, absurd as they may sound,
naturally arise as
prerequisites to any further action,
when one claims to put Everyone and Everything
on a same Utility Scale with an explicit formula.
The implicit premise of any discussion about Ethics
is that there are choices to be made that haven't been made,
and that the future is therefore unknown, and essentially so.
But the implicit premise behind Collectivist Utilitarianism
is that there is a knowable objective scale that allows decisions
to be made mechanically for everyone.
Of course, if the claim is that
"there is a scale, but it's unknowable and we can't use it",
then it's not much of a claim, but is more akin to
invisible pink unicorns and mystical gods retired from the world.
Now if the claim is "we don't know but we can act as if we knew",
then it's just a justification for the arbitrary power and dreadful decisions
of those entitled to act as if they knew even though they admittedly don't.
In any case, the pretense to sum over unknowable variables is bogus.
Inasmuch as it might possibly mean anything,
the knowledge of it is out of the reach
not just of mere humans but of any finite entity within the universe.
The entire project of a global utility calculation
is found to be meaningless.
In the end, any concept of utility, happiness or suffering
only has a clear or knowable validity but in the context of
an individual who makes decisions based on such a concept.
Those evaluations that are valid for one individual
are both invalid and unknowable for other individuals.
These evaluations do not meaningfully generalize to "collectives",
much less to a whole planet, even less across years,
and not at all to the entire Universe across future aeons.
Furthermore, a damning feature of any recommendation issued
in the name of any collective utility is that it presumes
totalitarian ownership by the few individuals
who may or may not decide based on such recommendations
over all the resources of all individuals in the alleged collective
that will be somehow shuffled by the decision.
Wherever such recommendations depart from those of Pareto Optimality,
they require violence toward some parties;
such violence implies that the party who imposes the decision
either can exercise some tyrannical supremacy,
or will experience abject failure in trying to enforce its recommendation.
In the first case, this violent tyranny
more than offsets the alleged positive consequences of the recommendation;
in the second, its failure invalidates the unachieved positive consequences
that allegedly justified it.
In either case, the denial of Pareto Optimality and
Libertarian property rights as a limit to action
by some proclaimed Utilitarian Elite
ignores the costs of enforcement of such action,
that offset any benefits, as showed by the
Law of Bitur-Camember.
Although really, it's usually the elite being
spiteful rather than ignorant of how victims are sacrificed
to fulfill their own personal preferences, plans, and lust for power;
and the same elites cultivating
ignorance, misunderstanding, apathy and disorganization
among the masses upon which they trample.
There remains the case where some individual
only acts on his own property
without initiating conflict with any other individuals,
while respecting every one else's individual property rights,
but claims to act in the name and interest of a Collective
for which he has established some Collective Utility Scale.
We can then start by examining whether the actions of this individual
is indeed compatible with said avowed Collective Utility Scale,
or if he are just lying to himself or to others;
but if he is indeed sincere and coherent, which he may be,
that only means that he has some possibly strange
but extended notion of himself.
Which might not be so strange when the Collective is actually
a tight knit family, clan, etc.
Everyone living in society has some notion of one's extended self.
But what is strange and probably mystically deluded is when one claims
to act in the interest of the entirety of Mankind,
when one in the end is only actually possibly helping but
a small number of very specific individuals out of billions of humans;
and this delusion becomes blatant evil
when under the claim of acting according to some Objective utility scale,
one systematically acts to give to people in proportion to
the objectively awful criteria
that these recipients should be the worst kind of lowlifes,
the dumbest, laziest, nastiest and
least shameful at demanding money "due" to them.
Even assuming the latter pitfall is avoided,
the implicit recognition of the preeminence
of the Universal Law of Libertarian Non-Aggression by those who respect it
casts a shadow on any attempt to "Maximize Utility"
that explicitly ignores or rejects this preeminence in its recommendations:
however you account for Mankind as a whole, you'll find that
violations of Universal Law are the one greatest source of dis-Utility
in the world, and that as long as such violations casually occur,
the greatest possible charitable act may be to act towards
the universal recognition and enforcement of this Universal Law.
This seals the notion of a Global Happiness to calculate and maximize
for what it is:
at best, a long diversion from Pareto Optimality back to Pareto Optimality;
usually, a useless unknowable, unactionable fantasy that only serves as
the cover for some people to grab the power to decide over others;
at worst, a criminal absurdity.
There is only one knowable and actionable way to foster Happiness,
which is to care for those you love and
to uphold the Law by defending victims from their aggressors.
Gross National Product, Gross National Happiness, etc.,
are but pretenses for bureaucrats and politicians and through them lobbyists
to steal from confused citizens.
And the particular variant of Global Happiness Maximization
that recommends Wireheading is but a criminal absurdity.
Nevertheless, there are often useful concepts to clarify
in examining the crazy fallacies of intelligent people;
indeed the same or some other intelligent people
may use these fallacies in other contexts and theories as well;
therefore let's examine the notion still.
The implicit hypothesis behind Wireheading is that "happiness" or "utility"
consists in an independent electro-chemico-mechanical phenomenon
that can be physically isolated within a human's brain or an animal's brain;
for instance, Jeff seems to equate it with some pleasure center of the brain
releasing or receiving dopamine.
Once isolated, this "happiness" can be replicated,
miniaturized, and mass-produced.
Therefore, this whole Wireheading is just another variant of the infamous
that anyone in the field is warned about:
an imagined runaway AI is tasked with producing paperclips;
it ascribes a positive utility to the sheer number of paperclips in the universe,
and is so successful at maximizing this utility that it turns everything
in particular this includes taking apart the bodies of humans and posthumans
to use them as raw material for its goal.
Wireheading is similar, except that paperclips have been replaced
by wireheads or by whichever streamlined reduction thereof (if any)
our pan-philanthropic utilitarian wireheading overlord comes up with.
Behind this wireheading proposal,
we have therefore identified a pixie dust theory of happiness.
A pixie dust theory is one where
some phenomenon (in this case happiness),
is believed to have its source in some kind of magical compound
(in this case, a brain in blissful stimulation,
or its isolated reduction to some unstable dopamine-receptor chemical complex),
each uniform physical or ethereal amount of which
(in this case, physical copies)
possesses value in itself that it additively contributes to the whole:
the more pixie dust, the more happiness.
In a previous essay
I debunked the pixie dust theory of freedom.
The pixie dust theory of happiness can be debunked the same way.
Freedom isn't possibly embodied in
quantum wave collapse, elementary particles to be called eleutherons,
or magic souls or microsouls that angels mystically attach
to the brains of individuals to endow them with free will.
Similarly, happiness is not possibly physically embodied in
dopamine-receptor reaction complexes,
elementary particles to be called hedons,
or actual dust sprinkled by invisible pixies
on the neurons of some people to make them happy.
Instead, both are phenomena that result from
the situational interaction of an individual with the rest of universe,
(of being "thrown" into the universe, would say Heidegger;
a good reading suggestion at this point is
"Understanding Computers and Cognition" by Winograd & Flores).
More than that, they are functional sub-phenomena of life,
i.e. of an organism behaving in such a way as to sustain itself,
in a feedback loop with its environment.
Freedom is the acknowledgement that the inside of the organism
controls its behaviour much more than any other entity outside it;
happiness is the judgment regarding the organism achieving its goals,
whatever they be.
Neither freedom nor happiness makes any sense for a neuron in a vat,
where it doesn't actually control an organism thrown into the universe,
where it doesn't have meaningful goals to achieve.
Its existence, artificially sustained by an outer organism,
is a pure waste of resources and of information-processing ability.
It is interesting how Jeff, who finds his satisfaction in trying hard
to maximize his impact on the world
(and indeed speaks of giving efficiently for maximized "impact"),
doesn't realize that happiness means nothing to brains in a vat
that can have no impact whatsoever on the world.
Trying to isolate these phenomena from the rest
of an individual organism's behavior,
what more to "maximize" them afterwards, is absurd on its face
— even more so than trying to isolate magnetic monopoles
so as to thereafter maximize their intensity.
It's not even impossible: it doesn't make sense at all.
Jeff recognizes that
the substrate of consciousness, and therefore of happiness,
need not be biological, but could be electronic, be it through emulation.
In this, he is totally right:
the ability to process information and act upon it
isn't tied to any specific chemical process,
and there could therefore be living organisms, and sentient beings,
based on a substrate of electronic computations on silicon transistors
rather than on organic chemistry inside and across neurons.
But combining this common piece of wisdom
with the above pixie dust theory of happiness,
he reaches new heights in absurd statements that he cannot get out of.
Indeed, from this substrate independence of consciousness
and consequently of happiness,
he concludes that a bone and flesh wirehead is no better than
an emulated wirehead in a physical simulation of the universe,
or in a properly optimized abstraction thereof.
Once you have captured the simulation blueprints of a "maximally happy moment"
(by unit cost of replication by the wirehead maximizer),
be it a junkie at the top of his high,
a rapist ejaculating inside a helpless high-profile victim,
a mass-murdering tyrant getting a kick out of slaughtering a billion victims,
or the three combined at once,
then you can run this maximally blissful moment in a closed loop,
and replicate that loop identically on gazillions of computers,
each a paperclip to our paperclip multiplier.
Now, a basic principle of software abstraction and optimization is that
programs that have identical input, output and side-effects
are indistiguishable and that one can replace the other.
Since this simulation in a closed loop
has no input, no output and no side-effect,
then it is indistinguishable from the empty program that does nothing.
You don't even need to run the simulation, or to capture it for that matter;
doing nothing is already a valid implementation of the paperclip multiplier
to unreachable transfinite heights,
far beyond what any physical implementation thereof
could even conceivably dream of achieving.
Jeff, faced with this prospect, tries to somehow demand
an actual expensive computation of some physical process,
his original pixie dust;
somehow you have to disable optimizations and observe
the blissful brain in its dopaminergic glory.
Jeff obviously hasn't read Greg Egan's Permutation City,
in which delicious SF novel all kinds of concepts related to
the emulation of consciousness are explored, to the point of absurdity
— with each reader getting to decide where it starts to be absurd and why.
Of course, even without optimization at the level observed by Jeff,
what about optimizations and pessimizations at an intermediate level below?
What if a hedonic simulation is run on a redundant computer, where
two identical processors in lockstep simultaneously carry the same computation?
Does that make the simulation count double or single?
What if the redundancy happens in software rather than in hardware?
On the contrary, what if a de-duplicating server optimizes zillions
of identical computations by running them on one virtual machine instance?
What if after the first simulation, the results are cached,
and the computation implemented as a virtual lookup,
itself optimized away, except when you plug a monitor to observe said results?
One way or the other, "linearity" (as in "linear logic"),
the resource-like nature of some phenomena,
is not preserved by software abstraction in general.
If happiness is embodied in some pixie dust,
then software emulation is going to do strange things to it
incompatible with accounting for utilons.
Yet Jeff, a software engineer, doesn't seem to register.
Jeff falls in the most basic insanity trap,
as notably identified by Korzybski's General Semantics:
confusing a thing and its representation.
The map is not the territory;
the representation of something is not the thing itself;
a software emulation of a happy man is not a happy man;
the neural switch that triggers a feeling of happiness
is not the feeling of happiness;
the feeling of happiness in a man itself
is but a representation, an internalization,
of a situation of happiness, real or not,
that if real relates the man to his environment:
the goals he is pursuing are reached;
his frustrations are overcome;
his impact is large;
his choices are vindicated;
his actions are meaningful;
What makes the mental representation of something worthwhile is
its connection to the reality,
with the thinking individual choosing how to act within in his real environment
based on this representation in his mind.
When the representation of reality is inadequate, the mind is mistaken.
When the relation to reality is broken, the mind is insane.
A concentration camp prisoner injected
with drugs inducing a feeling of happiness is not actually happy;
his feeling of happiness is not just a mistaken representation,
what more it is a representation that has been deliberately corrupted.
When there isn't even a reality to be adequately or inadequately represented,
and not even an individual acting in that reality,
then there isn't a mind at all that may be sane or insane;
and the person who claims that there is is himself deeply mistaken,
if not insane.
What makes mental representation meaningful is that
it's actually connected to sensors and actuators,
at which point this connection is making it worthwhile, or not;
and that's precisely the opposite of Jeff's view of utilons
as an isolatable physical phenomenon.
In conclusion, here was another case of
very intelligent, well-meaning people
making criminal recommendations
based on seriously accepting absurd ideas.
It is important to recognize these ideas as absurd
and these recommendations as criminal;
but as said Claude Bernard,
it isn't enough to say: "I was mistaken";
one must say how one was mistaken.
Only by analyzing the root of the issue can we possibly
prevent more people from following these same ideas,
which, if widespread would be used to justify actual crimes.
It is important to debunk these ideas,
to confront the essential contradictions in their underlying assumptions.
And as we do it, we can explore and restore a few interesting concepts
that are often overlooked but unlike the debunked ideas
can be used for effective ethical decision making.
I think you have misunderstood the genre of some of the conversations you've been having. Wireheading is a philosophical thought experiment, not a policy proposal. Getting angry and calling it a "criminal proposal" implies a significant misunderstanding of what is being talked about and what kind of conversation is being had.
Combining this with references to an in-person conversation where it isn't clear what was said, and links to a few posts that don't quite match the thing you're responding to, makes this whole post very confusing. I don't think I could discuss the topic at the object level without quite a few rounds of clarification first.
If wireheading were a serious policy proposal being actively pursued with non-negligible chances of success, I would be shooting to kill wireheaders, not arguing with them.
I am arguing precisely because Jeff and other people musing about wireheading are not actual criminals—but might inspire a future criminal AI if their argument is accepted.
Arguing about a thought experiment means taking it seriously, which I do. And if the conclusion is criminal, this is an important point that needs to be stated. When George Bernard Shaw calmly claims the political necessity of large scale extermination of people unfit for his socialist paradise, and doing it scientifically, he is not being a criminal—but it is extremely relevant to note that his ideas, if implemented, would be criminal, and that accepting them as true might indeed inspire criminals to act, and inspire good people to let criminals act.
If I am not to take wireheading seriously, there is nothing to argue. Just a good laugh to have.
And I am not angry at all about wireheading. But apparently, the first post of this series made a lot of commenters angry indeed who took it personally.
I've read (almost all of) this post and I believe in an objective utility function of the universe, so let me try to respond. I'm happy defending wireheading as an outcome even though no-one is seriously advocating for it.
Let's start by questioning the very concept of "happiness" or "utility". [...] But that doesn't mean that there is an objective way to always precisely measure everything in such terms: the notion is fuzzy and uncertain; ascribing measures is costly and imprecise, totally subjective, and relative to a narrow choice at hand. Even under the best hypotheses (von Neumann Morgenstern utility), it is only defined up to a costly measure and an affine transformation — and what more every measure modifies the system being observed. It certainly isn't a well-defined number easily accessible through introspection, even less so through external observation. Here's for one individual's "utility".
The problem here is that [the fact that we don't have the technology to measure utility yet] doesn't mean that it doesn't exist.
If The Symmetry Theory of Valence (STV) is true, then the bolded part is simply wrong. Such a number exists and it would probably be accessible through external observation. And if it can't be assessed exactly, then this is only a problem insofar as the measurement is flawed; if it's off by 5%, this makes wireheading at most 5% worse, which doesn't change much about it being a desirable outcome.
Given the STV and something like panpsychism, the next two chapters of your post become meaningless because 'individual' is not even a relevant abstraction. Consciousness and valence is just a thing fundamental to the universe, there is nothing special about heaps of atoms that form an individual. You have this mathematical representation that, insofar as we can derive it, tells us precisely how much valence there is for any heap of particles we apply it to.
Unfortunately, your post doesn't really include arguments why the Symmetry Theory of Valence wouldn't be true. And note that this is just an example of a particular theory; Jeff might not even be aware of it but believe that utility or valence can be precisely measured in some other way.
A lot of your post is just pointing out how the premise yields to unintuitive conclusions, which I agree with.
Indeed, from this substrate independence of consciousness and consequently of happiness, he concludes that a bone and flesh wirehead is no better than an emulated wirehead in a physical simulation of the universe, or in a properly optimized abstraction thereof. Once you have captured the simulation blueprints of a "maximally happy moment" (by unit cost of replication by the wirehead maximizer), be it a junkie at the top of his high, a rapist ejaculating inside a helpless high-profile victim, a mass-murdering tyrant getting a kick out of slaughtering a billion victims, or the three combined at once, then you can run this maximally blissful moment in a closed loop, and replicate that loop identically on gazillions of computers, each a paperclip to our paperclip multiplier. Now, a basic principle of software abstraction and optimization is that programs that have identical input, output and side-effects are indistiguishable and that one can replace the other. Since this simulation in a closed loop has no input, no output and no side-effect, then it is indistinguishable from the empty program that does nothing. You don't even need to run the simulation, or to capture it for that matter; doing nothing is already a valid implementation of the paperclip multiplier to unreachable transfinite heights, far beyond what any physical implementation thereof could even conceivably dream of achieving.
Indeed, from this substrate independence of consciousness and consequently of happiness, he concludes that a bone and flesh wirehead is no better than an emulated wirehead in a physical simulation of the universe, or in a properly optimized abstraction thereof. Once you have captured the simulation blueprints of a "maximally happy moment" (by unit cost of replication by the wirehead maximizer), be it a junkie at the top of his high, a rapist ejaculating inside a helpless high-profile victim, a mass-murdering tyrant getting a kick out of slaughtering a billion victims, or the three combined at once, then you can run this maximally blissful moment in a closed loop, and replicate that loop identically on gazillions of computers, each a paperclip to our paperclip multiplier.
Now, a basic principle of software abstraction and optimization is that programs that have identical input, output and side-effects are indistiguishable and that one can replace the other. Since this simulation in a closed loop has no input, no output and no side-effect, then it is indistinguishable from the empty program that does nothing. You don't even need to run the simulation, or to capture it for that matter; doing nothing is already a valid implementation of the paperclip multiplier to unreachable transfinite heights, far beyond what any physical implementation thereof could even conceivably dream of achieving.
This is basically an argument against Functionalism, by which in the idea that consciousness can be described in terms of inputs and outputs only. I agree that this is definitely not true.
Beyond that, it doesn't show much because an emulation of a brain can't just replicate the input/output behavior. At least, it can implement the same computation, which some believe would be enough to produce the same conscious states. You might also be a physicalist and think that this is not enough and what matters is what the particles do. Either way, what you said isn't an argument against happiness out of simulations. In theory, you could run a simulation such that the atoms produce positive valence.
That's all I have to say on object-level critiques. Note that I don't know much about consciousness research or anything. Insofar as understanding it requires understanding physics, I'm totally out of my depth because I don't understand much about physics and am not interested in learning it.
Now, let me go meta.
Once you have captured the simulation blueprints of a "maximally happy moment" (by unit cost of replication by the wirehead maximizer), be it a junkie at the top of his high, a rapist ejaculating inside a helpless high-profile victim, a mass-murdering tyrant getting a kick out of slaughtering a billion victims, or the three combined at once, then you can run this maximally blissful moment in a closed loop, and replicate that loop identically on gazillions of computers, each a paperclip to our paperclip multiplier.
I think this sentence is extremely bad and a good example of what's wrong with this post.
As someone with strong status-regulating emotions, I also bulk at sentences like this
Indeed, there is no question that Jeff is much more intelligent than most (I would notably wager that he's more intelligent than I, as measurable by an IQ test), and much more knowledgeable of this topic than anyone; few could oppose him a rational refutation like I'm going to offer (no false modesty from me here); yet one need not have identified the precise mistakes of a demonstration to reject its provably absurd conclusions.
Though I don't think my overall opinion of this post would be much different if this weren't there.
The STV supposes that pleasantness is valuable independently from the agent's embedding in reality—thus is a Pixie Dust Theory of Happiness, that I indeed argue against in my essay (see section "A Pixie Dust Theory of Happiness").
While the examples and repetition used in the paragraph cited are supposed to elicit a strong emotion, the underlying point holds: If you're trying to find the most emotional happiness intensive moment to reproduce, a violent joyful emotion from an insane criminal mastermind is more likely to be it than a peaceful zen moment by a mellow sage. The extreme negative cost to the victims, however great, is in this hypothesis only accounted once; it is thus dwarfed by the infinitely-replicated benefit to the criminal.
Emotions are a guide. You ought to feel them, and if they're wrong, you ought to explain them away, not ignore them. But, especially in an already long essay, it's easier and more convincing to show than to explain. If mass murder in the name of wireheading feels deeply wrong, that's a very strongly valid argument that indeed it is. Maybe I should update the essay to add this very explanation right afterwards.
Admittedly, my essay may not optimized to the audience of LessWrong, but that's my first couple essays, optimized for my preexisting audience. I wanted to share it here because of the topic, which is extremely relevant to LessWrong.
Finally I'll reply to meta with meta: if you are "totally out of [your] depth... and am not interested in learning it", that's perfectly fine, but then you should disqualify yourself from having an opinion on an "objective utility function of the universe" that you start by claiming you believe in, when you later admit that understanding one issue depends on understanding the other. Or maybe you somehow have an independent proof using a completely different line of argument that makes you confident enough not to look at my argument—in which case you should express more sympathy towards those who'd dismiss Jeff's argument as insane without examining his in detail.