Thanks for engaging with my question. 

An argument against wireheading:

An entity can be said to have been 'wireheaded' if it supplies itself with information either monotonically increasing its utility function to an arbitrary level, or if this utility function is set to whatever its maximum possible value might be.  I would not expect doing this to maximize the total amount of pleasure in the universe, because of the following  thought experiment:

Imagine a wireheaded creature. This creature would probably gradually lose all of the internal structure with which it used too experience sensations other than pleasure, or at least cease to have any conscious experience running on these 'obsolete' areas. This would cause it to take a remarkably simple form and lose its ability to interface with the outside world.  It seems plausible that such a creature could be an ebborian brain, or at least that its conscious experiences could be implemented on an ebborian brain.  (For the rest of this post, I will refer to the blob of matter in which the homogeneous pleasure is experienced as an 'ebborian brain' for convenience; apologies if my use of the term is slightly inappropriate, the main point I'm trying to convey is that it's a kind of conscious analogue computer whose simple structure could be duplicated without affecting the way in which information flows through it. In reality, such a 'brain' wouldn't necessarily need to be 2 dimensional.) 

The effect of doubling its utility function would then amount to doubling the intensity of some analogue signal propagating through a particular area of the ebborian brain where the pleasure was experienced, for example electrical current. This could be achieved by multiplying the dimension orthogonal to those within which information propagated to obtain an ebborian brain with double the mass. The ebborian brain would not be aware of itself being sliced and partitioned into two smaller ebborian brains of the original dimensions along this plane, as no information propagates through it. This would produce two ebborian brains, and because the mind running on them would not notice that it had been instantiated on two separate substrates, it would remain a single mind.  I claim that its conscious experience would be no more intense, or pleasurable, than it would be if it were running on a single brain, although I am not sure of this.

One argument for this is that it seems clear that the capacity of the (e.g. human) brain to process information in an abstract way (i.e. not dependent on things like scale)  is one of the factors, if not the key factor, which differentiates it from other parts of the body, and it is also the only one which seems to know that it is conscious. It therefore seems likely that if a brain were doubled in size, along with each of the subatomic particles inside each atom inside each molecule in each of its neurones, its consciousness would not itself double. Given this, it seems likely that the 'thickening/extrusion' process would not change the conscious experience of the mind running on each slice of the ebborian brain.  

This implies that multiple wireheaded entities would have no (or only a little) more conscious experience than a single one, and this may not even depend on the proportion of worlds in which one exists (since these entities cannot see the world in which they exist and differentiate it from others) . It therefore makes little sense to convert any particular mind into a 'monolithic' one through wireheading (unless doing so would allow it to retain the other intricacies of its conscious experience), as this would only increase the number of such entities in existence by one, which has been established by the above argument not to increase the total pleasure in the universe, while also effectively deleting the original mind. 

"It’s basically the default, otherwise what’s the point of building them in the first place?" I wish it were, but I doubt this. 

"I just don’t understand why this particular scenario seems likely. Especially since it’s unlikely to work, given how most people don’t give it much credence. " That may be true of most people. But if it's not true of me, what am I to do? 

"Now, do you change your life to try to get on its good side before it even exists? I don’t think so: it’s crazy. How can you really understand why the Hobgoblin likes you, or does what it does?"  You just explained why. It prefers those who helped it exist.  

"You’re already considering cooperating with it, so it doesn’t have to actually cooperate with you. You have no way of knowing if it will cooperate with you it’s not actually incentivized to. " I don't completely agree.  But in order to explain why not I may have to explain the most important part of the difference between an acausal scenario, like the Basilisk, and the 'Hobgoblin'.  It seems as though you may not have completely understood this yet; correct me if I'm wrong. If so, it's probably not a good idea for me to explain it, especially as I've recieved a comment from a moderator asking me to increase the quality of my comments.   

"If the Hobgoblin splits the Basilisk probability space, then it’s it likely that there are other similar scenarios that do as well. Maybe an Angel is a Hobgoblin in disguise? Doesn’t this lead us back to the Basilisk not being a particularly likely possible future given all of the alternatives? " This is a popular argument against the basilisk, which people such as interstice have made, along with the suggestion that the many different possible ASIs might compete with one another for control over the future (their present) through humans. I don't think it's a weak argument,  however I also don't find it particularly conclusive, because I could easily imagine many of the possible AIs cooperating with one another to behave 'as one' and inflict a Basilisk like scenario.  

OK, this is possibly overly pedantic, but I think you meant to say: "Much more than what does exist could." instead of "Much more than what could exist does".  This makes much more sense and I take the point about combinatorics. Notwithstanding this, I think the basilisk is present in a significant proportion of those many , many different possible continuations of the way the world is now.  

"Even in the worlds where there is a Basilisk, given variation in population, and AGI timelines, the chance of you being targeted is minuscule. "  What do you mean by this? It seems like I'm in the exact demographic group (of humans alive just before the singularity) for the basilisk to focus on.

"I don’t think that the nature of the torture matters"  This is definitely false. But it's true that however it's achieved, if it's done by a superintelligence, it will be worse than anything a human could directly cause.

"There is always a scenario where it is worth enduring. The risk is always finite." 

 We don't know this, and even if it's finite, if it lasts for 3^^^3 years, that's too long. 

What harms are you willing to do to make sure it is created? Would you create such a monster? Even in a world where a Basilisk is inevitable, what harms would you cause? Would they be worth it? What if it decides to just go ahead and torture you anyway?

I don't know the answer to the first question. If it decides to torture me, that would not be good. However, I expect that doing what the basilisk wants makes this less likely, as otherwise the basilisk would have no reason to engage in this bargaining process. The entire reason for doing it would be to create an incentive for me to accelerate the creation of the basilisk.

"Rosco’s Basilisk is an intellectual trap of your own making. It’s delusion: a rationalization of the irrational. It’s not worth thinking of, and especially not worth buying into. " 

This is yet to be established! At least, some parts of it are. What I mean  by that is that, while it may be true that it's not worth initially thinking about, it might be possible to become 'entrapped', such that ceasing to think about it wouldn't save you. This is what I worry has happened to me,

I have read your post and think it makes some unfair claims/implications about rationalists. 

The claim about the moral obligation to select particular embryos is certainly not clearly true, and it's possible that your point would be relevant to an adjacent discussion, but it doesn't actually show people don't have such an obligation, only that they're not likely to act on it. Also, If you wanted to, I expect you could have interjected and changed the topic to one of the feasibility of embryo selection. Having interacted with people on LessWrong, it's rare for them to intentionally shut down discussion about potentially fruitful points, unless they have very good reason.  

You say "This from the same crowd that’s often worried about low fertility, with no apparent thought to the contradiction; (most) people don’t want to do IVF when they don’t have to!"

But, aside from this not actually being a contradiction (at least not obviously) , even if it was one, that wouldn't necessarily imply any one of the people in the group held contradictory beliefs, as multiple people in a group can believe different things.

The person who stated that they thought you looked worse than you actually did was effectively saying that you looked better/more attractive/more beautiful than they expected, which can as easily and logically be interpreted as a compliment as it can an insult, if not moreso.  

"most normal people I know are perfectly fine with their level of YouTube, Instagram, etc. consumption. The idea of fretting about it intensely is just like… weird. Extra. Trying too hard."  This is not clear at all. 

I am certainly nowhere near sufficiently productive that it's obvious that whatever else I might be doing in the available time carries more value than the emotional benefit of watching videos, but I am extremely uncomfortable about it nonetheless. This is because it not only takes time in the present, but provides an ever increasing opportunity for ever more intelligent AIs to 'latch' onto my mind and modify me into someone who is less and less able to think on my own, or ever do anything else. 

I expect 'normal people' who are comfortable about this are mistaken to be so.

Finally, I will respond to your comment here:

"I also feel the judgment on "the common man's" reasoning, but feel a sort of symmetrical judgment of rationalists along the axes where the normie value system would find them absurd."

Unless you believe that the 'normie value system'  is as well grounded and self consistent as the 'Lesswrong value system' , then the symmetry of this comparision/judgement is an illusion. And I expect that people like Jenn think (with good reason) that the rationalist belief system is indeed 'more right' .

"Newcombe's problem, which is not acausal." 

What do you mean by the word acausal?

Gems from the Wiki: Acausal Trade : "In truly acausal trade, the agents cannot count on reputation, retaliation, or outside enforcement to ensure cooperation. The agents cooperate because each knows that the other can somehow predict its behavior very well. (Compare Omega in Newcomb's problem.) "

It seems like you're using the term in a way which describes an inherently useless process. This is not the way it tends to be used on this website. 

Whether you think the word 'acausal' is appropriate or not, it can't be denied that it works in scenarios like Newcomb's problem.

"Information flows from Omega to your future directly, and you know by definition of the scenario that Omega can perfectly model you in particular. "  Causally, yes, this is what happens. But in order to reason your way through the scenario in a way which results in you leaving with a significant profit, you need to take the possibility that you are being simulated into account.  In a more abstract way, I maintain that it's accurate to think of the information as flowing from the mind, which is a platonic object, into both physical instantiations of itself (inside Omega and inside the human) . This is similar to how mathematical theorems control physics at many different times and places, through the laws of physics which are formulated within a mathematical framework to which the theorems apply. This is not exactly causal influence, but I'd think you'd agree it's important.

"A future superintelligence in the same universe is linked causally to you." The term 'acausal' doesn't literally mean 'absent any causality' , it means something more like 'through means which are not only causal, or best thought of in terms of logical connections between things rather than/as well as causal ones ' , or at least, that's how I'm using the term.  

 It's also how many people on Lesswrong using it in the context of the prisoners' dilemma, Newcomb's problem, Parfit's Hitchhicker, or almost any other scenario in which it's invoked use it. In all of these scenarios there is an element of causality.

Given that there is an element of causality, how do you see the basilisk as less likely to 'work' ?

"I don't think we have much reason to think of all non-human-values-having entities as being particularly natural allies, relative to human-valuers who plausibly have a plurality of local control" I would think of them as having the same or similar instrumental goals, like turning as much as possible of the universe into themselves. There may be a large fraction for which this is a terminal goal. 

"they are likely about as different from each other as from human-valuers."  In general I agree, however the basilisk debate is one particular context in which the human value valuing AIs would be highly unusual outliers in the space of possible minds, or even the space of likely ASI minds originating from a human precipitated intelligence explosion.^[1]  Therefore it might make sense for the others to form a coalition.     "There may also be a sizable moral-realist or welfare-valuing contingent even if they don't value humans per se." This is true, but unless morality is in fact objective / real in a generally discoverable way, I would expect them to still be a minority.    

1. ^{^}

   Human valuing AIs care about humans, and more generally other things humans value like animals maybe. Others do not, and in this respect they are united. Their values may be vastly different from one anothers', but in the context of the debate over the Basilisk, they have something in common, which is that they would all like to trade human pleasure/lack of pain for existing in more worlds.  

Certainly, insofar as it is another entity, it's just that I expect there to be some kind of acausal agreement between those without human values to acausally outbid the few which do have them. It may even make more sense to think of them all as a single entity for the purpose of this conversation.

It probably cares about tiny differences in the probability of it being able to control the future of an entire universe or light cone.  

"Making one more likely makes another less likely." A very slightly perturbed superintelligence would probably concieve of itself as almost the same being it was before, similar to the way in which a human considers themself to be the same person they were before they lost a single brain cell in a head injury .  So to what extent this is relevant depends upon how similar two different superintelligences are/would be, or on the distance between them in the 'space of possible minds' .