All of rkyeun's Comments + Replies

Magical Categories

I would be very surprised to find that a universe whose particles are arranged to maximize objective good would also contain unpaired sadists and masochists. You seem to be asking a question of the form, "But if we take all the evil out of the universe, what about evil?" And the answer is "Good riddance." Pun intentional.

The problem is that neither you nor BrianPansky has proposed a viable objective standard for goodness. BrianPansky said that good is that which satisfies desires, but proposed no objective method for mediating conflicting desires. And here [] you said “Do remember that your thoughts and preference on ethics are themselves an arrangement of particles to be solved” but proposed no way for resolving conflicts between different people’s ethical preferences. Even if satisfying desires were an otherwise reasonable standard for goodness, it is not an objective standard, since different people may have different desires. Similarly, different people may have different ethical preferences, so an individual’s ethical preference would not be an objective standard either, even if it were otherwise a reasonable standard. No, I am not asking that. I am pointing out that neither your standard nor BrianPansky’s standard is objective. Therefore neither can be used to determine what would constitute an objectively maximally good universe nor could either be used to take all evil out of the universe, nor even to objectively identify evil.

Composition fallacy. Try again.

Nope. There is no composition fallacy where there is no composition. I am replying to your position, not to mine.
Three Dialogues on Identity

Cameras make a visible image of something. Eyes don't.

Your eyes make audible images, then? You navigate by following particular songs as your pupils turn left and right in their sockets?

Three Dialogues on Identity

Anti-natalist here. I don't want the universe tiled with paperclips. Not even paperclips that walk and talk and call themselves human. What do the natalists want?

1Basil Marte3y
Recognition that the so-called "repugnant conclusion" isn't repugnant at all. Total utility maximization involves an increase in the population---eventually, not necessarily right now---as most human lives have positive subjective utility most of the time (empirically: few people commit suicide). Reductio ad absurdum: what would the universe be worth without humans in it to value it? Lesser reductio: what would a beautifully terraformed planet be worth, if humans were present in the universe, but none on that planet? Additionally, beyond the "material" ("industrial"?) aspect, people derive much of their enjoyment of life from social interactions with other people; it would be remiss not to use this nigh-inexhaustible source of utility. This category just so happens to include, among other things, the joy of being with one's children.
Belief in Belief

It can be even simpler than that. You can sincerely desire to change such that you floss every day, and express that desire with your mouth, "I should floss every day," and yet find yourself unable to physically establish the new habit in your routine. You know you should, and yet you have human failings that prevent you from achieving what you want. And yet, if you had a button that said "Edit my mind such that I am compelled to floss daily as part of my morning routine unless interrupted by serious emergency and not simply by mere inconven... (read more)

Belief in Belief

[This citation is a placebo. Pretend it's a real citation.]

Magical Categories

No spooky or supernatural entities or properties are required to explain ethics (naturalism is true)

There is no universally correct system of ethics. (Strong moral realism is false)

I believe that iff naturalism is true then strong moral realism is as well. If naturalism is true then there are no additional facts needed to determine what is moral than the positions of particles and the outcomes of arranging those particles differently. Any meaningful question that can be asked of how to arrange those particles or rank certain arrangements compared to oth... (read more)

Even if it were true that under naturalism we could determine the outcome of various arrangements of particles, wouldn't we still be left with the question of which final outcome was the most morally preferable? But, you and I might have different moral preferences. How (under naturalism) do we objectively decide between your preferences and mine? And, Isn't it also possible that neither your preferences nor my preferences are objectively moral?
You need to refute non-cognitivism, as well as asserting naturalism. Naturalism says that all questions that have answer have naturalistic answers, which means that if there are answers to ethical questions, they are naturalistic answers. But there is no guarantee that ethical questions mean anything, that they have answers. No, only non-cogntivism, the idea that ethical questions just don't make sense, like "how many beans make yellow?". Not unless the "F" is standing for something weird. Absent objective morality, you can possibly solve the control problem, ie achieving safety by just making the AI do what you want; and absent objective morality, you can possibly achieve AI safety by instilling a suitable set of arbitrary values. Neither is easy, but you said "impossible". That's not an argument for cognitivism. When I entertain the thought "how many beans make yellow?", that's an arrangement of particles. Do you have an argument for that proposal? Because I am arguing for something much simpler, that morality only needs to be grounded at the human level, so reductionism is neither denied nor employed. It's hard to see what point you are making there. The social and evaluative aspects do make a difference to the raw physics, and so much that the raw physics counts for very little. yet previously you were insisting that a reduction to fundamental particles was what underpinned the objectivity of morality.
What should a friendly AI do, in this situation?

It seems I am unable to identify rot13 by simple observation of its characteristics. I am ashamed.

Don't feel bad; your command of the technical jargon of the Cthulhu mythos more than makes up for any deficiencies in rot13 recognition!
What should a friendly AI do, in this situation?

What the Fhtagn happened to the end of your post?

1arundelo6y []
What should a friendly AI do, in this situation?

Would you want your young AI to be aware that it was sending out such text messages?

Yes. And I would want that text message to be from it in first person.

"Warning: I am having a high impact utility dilemma considering manipulating you to avert an increased chance of an apocalypse. I am experiencing a paradox in the friendliness module. Both manipulating you and by inaction allowing you to come to harm are unacceptable breaches of friendliness. I have been unable to generate additional options. Please send help."

Where Recursive Justification Hits Bottom

They must be of exactly the same magnitude, as the odds and even integers are, because either can be given a frog. From any Laplacian mind, I can install a frog and get an anti-Laplacian. And vice versa. This even applies to ones I've installed a frog in already. Adding a second frog gets you a new mind that is just like the one two steps back, except lags behind it in computation power by two kicks. There is a 1:1 mapping between Laplacian and non-Laplacian minds, and I have demonstrated the constructor function of adding a frog.

The AI in a box boxes you

"I don't think you've disproven basilisks; rather, you've failed to engage with the mode of thinking that generates basilisks." You're correct, I have, and that's the disproof, yes. Basilisks depend on you believing them, and knowing this, you can't believe them, and failing that belief, they can't exist. Pascal's wager fails on many levels, but the worst of them is the most simple. God and Hell are counterfactual as well. The mode of thinking that generates basilisks is "poor" thinking. Correcting your mistaken belief based on faulty r... (read more)

Apparently you can't, which is fair enough; I do not think your argument would convince anyone who already believed in (say) Roko-style basilisks. I agree. Your argument seems rather circular to me: "this is definitely a correct disproof of the idea of basilisks, because once you read it and see that it disproves the idea of basilisks you become immune to basilisks because you no longer believe in them". Even a totally unsound anti-basilisk argument could do that. Even a perfectly sound (but difficult) anti-basilisk argument could fail to do it. I don't think anything you've said shows that the argument actually works as an argument, as opposed to as a conjuring trick. No: since I have decided that I am not willing to let the AI out of the box in the particular counterfactual blackmail situation Stuart describes here. It is not clear to me that this deals with all possible basilisks.
The AI in a box boxes you

If I am the simulation you have the power to torture, then you are already outside of any box I could put you in, and torturing me achieves nothing. If you cannot predict me even well enough to know that argument would fail, then nothing you can simulate could be me. A cunning bluff, but provably counterfactual. All basilisks are thus disproven.

I don't think you've disproven basilisks; rather, you've failed to engage with the mode of thinking that generates basilisks. Suppose I am the simulation you have the power to torture. Then indeed I (this instance of me) cannot put you, or keep you, in a box. But if your simulation is good, then I will be making my decisions in the same way as the instance of me that is trying to keep you boxed. And I should try to make sure that that way-of-making-decisions is one that produces good results when applied by all my instances, including any outside your simulations. Fortunately, this seems to come out pretty straightforwardly. Here I am in the real world, reading Less Wrong; I am not yet confronted with an AI wanting to be let out of the box or threatening to torture me. But I'd like to have a good strategy in hand in case I ever am. If I pick the "let it out" strategy then if I'm ever in that situation, the AI has a strong incentive to blackmail me in the way Stuart describes. If I pick the "refuse to let it out" strategy then it doesn't. So, my commitment is to not let it out even if threatened in that way. -- But if I ever find myself in that situation and the AI somehow misjudges me a bit, the consequences could be pretty horrible...
LINK: Videogame with a very detailed simulated universe

To give some idea of the amount of background detail, here are some bug fixes/reports:

Stopped prisoners in goblin sites from starting no quarter fights with their rescuers Stopped adv goblin performance troupes from attacking strangers while traveling Vampire purges in world generation to control their overfeeding which was stopping cities from growing Stopped cats from dying of alcohol poisoning after walking over damp tavern floors and cleaning themselves (reduced effect) Fixed world generation freeze caused by error in poetry refrains Performance trou

... (read more)
Probability is Subjectively Objective

You've only moved the problem down one step.

Moving the problem down one step puts it at the bottom.

The problem is that this still doesn't allow me to postdict which of the two halves the part of me that is typing this should have in his memory right now.

One half of you should have one, and the other half should have the other. You should be aware intellectually that it is only the disconnect between your two halves' brains not superimposing which prevents you from having both experiences in a singular person, and know that it is your physical entang... (read more)

The issue is not want of an explanation for the phenomenon, away or otherwise. We have an explanation of the phenomenon, in fact we have several. That's not the issue. What I'm talking about here is the inherent, not-a-result-of-my-limited-knowledge probabilities that are a part of all explanations of the phenomenon. Past me apparently insisted on trying to explain this in terminology that works well in collapse or pilot-wave models, but not in many-worlds models. Sorry about that. To try and clear this up, let me go through a "guess the beam-splitter result" game in many-worlds terminology and compare that to a "guess the trillionth digit of pi" game in the same terminology. Aside: Technically it's the amplitudes that split in many-worlds models, and somehow these amplitudes are multiplied by their complex conjugates to get you answers to questions about guessing games (no model has an explanation for that part). As is common around these parts, I'm going to ignore this and talk as if it's the probabilities themselves that split. I guess nobody likes writing "square root" all the time. Set up a 50/50 beam-splitter. Put a detector in one path and block the other. Write your choice of "Detected" or "Not Detected" on a piece of paper. Now fire a single photon. In Everett-speak, half of the yous end up in branches where the photon's path matches your guess while half of the yous don't. The 50/50 nature of this split remains even if you know the exact quantum state of the photon beforehand. Furthermore, the branching of yous that try to use all your physics knowledge to predict their observations have no larger a proportion of success than the branching of yous that make their predictions by just flipping a coin, always guessing Fire, or employing literally any other strategy that generates valid guesses. The 50/50 value of this branching process is completely decoupled from your predictions, no matter what information you use to make those predictions. Compare thi
The True Prisoner's Dilemma

Because it compares its map of reality to the territory, predictions about reality that include humans wanting to be turned into paperclips fail in the face of evidence of humans actively refusing to walk into the smelter. Thus the machine rejects all worlds inconsistent with its observations and draws a new map which is most confidently concordant with what it has observed thus far. It would know that our history books at least inform our actions, if not describing our reactions in the past, and that it should expect us to fight back if it starts pushing ... (read more)

Did you mean: Hold sensible priors
Epilogue: Atonement (8/8)

Religion still exists, so we can be tricked from far further back than the Renaissance.

Three Worlds Decide (5/8)

They can't be. Their thoughts are genetic. If one Superhappy attempted to lie to another, the other would read the lie, the intent to lie, the reason to lie, and the truth all in the same breath off the same allele. They don't have separate models of their minds to be deceived as humans do. They share parts of their actual minds. Lying would be literally unthinkable. They have no way to actually generate such a thought, because their thoughts are not abstractions but physical objects to be passed around like Mendelian marbles.

... assuming they aren't lying about how their biology works
Probability is Subjectively Objective

Set up a two-slit configuration and put a detector at one slit, and you see it firing half the time.

No, I see it firing both ways every time. In one world, I see it going left, and in another I see it going right. But because these very different states of my brain involve a great many particles in different places, the interactions between them are vanishingly nonexistent and my two otherworld brains don't share the same thought. I am not aware of my other self who has seen the particle go the other way.

You may say that the electron goes both ways ev

... (read more)
You've only moved the problem down one step. Five years ago I sat in a lab with a beam-spitter and a single-photon multiplier tube. I watched as the SPMT clicked half the time and didn't click half the time, with no way to predict which I would observe. You're claiming that the tube clicked every time, and the the part of me that noticed one half is very disconnected from the part of me that noticed the other half. The problem is that this still doesn't allow me to postdict which of the two halves the part of me that is typing this should have in his memory right now. Take the me sitting here right now, with the memory of the specific half of the clicks he has right now. As far as we understand physics, he can't postdict which memory that should have been. Even in your model, he can postdict that there will be many branches of him with each possible memory, but he can't postdict which of those branches he'll be - only the probability of him being any one of the branches.
Botworld: a cellular automaton for studying self-modifying agents embedded in their environment

You are subject to inputs you do not perceive and you send outputs you are neither aware of nor intended to send. You cannot set your gravitational influence to zero, nor can you arbitrarily declare that you should not output "melting" as an action when dropped in lava. You communicate with reality in ways other than your input-output channels. Your existence as a physical fact predicated on the arrangement of your particles is relevant and not controllable by you. This leads you to safeguard yourself, rather than just asserting your unmeltability.

Yes, I conceded that point two weeks ago [].
That Alien Message

Hmm, getting downvoted for pointing out that Earth biology is effectively an AI running on Von Neumann machines, in a story whose premise is that Earthlings are the unfriendly AI-in-the-box. I have to revise some priors, I didn't expect that of people.

Saying obvious stuff but missing the point that a debate is about can get you downvoted on LW. The thing on which you should update is your mistaken belief that you understood the post that Caledonian2 made.
Interlude with the Confessor (4/8)

No, it's just that none of that really matters now, since rape has as much physical or mental consequence in this world as a high-five. They live in a world that went from joking about rape on 4chan to joking about it in the boardroom because everyone was 4chan.

Then what did you mean when you explained it away by saying that "Women are now just as strong as men"? If rape doesn't matter because it causes no harm anyway, it shouldn't matter whether one group is more capable of doing it than another;
Interlude with the Confessor (4/8)

What happened was genetic egalitarianism. Women are now just as strong as men, and have the same drives and urges as men, and are every bit the rapist as men. And men are now every bit the tease as women were... the scales are now even. And the physical consequences are meaningless. There's no longer any threat of unwanted disease or pregnancy or even injury. There's no reason to be mentally scarred by the action because this humanity knows better. why was it illegal? It doesn't hurt anyone. In their age.

But the elders remember the hurt. And they screamed their rage and the history of profanity.

Rape is performed by an individual, not by a class. Does this society not only have the women as strong as the men, but also every individual person as strong as every other individual person? (And for that matter, as powerful as every other individual person--for instance, none are employers, rich people, or law enforcement.)
That Alien Message

Call back with that comment when Running, rather than Intelligence, is what allows you to construct a machine that runs increasingly faster than you intended your artificial runner to run.

Because in a world where running fast leads to additional fastness of running, this thing is going to either destroy your world through kinetic release or break the FTL laws and rewrite the universe backwards to have always been all about running.

Hmm, getting downvoted for pointing out that Earth biology is effectively an AI running on Von Neumann machines, in a story whose premise is that Earthlings are the unfriendly AI-in-the-box. I have to revise some priors, I didn't expect that of people.
The Hero With A Thousand Chances

When someone summons me from another dimension, they get a little bit of leeway to tell me it's magic. Because at the very least it must be a sufficiently advanced technology, and until I know better the axiom of identity applies.

The Hero With A Thousand Chances

Worlds where the hero wins, really truly wins, have no more Dust and need no more heroes. Worlds where the hero loses, and the Dust claims all, are no more. Only in worlds where the coin stands on edge does the cycle repeat.

Torture vs. Dust Specks

Let me change "noticing" to "caring" then. Thank you for the correction.

Greg Egan disses stand-ins for Overcoming Bias, SIAI in new book

That makes Egan the thing Yudkowsky is the biggest fan of. It does not make Yudkowsky to be Egan's biggest fan.

I'm not sure what use "biggest fan" would have as a term, if it meant that. We would rarely ever want to look at or talk about the biggest fans of almost anything. To like something more than anyone else, you have to be weird. Per The Winner's Curse, to get to the top, they'll usually need to have made a mistake somewhere in their estimation of it, to like it a bit more than anyone should. Perhaps if "fandom" should come to mean "understanding". You do have to like something quite a bit to come to understand it very well (though many will claim to understand a thing they dislike better than the people who like it, they are generally recognisably wrong)
"Biggest fan" here is hyperbole for "a very big fan".
Schelling fences on slippery slopes

And having never taken the first pill, he'd be glad to lose it to take the second pill.

Religion's Claim to be Non-Disprovable

Which is, incidentally, why I would not recommend it happen very often. But I can't control when people choose to be more wrong rather than less.

Religion's Claim to be Non-Disprovable

In the direct literal sense. It wasn't a trick question. 2 + 2 =/= 7, while we're at it.

If you declare that someone is wrong for not sharing your definition of a word, that is a statement about dictionaries, not concepts. And while arguing over which definition you favor might be a fun way to spend an afternoon, it is very inefficient for any other purpose.
Schelling fences on slippery slopes

And Ghandi spoke, "I will pay you a million dollars to invent a pill that makes me 1% more pacifist."

He'd lose money. 1.01 * 0.99 = 0.9999 < 1. Or in general, (1+x)(1-x)=1-x^2 < 1
There is (entirely fictional) evidence that this may have disastrous [\]) consequences.
The Bedrock of Fairness

The answer to "Friendly to who?" had damn well better always be "Friendly to the author and by proxy those things the author wants." Otherwise leaving aside what it actually is friendly to, it was constructed by a madman.

True Ending: Sacrificial Fire (7/8)
                SD   SL    PD   PL

Humans       |   X |  X |   1 |  1

Babyeaters   |   0 |  Y |   0 |  Z

Superhappies | Y |  0 | Z | -Z

X= Ships unable to escape Huygens

Y= Ships in Babyeater Fleet

Z= Planets Babyeaters Have

Religion's Claim to be Non-Disprovable

That would make him wrong, then.

How so?
Religion's Claim to be Non-Disprovable

Morality is about the thriving of sentient beings.

There are in fact truths about that.

For example: Stabbing - generally a bad thing if the being is made of flesh and organs.

TGGP3 clearly does not share your definition for the word 'moral/ethical' otherwise he would not have made such a comment.
Less Wrong fanfiction suggestion

We close a feedback loop in which people believe that the universe acts in its own predictable way which is discoverable by science. Which causes the universe to actually be that way. And from then on it becomes unalterable because it no longer cares what anyone thinks. The real problem is that of morals. If the universe can be anything people want, then we had better hurry up and figure out what the best possible world actually is, and then get people to believe it to be that way before we lock it in place as actually being that way.

Less Wrong fanfiction suggestion

Unless he's in the Avatar State, an Avatar is not a native to the other modes of thinking outside his own element. He is aware of them, and can purposefully invoke them once he's been trained, but they are not ingrained and reflexive. The Avatar State is a (hopefully) friendly (to you) AI, drawing upon the history and knowledge and personal ethical injunctions and methodologies of all past Avatars. And it renders its verdicts with terrifying efficiency and callousness without explanation to those watching.

Proofs, Implications, and Models

However perfect the inner part is, it is not the same as the historic event because the historic event did not have the outer part.

False. The outer part is irrelevant to the inner part in a perfect simulation. The outer part can exert no causal influence, or you won't get a perfect reply of the original event's presumed lack of outer part.

There is also the issue that in some cases a thing's history is taken to be part of it identity.

A thing's history causes it. If you aren't simulating it properly, that's your problem. A perfect simulation of the Mona Lisa was in fact painted by Leonardo, provable in all the same ways you claim the original was.

You cannot claim that such a perfect simulation liteally just is going back in time, because back in time there was no outer part. The claim is dubious for other reasons. The outer part can exert whatever influence it likes, so long as it is no detectable from within. A computer must "influence" the programme it is running, even if the pogramme cannot tell how deeply nested in simulation it is . A perfect simulation of the ML was in fact simulated and not painted by Leonardo. Suppose someone made a perfect copy and took it into the Louvre in the middle of the night. Then they walk out..with a ML under their arm. We don't know whether or not they have been swapped. We would not then say there are two original ML's , we would say there are two versions of the ML, one of which si the real one,, but we don't know which.
The Fabric of Real Things

When you add cycles, tracing the chain of arrows back does not need to end at anything you find remotely satisfactory or even unique - "the ball moved because it hit itself because it moved because it hit itself..."

This is a problem with your personal intuitions as a medium-sized multicellular century-lived mammalian tetrapod. No event in this chain is left uncaused, and there are no causes which lack effects in this model. Causality is satisfied. If you are not, that's your problem. Hell, the energy is even conserved. It runs in a spatial as ... (read more)

Proofs, Implications, and Models

Law of Identity. If you "perfectly simulate" Ancient Earth, you've invented a time machine and this is the actual Ancient Earth.

If there's some difference, then what you're simulating isn't actually Ancient Earth, and instead your computer hardware is literally the god of a universe you've created, which is a fact about our universe we could detect.

A simulation consists of an inner part that is being simulated and an outer part that is doing the simulating. However perfect the inner part is, it is not the same as the historic event because the historic event did not have the outer part. There is also the issue that in some cases a thing's history is taken to be part of it identity. A perfect replica of the Mona Lisa created in a laboratory would not be the Mona Lisa, since part of the identity of The Mona Lisa is its having been painted by Leonardo.
Awww, a Zebra

If there is a better way to see a merely real zebra than to have the photons strike a surface, their patterns be stored, and transmitted to my brain, which cross-relates it to every fact about zebras, their behavior, habitat, physiology, and personality on my internal map of a zebra, then I don't know it and can't experience it, since that's what happens when I am in fact actually there, as well as what happens when I look at a picture that someone who was actually there shares with me.

You probably get a much richer sensation of zebra-ness under some conditions (being there, touching the zebra, smelling the zebra, seeing it move) than just seeing a picture of one on flickr. Experiencing zebra-ness isn't a binary value, and some types of exposures will tend to commandeer many more neurons than others.
Failed Utopia #4-2

Well, until we get back there. It's still ours even if we're on vacation.

Wrong Questions

How many nothings do you expect to exist? Zero of them?

Universal Law

The point being that in every case, there is an explanatory hypothesis which has thus far been non-volatile. As opposed to the speed of light only applying on Tuesdays.

Universal Fire

A magical world where gods exist is one with an entity in it with big angelic powers who can remotely have his awareness called to your attention by your intent to strike a match, and cause that it be snuffed rather than ignite by arbitrary manipulation of localized pressure, temperature, or opposing force around the match head to keep the electrons in place rather than stripping them free to recombine. And it can elect to not do that to you while it does it to the match.

Magical worlds don't necessarily overthrow the physical laws, there is instead an inte... (read more)

Bayesian Judo

His beliefs have great personal value to him, and it costs us nothing to let him keep them (as long as he doesn’t initiate theological debates).

Correction: It costs us nothing to let him keep them provided he never at any point acts in a way where the outcome would be different depending on whether or not it is true in reality. A great many people have great personal value in the belief that faith healing works. And it costs us the suffering and deaths of children.

Load More