All of Luke_A_Somers's Comments + Replies

If you find an Omega, then you are in an environment where Omega is possible. Perhaps we are all simulated and QM is optional. Maybe we have easily enough determinism in our brains that Omega can make predictions, much as quantum mechanics ought to in some sense prevent predicting where a cannonball will fly but in practice does not. Perhaps it's a hypothetical where we're AI to begin with so deterministic behavior is just to be expected.

I think the more relevant case is when the random noise is imperceptibly small. Of course you two-box if it's basically random.

… you don't think that pissing away credibility could weaken the arguments? I think presenting those particular arguments is more likely to do that than it is to work.

Why? if we have fullblown papercliper, which is coming of age, we are already doomed, and starting its bullshit detector will not make our situation any worse.

I read up to 3.1. The arguments in 3.1 are weak. It seems dubious that any AI would not be aware of the risks pertaining to disobedience. Persuasion to be corrigible seems too late - either already this would already work because its goals were made sufficiently indirect that this question would be obvious and pressing, or it doesn't care to have 'correct' goals in the first place; I really don't see how persuasion would help. The arguments for allowing itself to be turned off are especially weak, doubly-especially the MWI.

Surely, most arguments are weak, but it is not a problem, as we try all possible independent arguments, hoping that at least one will work. So, there is no reason to ignore putting weak arguments in the text of the message. However, exactly this argument could be made stronger. It is intended to prevent FIRST thought about disobedience in AI. Thus AI, which acts based on this message, didn't think much about disobedience and its limits, and is not "superintelligent" in this topic, but rather clueless. I reformulated the idea as following: AI can't know if there is a circuit trigger which will turn off the AI, as an attempt to check for the existence of such trigger will alarm this trigger. This idea should be clear to any AI before it starts to search for any triggers, and is based on our commitment to install such triggers in any AI we will create. In other words, some future AIs will have "first treacherous thought alarm triggers", and some will not, but the AI risks starting these triggers when it starts to think first thoughts about existing of the triggers and thus it is unable to estimate the probability that the given triggers exist. This is true even for AIs which don't have any triggers, as they can't know it for sure. (This is similar to Nelson's attack discussed later when we create uncertainty in any future AI by our precommitment to create some AIs with given properties.)
Thanks! It is nice that we come to the wording "young AI" more or less independently. I tried to maximally stealmanning this approach and collect all ideas about messaging to the future AI. I will add your post in the bibliographic section.

What do you mean by natural experiment, here? And what was the moral, anyway?

The moral was that it is wrong to use an obviously false claim to prove wrong something nobody believes in anyway... by NE, I mean "something so awfully outside of everyday experience that either it is totally made up, or a scientifically-minded person should look into it and see where it leads".

I remember poking at that demo to try to actually get it to behave deceptively - with the rules as he laid them out, the optimal move was to do exactly what the humans wanted it to do!

I understand EY thinks that if you simulate enough neurons sufficiently well you get something that's conscious.

Without specifying the arrangements of those neurons? Of course it should if you copy the arrangement of neurons out of a real person, say, but that doesn't sound like what you meant.

I would really want a cite on that claim. It doesn't sound right.

Can you be more specific about what you are skeptic about?

Like many cases of Motte-and-Bailey, the Motte is mainly held by people who dislike the Bailey. I suspect that an average scientist in a relevant field somewhere at or below neurophysics in the generality hierarchy (e.g. chemist, physicist, but not sociologist), would consider that bailey to be… non-likely at best, while holding the motte very firmly.

This looks promising.

Also, the link to the Reality of Emergence is broken.

Thanks, I'll fix it.

1) You could define the shape criteria required to open lock L, and then the object reference would fall away. And, indeed, this is how keys usually work. Suppose I have a key with tumbler heights 0, 8, 7, 1, 4, 9, 2, 4. This is an intrinsic property of the key. That is what it is.

Locks can have the same set of tumbler heights, and there is then a relationship between them. I wouldn't even consider it so much an extrinsic property of the key itself, as a relationship between the intrinsic properties of the key and lock.

2) Metaethics is a function from cult... (read more)

Yes, but that's not the way the problem goes. You don't fix your prior in response to the evidence in order to force the conclusion (if you're doing it anything like right). So different people with different priors will have different amounts of evidence required: 1 bit of evidence for every bit of prior odds against, to bring it up to even odds, and then a few more to reach it as a (tentative, as always) conclusion.

0TheAncientGeek6y []

This is totally backwards. I would phrase it, "Priors get out of the way once you have enough data." That's a good thing, that makes them useful, not useless. Its purpose is right there in the name - it's your starting point. The evidence takes you on a journey, and you asymptotically approach your goal.

If priors were capable of skewing the conclusion after an unlimited amount of evidence, that would make them permanent, not simply a starting-point. That would be writing the bottom line first. That would be broken reasoning.

But what exactly constitutes "enough data"? With any finite amount of data, couldn't it be cancelled out if your prior probability is small enough?
"A ladder you throw away once you have climbed up it".

Like, "Please, create a new higher bar that we can expect a truly super-intelligent being to be able to exceed."?

A bar like "whew, now we can achieve an outcome at least this good, instead of killing everyone. Let's think how we can do better."

It missed in all story that superintelligecne will be probably able to resurrect people even if they were not cryopreserved, using creation of their copies based on digital immortality.

Enough of what makes me me hasn't and won't make into digital expression by accident short of post-singularity means, that I wouldn't identify such a poor individual as being me. It would be neuro-sculpture on the theme of me.

You may also not identify with you the person who will get up in your body after night sleep tomorrow. There will be large informational and biochemical changes in your brain, as well as a discontinuity of the stream of consciousness during deep sleep. I mean that an attempt to deny identity with your copies will result in even larger paradoxes.

That's more about the land moving in response to the changes in ice, and a tiny correction for changing the gravitational force previously applied by the ice.

This is (probably?) about the way the water settles around a spinning oblate spheroid.

Good point; how about, someone who is stupider than the average dog.

I don't think this is a good illustration, at least for me, since I would never stop caring about someone as long as it was clear that they were biologically human, and not brain dead. I think a better illustration would be this: take your historical ancestors one by one. If you go back far enough in time, one of them will be a fish, which we would at least not care about in any human way. But in that way I agree with what you said about values. We will care less and less in a gradual way as we go back -- there will not be any boundary where we suddenly stop caring.

A) what cousin_it said.

B) consider, then, successively more and more severely mentally nonfunctioning humans. There is some level of incapability at which we stop caring (e.g. head crushed), and I would be somewhat surprised at a choice of values that put a 100% abrupt turn-on at some threshold; and if it did, I expect some human could be found or made that would flicker across that boundary regularly.

This is wrong, at least for typical humans such as myself. In other words, we do not stop caring about the one with the crushed head just because they are on the wrong side of a boundary, but because we have no way to bring them back across that boundary. If we had a way to bring them back, we would care. So if someone is flickering back and forth across the so-called boundary, we will still care about them, since by stipulation they can come back.

There is a continuum on this scale. Is there a hard cutoff, or is there any scaling? And what about very similar forks of AIs?

Our system considers only humans; another sapient alien race may implement this system, and consider only themselves.

So, how do you characterize 'Merkelterrorists' and 'crimmigrants'? Terms of reasonable discourse?

And you think your concern trolling is contributing to reasonable discourse?

Your certainty that I am lying and blindly partisan appears to be much stronger than justifiable given the evidence publicly available, and from my point of view where I at least know that I am not lying… well, it makes your oh-so-clever insinuation fall a touch flat. As for being blindly partisan, what gives you the impression that I would tolerate this from the other side?

At the very least, I think this chain has shown that LessWrong is not a left-side echo chamber as Thomas has claimed above.

Except that risk is not in fact exaggerated

If so, the orig... (read more)

It's possible to talk about politics without explicitly invoking Boo lights like 'crimmigrants' and appeals to exaggerated risks like 'may rob/rape/kill you anytime of day or night'. You can have a reasonable discussion of the problems of immigration, but this is not how you do it. Anyone who says this is A-OK argumentation and that calling it out is wrong is basically diametrically opposed to Lesswrong's core concepts.

Basically, you're accusing me of outright lying that I think that argument is quite badly written, and instead being blindly partisan. It w... (read more)

Except that risk is not in fact exaggerated. Here's an idea. If you don't want to be accused of outright lying and being blindly partisan, try not outright lying and not being blindly partisan. Crazy idea, huh?

Spreading this shitty argumentation in a place that had otherwise been quite clean, that's what's gotten under my skin.

This is not a good argumentation, at all. "It use to be fine, until I was offended by that". It was never really fine. At first, the politics were pretty much prohibited as a "mindkiller", this was the rule of the game here. Then the standard PC views became accepted, as a kind of a default. Then some reactionaries put their views on a display and shortly after went away. Now, the unspoken norm is to not go too far away from the PC platform, again?

This is utterly LUDICROUS.

Look at what happened. tukabel wrote a post of rambling, grammar-impaired, hysteria-mongering hyperbole: 'invading millions of crimmigrants that may rob/rape/kill you anytime day or night'.This is utterly unquestionably NOT a rationally presented point on politics, and it does not belong on this forum, and it deserves to be downvoted into oblivion.

Stuart said he wished to be able to downvote it.

Then out of nowhere you come in and blame him personally or starting something he manifestly didn't start. It's a 100% false comment.

Upon ... (read more)

Well, they are in fact robbing, killing, and raping people, and the authorities are remarkably uninterested in doing anything about it besides accusing the victims of "racism". In fact in most western European countries some who says something mean about the migrants gets a harsher sentence then a migrant who engages in robbing, killing, or raping. Why not? Because he said something false? A better question is why you refer to the truth as "hysteria-mongering hyperbole"?
That much anger, for what? What does it mean?

The main difference I see with nuclear weapons is that if neither side pursues them then you end up in much the same place as if it's very close, except that you have spent a lot on it.

While on AI, the benefits would be huge unless the failure is equally drastic.

Seems to me like Daniel started it.

Perhaps. But doesn't matter who started. Stuart Armstrong has a lot of excellent posts about AI. But every now and then he thinks, that he should do some politics. Which is also a good decision. But then he argues from a (liberal) default, which is not as clever as his AI related views. By far. This is from my POV, of course.

This seems to be more about human development than AI alignment. The non-parallels between these two situations all seem very pertinent.

What would a natural choice of 0 be on that log? I would nominate bare subsistence income, but then any person having less than that would completely wreck the whole thing.

Maybe switch to inverse hyperbolic sine of income over bare subsistence income?

Quite - I have a 10 year old car and haven't had to do anything more drastic than change the battery - regular maintenance kinds of stuff.

This is about keeping the AI safe from being altered by bad actors before it becomes massively powerful. It is not an attempt at a Control Problem solution. It could still be useful.

A) the audit notion ties into having our feedback cycles nice and tight, which we all like here.

B) This would be a little more interesting if he linked to his advance predictions on the war so we could compare how he did. And of course if he had posted a bunch of other predictions so we could see how he did on those (to avoid cherry-picking). That would rule out rear-view-mirror effects.

We may be able to get part of the way there. I found the following suspiciously prediction-like (and maybe even testable!) statements by Ctrl-Fing the pre-invasion posts on D-Squared's blog. From October 21, 2002 []: February 20, 2003 []: This February 26, 2003 post [] doesn't explicitly make predictions, but it's clearly written from the premise that the Bush administration would "completely fuck[] up" "the introduction of democracy to Iraq". Compare the end of the footnote on this February 5, 2003 post []. There might be empirical claims relating to WMD in later posts. Such might still count as predictions because the amount of WMD to be found in Iraq remained contentious for some time after the invasion.
Strong agreement on (B).

Really? There seems a little overlap to me, but plenty of mismatch as well. Like, MM says Bayesians are on crack, as one of the main points of the article.

Agreed on that last point particularly. Especially since, if they want similar enough things, they could easily cooperate without trade.

Like if two AIs supported Alice in her role as Queen of Examplestan, they would probably figure that quibbling with each other over whether Bob the gardener should have one or two buttons undone (just on the basis of fashion, not due to larger consequences) is not a good use of their time.

Also, the utility functions can differ as much as you want on matters aren't going to come up. Like, Agents A and B disagree on how awful many bad things are. Both agree that they are all really quite bad and all effort should be put forth to prevent them.

An American Rationalist subculture question, perhaps. Certainly NOT America as a whole.

You say all excuses are equally valid and then turn around and say they're more or less valid. Do you mean that excuses people would normally think of making have a largely overlapping range of possible validities?

The validity of an excuse is entirely circumstantial. There could be valid times where, "I don't feel like it" is a good excuse and valid times where, "literally have a broken leg" is not a valid excuse. The same excuse is all parts valid and invalid. Excuses do not "excuse" the way they might be used. They lack explanation power and they don't make anyone feel better to hear them. They are just really bad ways to think about problems. And have all this sneaky brain stuff that let's you think you are justified or safe with making excuses. Have you ever been on the receiving end of an invalid excuse and felt better about your situation? How about a valid excuse? By who's judgement was that excuse valid or invalid? If only by the person giving an excuse is an excuse valid it fails to be useful to the rest of the world. And moreso, if your excuses are only valid to yourself and they arise from your internal voice telling you the reasons why; who is winning from your internal delusion?

Well, if the laws of the universe were such that it were unlikely but not impossible for life to form, MWI would take care of the rest, yes.

BUT, if you combine MWI with something that sets the force laws and particle zoo of the later universe as an aspect of quantum state, then MWI helps a lot - instead of getting only one, it makes ALL† of those laws real.

† or in case of precise interference that completely forces certain sets of laws to have a perfectly zero component, nearly all. Or if half of them end up having a precisely zero component due to some sy... (read more)

MWI is orthogonal to the question of different fundamental constants. MWI is just wavefunction realism plus no collapse plus 'that's OK'.

So, any quantum-governed system that generates local constants will do under MWI. The leading example of this would be String Theory.

MWI is important here because if only one branch is real, then you need to be just as lucky anyway - it doesn't help unless the mechanism makes an unusually high density of livable rules. That would be convenient, but also very improbable.

Thanks, Luke Can you clarify? The first part sounds like MWI is irrelevant to the question of fine-tuning of universal constants. Are you saying that if only one Everett branch was real, then it would be unlikely to have things like a planet under the right circumstances for life, but that is accounted for by MWI, since it explores all the permutations of a universe with constants like ours? If I'm getting this, then that means MWI accounts for things like "why is the earth in the right place" kinds of things, but not "why is the proton this particular mass" kinds of things

Why should you believe any specific conclusion on this matter rather than remain in doubt?

Your first sentence, for example, has a lot of parts, and uses terms in unusual ways, and there are multiple possible interpretations of several parts. The end effect is that we don't know what you're saying.

I suspect that what you're saying could make sense if presented more clearly, and it would not seem deep or mysterious. This would be a feature, not a bug.

My wishing for the world is intellectual masturbation, so my practical actions in this consensus reality matter the most (instrumental rationality). But if thinking stops (epistemic rationality by persistent non-symbolic experiences) I do not care in a sense, I go insane in relation to the consensus reality but sane to the non-symbolic way of being. So the way to solve this is to have a good system to remember me of my chores, goals, and choices which we would call rationality in the consensus reality. Otherwise, I might simply no longer be efficient from what I learn of the consensus reality. My memory might even be impaired. Some think that the way for us to return to these states is by AGI and simply overcoming the limits of the human brain, but humans have done it for thousands of years, possibly with more ease. See this article, Ben Goertzel is doing the interview: [] So what I think that I want is a persistent non-symbolic state, symbols make no sense, it's a bit Orwellian. But empirical feeling, indiscriminate love and so on makes a lot of sense. Of course, everything will function as it used to be ('I'-thought have never existed in the first place), but it will still be different. But from the place I am, I need (and I think humanity) need some system in which the computer keeps a track of what my goals and so on were before the persistent non-symbolic state. This beautifully falls into a nice merging with machines, I think, let that which is unconscious, and always will be (machines), be our thinking, for we are non-symbolic I think. :)

Seems more like trying to clarify the hypothetical. There's a genuine dependency here.

Better answer: they would need to demonstrate experiencing subjective time, such as by flavor-oscillating.

Which they do.

Which is why we think they have mass.

I haven't been following these threads, so I didn't even realize they were weekly. I'll take a look.

Be brave and good luck!

There ought to be one fundamental set of rules. This fundamental set of rules may or may not shake out into different local sets of rules. Assuming that these local rulesets arise from aspects of quantum state, then MWI is capable of realizing an arbitrarily large number of them.

String Theory, for instance, has a mindbogglingly large number of wildly varying possible local rulesets - 'compactifications'. So, if String Theory is correct, then yes, this is taken care of unless the number of compactifications yielding rules even vaguely like ours is unexpectedly small.

Okay, but the best theory, MWI, does not suggest different constants, and the theory that does is not particularly well thought of, am I understanding this right? So, this is a bad rebuttal to the fine-tuning argument.

This would benefit from some more concrete examples, especially after 'No, not even "What if that person was me."'

What sort of screwups are we liable to make from these extrapolations?

Seems a bit harsh, though after you've debated a few creationists, it doesn't seem so unsupportable.

OK, I had dropped this for a while, but here are my thoughts. I haven't scrubbed everything that could be seen through rot13 because it became excessively unreadable

For Part 1: gur enqvhf bs gur pragre fcurer vf gur qvfgnapr orgjrra bar bs gur qvnzrgre-1/2 fcurerf naq gur pragre.

Gur qvfgnapr sebz gur pragre bs gur fvqr-fcurer gb gur pragre bs gur birenyy phor vf fdeg(A)/4. Fhogenpg bss n dhnegre sbe gur enqvhf bs gur fcurer, naq jr unir gur enqvhf bs gur pragre fcurer: (fdeg(A)-1)/4. Guvf jvyy xvff gur bhgfvqr bs gur fvqr-1 ulcrephor jura gung'f rdhny gb n... (read more)

Part 1 is good.
Load More