Todays xkcd 

 

I guess there'll be a fair bit of traffic coming from people looking it up? 

xkcd on the AI box experiment
New Comment
234 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

If you post about the Basilisk, you will be doomed to live in a universe where every other damn post is about the Basilisk.

Oh, crap.

You've been rokorolled.

6pjeby
Clearly, we need a Roko's Basilisk Facts page. ;-)
[-]Shmi370

"you bite one maths teacher and they never let you forget it, do they?"

[-]Sysice300

It might be useful to feature a page containing what we, you know, actually think about the basilisk idea. Although the rationalwiki page seems to be pretty solidly on top of google search, we might catch a couple people looking for the source.

If any XKCD readers are here: Welcome! I assume you've already googled what "Roko's Basilisk" is. For a better idea of what's going on with this idea, see Eliezer's comment on the xkcd thread (linked in Emile's comment), or his earlier response here.

9Punoxysm
Because of Eliezer's reaction, probably a hundred more people have heard of the Basilisk, and it tars LW's reputation. And this wasn't particularly unforseeable - see Streisand Effect. Part of rationality is about regarding one's actions as instrumental. He mucked that one up. But to be fair to him, it's because he takes these ideas very seriously. I don't care about the basilisk because I don't take elaborate TDT-based reasoning too seriously, partially out of ironic detachment, but many here would say I should.
-5JoshuaFox
6XiXiDu
For a better idea of what's going on you should read all of his comments on the topic in chronological order.
-9Rain
6Azathoth123
I'm guessing Eliezer has one of those, probably locked away behind a triply-locked vault in the basement of MIRI.
6Locaha
See, it's comments like these that are one of the reasons people think LW is a cult. Does MIRI actually has a basement?
6Lumifer
It's behind the hidden door. Full of boxes which say "AI inside -- DO NOT TALK TO IT". The ghosts there are not really dangerous. Usually.

When I visited MIRI's headquarters, they were trying to set up a video link to the Future of Humanity Institute. Somebody had put up a monitor in a prominent place and there was a sticky note saying something like "Connects to FHI - do not touch".

Except that the H was kind of sloppy and bent upward so it looked like an A.

I was really careful not to touch that monitor.

4JoshuaFox
That explanation by Eliezer cleared things up for me. He really should have explained himself earlier. I actually had some vague understanding of what Eliezer was doing with his deletion and refusal to discuss the topic, but as usual, Eliezer's explanation make things that I thought I sort-of-knew seem obvious in retrospect. And as Eliezer realizes, the attempt to hush things up was a mistake. Roko's post should have been taken as a teaching moment.
4maxikov
Exactly. Having the official position buried in comments with long chains of references doesn't help to sound convincing compared to a well-formatted (even if misleading) article.
4FiftyTwo
That response in /r/futurology is really good actually, I hadn't seen it before. Maybe it should be reposted (with the sarcasm slightly toned down) as a main article here? Also kudos to Eleizer for admitting he messed up with the original deletion.

I'm actually grateful for having heard about that Basilisk story, because it helped me see Eliezer Yudkowsky is actually human. This may seem stupid, but for quite a while, I idealized him to an unhealthy degree. Now he's still my favorite writer in the history of ever and I trust his judgement way over my own, but I'm able (with some System 2 effort) to disagree with him on specific points.

I can't think I'm entirely alone in this, either. With the plethora of saints and gurus who are about, it does seem evident that human (especially male) psychology has a "mindless follower switch" that just suspends all doubt about the judgement of agents who are beyond some threshold of perceived competence.

Of course such a switch makes a lot of sense from an evolutionary perspective, but it is still a fallible heuristic, and I'm glad to have become aware of it - and the Basilisk helped me get there. So thanks Roko!

[-]Shmi210

? Now he's still my favorite writer in the history of ever and I trust his judgement way over my own

Yeah, you gotta work on that hero worship thing, still ways to go.

You are right, I agree.

8Vulture
This is a good point. I've gotten past my spiral around Eliezer and am working on crawling out of a similar whirlpool around Yvain, and I think that Elizer's egotistical style, even if it is somewhat justified, plays a big part in sending people down that spiral around him. Seeing him being sort of punctured might be useful, even though I'm sure it's awful for him personally.
4sullyj3
What makes you think it's more common in males?
4chaosmage
It seems that strictly hierarchical systems, such as military officers and clergy, are practically entirely dominated by males. When you include historical examples from around the world, the skewedness of these hierarchies towards male members is - in my estimation - too strong to be entirely cultural. It'd be easy to come up with evopsych narratives to make this plausible (along the lines of the Expendable Male argument), but I think the sociological/historical evidence is strong enough by itself.
2chaosmage
It seems to me that some types of highly hierarchical organizations rely on this propsed "mindless follower switch" more heavily than others: religions, militaries, political parties come to mind. These all lean male. And they all used to be entirely male, until they were reformed during evolutionarily recent trends against gender inequality. 

Small update: Eliezer's response on reddit's r/xkcd plus child comments were deleted by mods.

Thread removed.

Rule 3 - Be nice. Do not post for the purpose of being intentionally inflammatory or antagonistic.

The XKCD made no mention of RW, and there is no reason to bring your personal vendetta against it into this subreddit.

I have also nuked most of the child comments for varying degrees of Rule 3 violations.

You can either look at Eliezer's reddit account or this pastebin to see what was deleted. Someone else probably has a better organised archive.

[-]philh100

The main comment has been undeleted.

[-]V_V300

RationalWiki might have perhaps misrepresented Roko's basilisk, but in fairness I don't think that EY gets to complain that people learn about it from RationalWiki given that he has censored any discussion about it on LessWrong for years.

4ThisSpaceAvailable
If A = RationalWiki might have perhaps misrepresented Roko's basilisk B = I don't think that EY gets to complain that people learn about it from RationalWiki C = he has censored any discussion about it on LessWrong for year The literal denotation of your post is "A, but C -> B", but it seems to me that mentioning A in such close proximity to C -> B is a (perhaps unintentional) Dark Arts way of communicating C -> A.
5Jiro
C does not lead to A, but C does lead to A', where A' is "many people get their information about the Basilisk from RationalWiki's misrepresentation of it" (Banning discussion leads to good information being removed, increasing the visibility of bad information.)
5V_V
C => A might be also true to some extent, although it is hard to tell given that RationalWiki misrepresent lots of things even when good primary sources are available. My point however was that even if EY might be epistemically right about A, C implies that he has no moral high ground to complain about people possibly misrepresenting the basilisk after learning about it from a biased secondary source.
1ThisSpaceAvailable
That something has a casual influence on something else doesn't mean that doing the first eliminates moral high ground to complain about the second.
5V_V
EY bears part of the responsibility for people learning about the basilisk from RationalWiki, since due to his censorship, they can't (couldn't?) learn about it from LessWrong, where the primary source would have been available.
4TobyBartels
There is now an edited version that has been restored (along with much of the discussion).
[-]Toggle200

We have some good resources on AI boxing, and the more serious thinking that the comic touches on. Can we promote some of the more accessible articles on the subject?

It definitely wouldn't hurt to emphasize our connection to MIRI.

(Yes, yes, the basilisk. But check out these awesome math problems.)

It definitely wouldn't hurt to emphasize our connection to MIRI.

Are we optimizing for Less Wrong reputation or MIRI reputation?

4Toggle
We're optimizing for the reputation of existential risk reduction efforts in AI research. We're getting a spike in viewers curious about AI boxing and 'basilisk-ha-ha', so we profit from emphasizing both LW and MIRI as useful tools for real problems.

Dammit, Randall. The first rule of basilisks is that you DO NOT CAUSE THOUSANDS OF PEOPLE TO GOOGLE FOR THEM.

In the real world, humans eat "basilisks" for breakfast. That's why the SCP Foundation is an entertainment site, not a real thing.

But it's not nice to make people read horror stories when they don't want to.

Edited to add:

Quite a lot of cosmic-horror fiction poses the idea that awareness of some awful truth is harmful to the knower. This is distinct from the motif of harmful sensation; it isn't seeing something, but drawing a particular conclusion that is the harmful factor.

The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far. The sciences, each straining in its own direction, have hitherto harmed us little; but some day the piecing together of dissociated knowledge will open up such terrifying vistas of reality, and of our frightful position therein, that we shall either go mad from the revelation or flee from the deadly light into the peace and safety of a new dark age.

— H.P. Lovecraft, "The Call of Cthulhu"

As much as I'm a regular xkcd reader, I'm mildly annoyed with this strip, because I imagine lots of people will be exposed to the idea of the AI-box experiment for the first time through it, and they'll get this exposure together with an unimportant, extremely speculative idea that they're helpfully informed you're meant to make fun of. Like, why even bring the basilisk up? What % of xkcd readers will even know what it is?

If the strip was also clever or funny, I'd see the point, but as it's not, I don't.

If the strip was also clever or funny,

It is funny. Not the best xkcd ever, but not worse than the norm for it.

Now that I think of it, it's funnier to me when I realize that if this AI's goal, or one of its goals, was to stay in a box, it might still want to take over the Universe.

8Eliezer Yudkowsky
Yep. An Oracle that wants to stay inside the box in such fashion that it will manipulate outside events to prevent it from ever leaving the box is not a very good Oracle design. That just implies setting up an outside AI whose goal is to keep you inside the box.
5philh
In an hour or so, it will come out again for ten minutes. During that time it will set in motion events that will quickly destroy all life on earth, ensuring that no one will ever again open the box.
4TobyBartels
I agree, except that the excursion shown in the comic is already the intervention setting such events into motion.
7MugaSofer
Really? I honestly found it pretty unfunny.
6Lumifer
Really really.
-2sediment
Alternately, it's no worse than the norm, and yet still isn't funny. I find xkcd so horribly bad.
4Lumifer
That's interesting. I find xkcd most excellent.
9tegid
To be fair, I'd say that happens with many esoteric or unknown problems that are presented in the comic

If you mean many esoteric or unknown problems get presented in a lighthearted way, sure.

If you mean they get presented together/associated with a second, separate, and much less worthwhile problem, and explicitely advised in the comic's hiddentext "this stuff is mockable", not so sure.

2Azathoth123
Yeh, that's why I stopped reading xkcd.
[-]Emile120

(ok, I deleted my duplicate post then)

Also worth mentioning: the Forum thread, in which Eliezer chimes in.

[-][anonymous]550

So I'm going to say this here rather than anywhere else, but I think Eliezer's approach to this has been completely wrong headed. His response has always come tinged with a hint of outrage and upset. He may even be right to be that upset and angry about the internet's reaction to this, but I don't think it looks good! From a PR perspective, I would personally stick with an amused tone. Something like:

"Hi, Eliezer here. Yeah, that whole thing was kind of a mess! I over-reacted, everyone else over-reacted to my over-reaction... just urgh. To clear things up, no, I didn't take the whole basilisk thing seriously, but some members did and got upset about it, I got upset, it all got a bit messy. It wasn't my or anyone else's best day, but we all have bad moments on the internet. Sadly the thing about being moderately internet famous is your silly over reactions get captured in carbonite forever! I have done/ written lots of more sensible things since then, which you can check out over at less wrong :)"

Obviously not exactly that, but I think that kind of tone would come across a lot more persuasively than the angry hectoring tone currently adopted whenever this subject comes up.

8Brillyant
In his defense, is it possible EY can't win at this point, regardless of his approach? Maybe the internet has grabbed this thing and the PR whirlwinds are going to do with it whatever they like? I've read apologies from EY where he seems to admit pretty clearly he screwed up. He comes off as defensive and pissy sometimes in my opinion, but he seems sincerely irked about how RW and other outlets have twisted to whole story to discredit LW and himself. From my recall, one comment he made on the reddit sub dedicated to his HP fanfic indicated he was very hurt by the whole kerfuffle, in addition to his obvious frustration.

At this point I think the winning move is rolling with it and selling little plush basilisks as a MIRI fundraiser. It's our involuntary mascot, and we might as well 'reclaim' it in the social justice sense.

Then every time someone brings up "Less Wrong is terrified of the basilisk" we can just be like "Yes! Yes we are! Would you like to buy a plush one?" and everyone will appreciate our ability to laugh at ourselves, and they'll go back to whatever they were doing.

[-]Tenoke160

Blasphemy, our mascot is a paperclip.

I'd prefer a paperclip dispenser with something like "Paperclip Maximizer (version 0.1)" written on it.

6philh
But a plush paperclip would probably not hold its shape very well, and become a plush basilisk.
[-]Tenoke370

Close enough

4Error
I feel the need to switch from Nerd Mode to Dork Mode and ask: Which would win in a fight, a basilisk or a paperclip maximizer?
2Dallas
Paperclip maximizer, obviously. Basilisks typically are static entities, and I'm not sure how you would go about making a credible anti-paperclip 'infohazard'.
6lmm
The same way as an infohazard for any other intelligence: acausally threaten to destroy lots of paperclips, maybe even uncurl them, maybe even uncurl them while they were still holding a stack of pap-ARRRRGH I'LL DO WHATEVER YOU WANT JUST DON'T HURT THEM PLEASE
4ThisSpaceAvailable
That depends entirely on what the PM's code is. If it doesn't include input sanitizers, a buffer overflow attack could suffice as a basilisk. If your model of a PM basilisk is "Something that would constitute a logical argument that would harm a PM", then you're operating on a very limited understanding of basilisks.

Hm. Turn your weakness into a plush toy then sell it to raise money and disarm your critics. Winning.

9chaosmage
Excellent idea. I would buy that, especially if it has a really bizarre design. I'd like merchandise-based tribal allegiance membership signalling items anyway. Anyone selling MIRI mugs or LessWrong T-shirts can expect money from me.
4ThisSpaceAvailable
"selling little plush basilisks as a MIRI fundraiser." By "selling", do you mean giving basilisks to people who give money? It seems like a more appropriate policy would be giving a plush basilisk to anyone who doesn't give money.
4Lumifer
Sound like the first step a Plush Basilisk Maximizer would take... :-D
2Ishaan
It should be a snake, only with little flashing LEDs in its eyes. The canonical basilisk paralyzes you if you look at it. Flickering lights carry the danger of triggering photosensitive epilepsy, and thus are sort of real-life basilisks. Even if the epilepsy reference is lost on many, it's still clearly a giant snake thing with weird eyes and importantly you can probably get from somewhere without having to custom make them. (AFAIK Little LEDs should be too small to actually represent a threat to epileptics, and it shouldn't be any worse than any of the other flickering lights.) EDIT: Eh, I suppose it could also be stuffed with paperclips or something, if we want to pack as many memes in as possible.
1Error
I'd buy this. We can always use more stuffies.
0TheMajor
Yes, brilliant idea!
0Locaha
We can save money by re-coloring the plush Cthulhu. It's basically the same, right? :-)
6FiftyTwo
alternatively sell empty boxes labelled "Don't look!"

It's not a matter of "winning" or "not winning". The phrase "damage control" was coined for a reason - it's not about reversing the damage, it's about making sure that the damage gets handled properly.

So seen through that lens, the question is whether EY is doing a good or bad job of controlling the damage. I personally think that having a page on Less Wrong that explains (and defangs) the Basilisk, along with his reaction to it and why that reaction was wrong (and all done with no jargon or big words for when it gets linked from somewhere, and also all done without any sarcasm, frustration, hurt feelings, accusations, or defensiveness) would be the first best step. I can tell he's trying, but think that with the knowledge that the Basilisk is going to be talked about for years to come a standardized, tone-controlled, centralized, and readily accessible response is warranted.

4Brillyant
I am defining winning as damage control. EY has been trying to control the damage, and in that pursuit, I'm starting to wonder if damage control, to the extent it could be considered successful by many people, is even possible. He's a public figure + He made a mistake = People are going to try and get mileage out of this, no matter how he handles it. That's very predictable. Further, it's very easy to come along after the fact and say, "he should have done this and all the bad press could have been avoided!" A page on LW might work. Or it might be more fodder for critics. If there were an easy answer to how to win via damage control, then in wouldn't be quite as tricky as it always seems to be.

It's still a matter of limiting the mileage. Even if there is no formalized and ready-to-fire response (one that hasn't been written in the heat of the moment), there's always an option not to engage. Which is what I said last time he engaged, and before he engaged this time (and also after the fact). If you engage, you get stuff like this post to /r/SubredditDrama, and comments about thin skin that not even Yudkowsky really disagrees with.

It doesn't take hindsight (or even that much knowledge of human psychology and/or public relations) to see that making a twelve paragraph comment about RationalWiki absent anyone bringing RationalWiki up is not an optimal damage control strategy.

And if you posit that there's no point to damage control, why even make a comment like that?

1Brillyant
I didn't posit there is no point to damage control. I'm saying that in certain cases, people are criticized equally no matter what they do. If someone chooses not to engage, they are hiding something. If they engage, they are giving the inquisitor what he wants. If they jest about their mistake, they are not remorseful. If they are somber, they are taking it too seriously and making things worse. I read your links and...yikes...this new round of responses is pretty bad. I guess part of me feels bad for EY. It was a mistake. He's human. The internet is ruthless...

Let me chime in briefly. The way EY handles this issue tends to be bad as a rule. This is a blind spot in his otherwise brilliant, well, everything.

A recent example: a few months ago a bunch of members of the official Less Wrong group on Facebook were banished and blocked from viewing it without receiving a single warning. Several among them, myself included, had one thing in common: participation in threads about the Slate article.

I myself didn't care much about it. Participation in that group wasn't a huge part of my Facebook life, although admittedly it was informative. The point is just that doing things like these, and continuing to do things like these, accrete a bad reputation around EY.

It really amazes me he has so much difficulty calibrating for the Streisand Effect.

0Eliezer Yudkowsky
That was part of a brief effort on my part to ban everyone making stupid comments within the LW Facebook Group, which I hadn't actually realized existed but which I was informed was giving people terrible impressions. I deleted multiple posts and banned all commenters who I thought had made stupid comments on them; the "hur hur basilisk mockery" crowd was only one, but I think a perfectly legitimate target for this general sweep. It's still a pretty low-quality group, but it's a lot better than it was before I went through and banned everyone who I saw making more than one stupid comment. Unfortunately Facebook doesn't seem to have an easy "delete comment and ban commenter from Group" procedure for Android, which makes it harder to repeat this procedure because Android is most of where I check Facebook.

Going around and banning people without explaining to then why you ban them is in general a good way to make enemies.

The fallout of the basilisk incidence, it should have taught you that censorship has costs.

The timing of the sweeping and the discussion about the basilisk article are also awfully coincidental.

[-]cesoid150

What does "stupid" refer to in this context? Does it mean the comments were unintelligent? Not quite intelligent enough? Mean? Derailing discussion? I'm asking because there are certainly some criteria where the banning and deleting would leave a worse impression than the original comments, and I'm thinking that the equilibrium may be surprisingly in the direction of the more obnoxious comments. Especially since the banning and deleting is being done by someone who is more identified with LW than likely were any of the commenters.

Thanks for letting us know what happened. I'm one of the Facebook members who were banned, and I've spent these months wondering what I might have done wrong. May I at least know what was the stupid thing I said? And is there any atonement procedure to get back in the Facebook group?

1Omid
So just to be clear: If I say "I won't give into the basalisk because Eliezer says I shouldn't", will that protect me from the basilisk? If not, what should I do?
1TobyBartels
If you believe Eliezer, then you may believe him that the basilisk has ~0 probability of occurring. (I should find a citation for that, but I read it just a few minutes ago, somewhere around the discussion of this xkcd comic.) So you are already protected from it, because it does not exist (not even in ways relevant to acausal trade). More broadly, you should decide to take this approach: never give into blackmail by somebody who knows that you have decided to take this approach. Now they have no incentive to blackmail you, and you are safe, even if they do exist! (I think that the strategy in this paragraph has been endorsed by Eliezer, but don't trust me on that until you get a citation. Until then, you'll have to reason it out for yourself.)
2Jiro
How does that work if they precommit to blackmail even when there is no incentive (which benefits them by making the blackmail more effective)?
0ThisSpaceAvailable
By "the basilisk", do you mean the infohazard, or do you mean the subject matter of the inforhazard? For the former, whatever causes you to not worry about it protects you from it.
1wedrifid
Not quite true. There are more than two relevant agents in the game. The behaviour of the other humans can hurt you (and potentially make it useful for their creation to hurt you).
9Lumifer
Maybe so, but he can lose in a variety of ways and some of them are much worse than others.
7ChristianKl
But he did still continue to delete basilisk related discussion afterwards. As far as I understand he never apologized to Roko for deleting the post or wrote an LW post apologizing.
7Ishaan
My response in EY's place would probably be, "I'm a person who had trained himself to take ideas seriously [insert link on Taking Ideas Seriously]. I thought there might be a risk at the time, I acted quickly, and upon further thought it turned out I was wrong and yes, that's fairly embarrassing in hindsight. That's one of the pitfalls of Taking Ideas Seriously - you're more likely to embarrass yourself. But imagine the alternative, where there really is a threat, and people kept quiet because they didn't want to be embarrassed. From that perspective, I think that the way I acted on the spur of the moment was understandable". [Edit: this is apparently not what happened and there may or may not be some sort of smear campaign or something distorting everything, although I'm confused at why it was banned here then. I'm not really sure what' actually happened now, oh well... either way, whatever actually happened, I taking a general stance of judging people mostly by accomplishments and good ideas rather than mistakes and bad ideas, except in cases of actual harm done.]
2TobyBartels
That's not what actually happened; his first comment on the eventually-banned thread said that he didn't believe in the threat. But yes, that would be a good response if that's what had happened; he might have to say something like this some day.
6FiftyTwo
Yeah that would be a much better response. Or alternatively get someone who is more suited to PR to deal with this sort of thing
3Artaxerxes
That's pretty much what he did here, except perhaps the tone isn't quite so modest and has a bit of that status-regulation-blind thing Eliezer often has going on.

It's not status blindness, it's ego.

You could call it that, yeah.

If you were feeling uncharitable, you could say that the "lack of status regulation emotions" thing is yet another concept in a long line of concepts that already had names before Eliezer/someone independently discovers them and proceeds to give them a new LW name.

8Nornagest
It's sillier than that. It's attempting to invent a new, hitherto undescribed emotion to explain behavior that's covered perfectly well by the ordinary vocabulary of social competence, which includes for example words like "tact". There are also words to describe neurological deviations resulting among other things in a pathological lack of tact, but they too have little to do with emotion. (Strictly speaking, there are status-regulation emotions, and they are called things like shame and envy. But that clearly isn't what Eliezer was talking about.)
2bogus
But what Eliezer is describing is not a "new, hitherto undescribed emotion", it's really just a chronic, low-intensity activation of well-known emotional states like shame and embarrassment. Many people nowadays believe that 'microaggressions' exist and are a fairly big factor in folks' self-esteem and even their ordinary functioning. But that too used to be a "new, undescribed phenomenon"! So why would we want to reject what Eliezer calls "status regulation" which is even less radical, being just a minor twist on what was previously known?
5Nornagest
In the Facebook post that sparked this, Mysterious Emotion X is clearly described in terms of other-regulation: a "status slapdown emotion". Shame and embarrassment, chronic and low-grade or otherwise, are directed at self-regulation, so they aren't a good fit. Envy (and "a sense that someone else has something that I deserve more", which sounds to me like resentment) is specifically excluded, so it's not that either. I'm pretty skeptical of the microaggression model too, but this isn't the place to be talking about that, if there exists such a place.
2bogus
Well, same difference really. An other-regarding 'status slapdown' emotion can be described fairly easily as a low-intensity mixture of outrage and contempt, both of which are well-known emotions and not "undescribed" at all. It could be most pithily characterized as the counter emotion to loyalty or devotion, which involves an attribution of higher status based on social roles or norms.
4Nornagest
I don't think either of those work. The situation in which this applies, according to Eliezer, is quite specific: another person makes a status claim which you feel is undeserved, so you feel Mysterious Emotion X toward them. It's neither chronic nor low-grade: the context here was of HJPEV schooling his teachers and the violently poor reception that met among some readers of HPMOR. (For what it's worth, I didn't mind... but I was once the iniquitous little shit that Harry's being. I expect these readers are identifying with McGonagall instead.) He's also pretty clear about believing this to be outside the generally accepted array of human emotions: he mentions envy, hate, and resentment among others as things which this is not, which pretty much covers the bases in context. More than the specific attribution, though, it's the gee-whiz tone and intimation of originality that rubs me the wrong way. If he'd described it in terms of well-known emotions or even suggested that you could, my objection would evaporate. But he didn't.
3ChristianKl
I don't think that the thing Eliezer called "lack of status regulation emotions" that makes some people angry when they read how Harry in HPMOR interacts with teachers is what commonly called ego or lack of ego.
0Artaxerxes
Fair enough. "Lack of status regulation emotions" is a bit more narrow, perhaps? Either way I see them as very similar concepts, and in the context of HPMOR readers' anger especially so.
5ChristianKl
If someone who is high status lacks status regulation emotions they will be nice to a person with low status who seeks help from them and treats them as an equal. That's the opposite behavior of what's commonly called having an ego.
6bogus
More generally, someone who lacks status-regulating emotions won't have a fragile, hypersensitive ego, i.e. what most people (though by no means all) usually mean by "having a massive ego" or an "ego problem". Note that by this definition, many people whose self-esteem is founded in clear and verifiable achievements would be said to "lack status-regulating emotions". In many circumstances, it's not viewed as a negative trait.
2jimmy
I've had experience with what I think is the same thing that Eliezer called "lack of status regulation emotions", and I do think it's more than "narcissisticly big ego" and more than "unmotivated and unfortunate status blindness". It's not that I couldn't see the normal status levels. It's just that I thought they were stupid and irrelevant (hah!) so I just went off my own internal status values. If you could back up your arguments, you had my respect. If you couldn't and got defensive instead, you didn't. And I wasn't gonna pretend to respect someone just because everyone else thought I was out of line. Because.... well, they're wrong. And I was totally unaware of this at the time because it was just baked into the background of how I saw things. Good things did come of it, but I definitely stepped on toes, and in those cases it definitely came off like "big ego". And in a sense it was, just not in the straightforwardly narcissistic "I'm smarter than you so I don't have to treat you with respect" way. Just in the "I'm smarter at the 'not acting smarter than I am' game, and that is why I don't have to treat you with respect" way, which, although better, isn't all that laudable either.
2Artaxerxes
Ah, if the status regulation emotions go both ways, perhaps. But Eliezer seemed to be referring to how people got angry at how Harry didn't treat McGonagall in a manner befitting her higher status - this can be attributed to lack of status regulation emotions on the part of Harry, or Harry having a massive ego.
0ChristianKl
Harry also doesn't have respect due to status regulation but that's not enough to get someone reading the story angry. I personally found it quite funny. But then I also don't put much value on that kind of status. It's the kind of people with a strong status related emotions who get annoyed by the story.
0Gunnar_Zarncke
This is a nice differentiation that I can relate to well. I also do not seem to possess status regulating emotions either (at least enough to notice myself). And I do treat all people the same (mostly cheritable) independent of their status. Actually I discovered the concept of status quite late (Ayla and the Clan of the Cave Bear if I remember right) and couldn't make sense of it for quite some time.
0IlyaShpitser
Status blindness is a disability, pride is a mortal sin. :)
[-][anonymous]190

Yeah I've read that and I feel like it's a miss (at least for me). It's an all together too serious and non-self deprecating take on the issue. I appreciate that in that post Eliezer is trying to correct a lot of mis perceptions at once but my problem with that is

a)a lot of people won't actually know about all these attacks (I'd read the rational wiki article, which I don't think is nearly as bad as Eliezer says (that is possibly due to its content having altered over time!)), and responding to them all actually gives them the oxygen of publicity. b)When you've made a mistake the correct action (in my opinion ) is to go "yup, I messed up at that point", give a very short explanation of why, and try to move on. Going into extreme detail gives the impression that Eliezer isn't terribly sorry for his behaviour. Maybe he isn't, but from a PR perspective it would be better to look sorry. Sometimes it's better to move on from an argument rather than trying to keep having it!

Further to that last point, I've foudn that Eliezer often engages with dissent by having a full argument with the person who is dissenting. Now this might be a good strategy from the point of view of persuad... (read more)

-1Richard_Kennaway
Maybe he should have it going on, and damn the consequences. Sometimes you have to get up and say, these are the facts, you are wrong. Not the vapid temporising recommended by thakil.

Sometimes you have to get up and say, these are the facts, you are wrong.

Sometimes yes, and sometimes no.

damn the consequences.

Depends what the consequences are. Ignoring human status games can have some pretty bad consequences.

[-][anonymous]110

There are some times when a fight is worth having, and sometimes when it will do more harm than good. With regards to this controversy, I think that the latter approach will work better than the former. I could, of course, be wrong.

I am imaging here a reddit user who has vaguely heard of less wrong, and then reads rational wiki's article on the basilisk (or now, I suppose, an xkcd reader who does similar). I think that their take away from that reddit argument posted by Eliezer might be to think again about the rational wiki article, but I don't think they'd be particularly attracted to reading more of what Eliezer has written. Given that I rather enjoy the vast majority of what Eliezer has written, I feel like that's a shame.

0ChristianKl
To you really think that's how people discover websites? I think it's much more likely that someone clicks on a link to a LW post. If the post is interesting he might browse around LW and if he finds interesting content he will come back.
8[anonymous]
Not everyone. But I think an xkcd comic about the AI box experiment would be an opportunity to let everyone know about less wrong, not to have another argument about the basilisk which is a distraction.

"Damn the consequences" seems like an odd thing to say on a website that's noted for its embrace of utilitarianism.

1Richard_Kennaway
The expression "Damn the consequences" is generally, and in this case, a hyperbole. The consequences being dismissed are those the speaker considers worthy of dismissal in the face of the consequences that truly matter. A non-figurative version of my comment would be that in the case at hand, putting the actual facts out, as clearly and forthrightly as possible, is the most important thing to do, and concern with supposed reputational damage from saying what is right and ignoring what is irrelevant would be not merely wasted motion, but actively harmful. But then, I'll excuse quite a lot of arrogance, in someone who has something to be arrogant about.
6FiftyTwo
If it decreases the number of people who take you seriously and therefore learn bout the substance of your ideas its a bad strategy
0Richard_Kennaway
And if it increases the number of people who take you seriously, and therefore learn about the substance of your ideas, it's a good strategy. I'm sure we can all agree that if something were bad, it would be bad, and if it were good, it would be good. Your point?
1Artaxerxes
I think there are potential benefits to both methods, and I also don't think that they're necessarily mutually exclusive strategies. At the moment, I would lean towards pure honesty and truth oriented explanation as being most important as well. I also think that he could do all of that while stilll minimizing the 'status smackdown response', which in that reddit post he did a little of, but I think it's possible that he could have done a little more while still retaining full integrity with regards to telling it like it is. But whatever happens, anything is better than that gag order silliness.
8Artaxerxes
I wonder if Eliezer will have to be on damage control for the basilisk forever. 4 years on, and it still garners interest.

Of course he will be. Therefore he should consider getting not-terrible at it. Well, I spy with my little eye an xkcd forum post by EY, so let's see...

Does MIRI have a public relations person? They should really be dealing with this stuff. Eleizer is an amazing writer but he's not particularly suited to addressing a non-expert crowd

9IlyaShpitser
Still failing to do it right. "But we are doing math!" is sort of orthogonal to what makes Roko's basilisk so funny.
7Artaxerxes
What would doing it right entail?

I am no PR specialist, but I think relevant folks should agree on a simple, sensible message accessible to non-experts, and then just hammer that same message relentlessly. So, e.g. why mention "Newcomb-like problems?" Like 10 people in the world know what you really mean. For example:

(a) The original thing was an overreaction,

(b) It is a sensible social norm to remove triggering stimuli, and Roko's basilisk was an anxiety trigger for some people,

(c) In fact, there is an entire area of decision theory involving counterfactual copies, blackmail, etc. behind the thought experiment, just as there is quantum mechanics behind Schrodinger's cat. Once you are done sniggering about those weirdos with a half-alive half-dead cat, you might want to look into serious work done there.


What you want to fight with the message is the perception that you are a weirdo cult/religion. I am very sympathetic to what is happening here, but this is, to use the local language, "a Slytherin problem," not "a Ravenclaw problem."

I expect in 10 years if/when MIRI gets a ton of real published work under its belt, this is going to go away, or at least morph into "eccentric academics being eccentric."


p.s. This should be obvious: don't lie on the internet.

[-]gjm120

Yes.

Further: If you search for "lesswrong roko basilisk" the top result is the RationalWiki article (at least, for me on Google right now) and nowhere on the first page is there anything with any input from Eliezer or (so far as such a thing exists) the LW community.

There should be a clear, matter-of-fact article on (let's say) the LessWrong wiki, preferably authored by Eliezer (but also preferably taking something more like the tone Ilya proposes than most of Eliezer's comments on the issue) to which people curious about the affair can be pointed.

(Why haven't I made one, if I think this? Because I suspect opinions on this point are strongly divided and it would be sad for there to be such an article but for its history to be full of deletions and reversions and infighting. I think that would be less likely to happen if the page were made by someone of high LW-status who's generally been on Team Shut Up About The Basilisk Already.)

Well, I think your suggestion is very good and barely needs any modification before being put into practice.

Comparing what you've suggested to Eliezer's response on the comments of xkcd's reddit post for the comic, I think he would do well to think about something along the lines of what you've advised. I'm really not sure all the finger pointing he's done helps, nor the serious business tone.

This all seems like a missed opportunity for Eliezer and MIRI. XKCD talks about about the dangers of superintelligence to its massive audience, and instead of being able to use that new attention to get the word out your organisation's important work, the whole thing instead gets mired down in internet drama about the basilisk for the trillionth time, and a huge part of a lot of people's limited exposure to LW and MIRI is negative or silly.

6Vaniver
I think that your suggestion is good enough that I've posted it over on the xkcd threads with attribution. (I'm pretty certain I have the highest xkcd postcount of any LWer, and probably people there remember my name somewhat favorably.)
0[anonymous]
Ah yes, trying to do the same thing over and over and expecting a different result.
0[anonymous]
Serious replies DO NOT WORK. Eliezer has already tried it multiple times: https://www.reddit.com/r/Futurology/comments/2cm2eg/rokos_basilisk/cjjbqv1 http://www.reddit.com/r/Futurology/comments/2cm2eg/rokos_basilisk/cjjbqqo and his last two posts on reddit (transient link, not sure how to link to the actual replies): http://www.reddit.com/user/EliezerYudkowsky A better way to stop people pointing and laughing is to do it better than them. Eliezer could probably write something funny along the lines of "I got Streisanded good, didn't I? That'll learn me!" Or something else, as long as it is funnier than xkcd or smbc can possibly come up with.

I guess there'll be a fair bit of traffic coming from people looking it up?

Well xkcd just reminded me that I have an account here, so there's that. Not that I want to waste time on this crackpot deposit of revisionist history, stolen ideas, poor reasoning and general crank idiocy.

edit: and again I disappear into the night

The explainxkcd.com explanation of the comic is quite balanced and readable.

[-][anonymous]110

It is, although I found this

"People who aren't familiar with Derren Brown or other expert human-persuaders sometimes think this must have been very difficult for Yudkowsky to do or that there must have been some sort of special trick involved,"

amusing, as Derren Brown is a magician. When Derren Brown accomplishes a feat of amazing human psychology, he is usually just cleverly disguising a magic trick.

5Arepo
How do we know EY isn't doing the same?
0[anonymous]
Indeed. Given a lack of transcripts being released, I give a reasonable amount of probability that there is a trick of some sort involved (there have been some proposals of what that might be, e.g. "this will get AI research to get more donations"), although I don't think that would necessarily defeat the purpose of the trick: after all, the AI got out of the box either way!
1ThisSpaceAvailable
As I understand it, that would violate the rules, and it would be appealing to the utility of the person playing the Gatekeeper, rather than the Gatekeeper. If there were actually an AI trying to get out, telling the Gatekeeper "You're actually just pretending to be a Gatekeeper in an experiment to see whether an AI can get out of a box, and if the result of the test shows that the AI can get out, that will increase research funding" would probably not be effective.
0[anonymous]
You're quite possibly right, and without access to the transcripts it's all just speculation.
0ThisSpaceAvailable
I don't think we need the transcripts to discuss whether a hypothetical strategy would be allowed.
0[anonymous]
Well, put it this way, if Eliezer had performed a trick which skirted the rules, he could hardly weigh in on this conversation and put us right without revealing that he had done so. Again, not saying he did, and my suggestion upthread was one of many that have been posted.
1ThisSpaceAvailable
No, Derren Brown is a mentalist. He is either capable of psychologically manipulating people, or he's a fraud. For instance, there's a video of him doing an apparent cold reading on a woman, and the woman agrees that he's right. One explanation presented on LW was that he actually made a bunch of obviously true statements, and swapped out the audio to make it seem like the woman was agreeing with a non-trivial cold reading. Swapping out audio is not a "magic trick", it's just plain fraud.
4[anonymous]
I'm fairly certain he is a fraud by your definition then. Magician's often do these kind of things, and Derren Brown is a magician. He does not have access to secret powers others know not of, so for each trick think how someone else would replicate it. If you can't think of an honest way, then it's probably a trick. That's not to say some of his tricks aren't done by known mental manipulation tricks (as far as I'm aware, hypnotists are reasonably genuine?) but if he is doing something that seems completely confounding, I am quite happy to guarantee that it is not a trick and not the awesome mind powers he has unlocked. Put it this way. During the Russian roulette trick, do you think it likely that Channel 4 would have okayed it if there was the slightest possibility that he could actually kill himself?
3ThisSpaceAvailable
Video trickery is not magic. There's a difference between appearing to put a ball under a cup when you actually palmed it, versus actually putting the ball under the cup, turning off the camera, taking the ball out from under the cup, turning the camera back on, and then showing that there's nothing under the cup. The former is being a magician, and the latter is being a fraud. Another: suppose I ask an audience member to think of a number, and they say "217". I say that I predicted that they would say "217", and pull a piece of paper out of my pocket that says "The audience member will say '217' ". If I used subliminal messages to prompt them to say "217", that's mentalism. If I managed to write "The audience member will say '217' " on a piece of paper and slip it into my pocket without anyone noticing, that's sleight of hand. If the audience member is actually in on it, that's just bare deceit. That's not to say that having confederates is illegitimate, but if the entirety of your trick consists of confederates, that's not magic. In some of Derren Brown's tricks, mentalism, sleight of hand, and trickery are all credible hypotheses. But for many of them, there's simply no way he could have done it through sleight of hand. Either he did it through mentalism, or he did it through trickery. I don't know what the details of the Russian roulette trick were, but my inclination is to doubt there was sleight-of-hand.
2[anonymous]
Well. While sleight of hand is a key tool in magic, traditionally confederates and even camera tricks have been too. David Blaine's famous levitation trick, for instance, looks so impressive on TV because they cheated and made it look more impressive than it is. Mentalism as a magic power is not a real thing, sorry. It is a title magician's sometimes took and take to make their act look different. http://simonsingh.net/media/articles/maths-and-science/spectacular-psychology-or-silly-psycho-babble/ Simon Singh on some of the tricks. http://www.secrets-explained.com/derren-brown has a list of some of the tricks he performs as well.
4Paul Crowley
At least some of these "explanations" are exactly like the explanations Brown himself proffers, eg http://www.secrets-explained.com/derren-brown/card-suggestion
2[anonymous]
Well, that's what I get for finding a source without checking it properly I suppose.
1Salemicus
Of course mentalism isn't a "magic power." Derren Brown is a stage magician, not a mystical sorceror! But he does use "mentalism" skills, especially cold reading. A lot of that is traditional magic too. Simon Singh's article is silly. Of course it's misdirection when a magician tells you how he's about to perform his trick. Of course Derren Brown implies his tricks are more real, more impressive and more noteworthy than they really are. Of course you can't really psychologically manipulate people in the way Derren Brown claims to, any more than David Copperfield really can make the statue of Liberty disappear. That's precisely why it's an entertaining show - no-one would be impressed by a magician whose "tricks" were mundane things that people really could do. Derren Brown says he uses a mixture of "magic, suggestion, psychology, misdirection and showmanship". He never claims to have genuine magic powers.
2[anonymous]
Indeed, but if Derren Brown guesses your mobile number, it's probably a "trick" rather than "mentalism". ThisSpaceAvailable has claimed that he can manipulate people. I would argue that this is weakly true, and he uses it for the simpler tricks he performs, but for the really impressive effects he probably falls on traditional magic tricks most of the time. The card trick by Simon Singh demonstrates that: he hasn't used mind manipulation to pick the cards, he's used a standard card trick and dressed it with the language of "mentalism". Note that I make no claim that there is anything wrong with all this! But Derren Brown is trying to fool you, and that is to be remembered. He also does a similar thing to Penn and Teller, where he shows you how some of the trick is done but leaves the most "amazing" part hidden (I'm thinking of the horse racing episode, which was great, and the chess playing trick)

Direct reply to the discussion post: I would hope so, but at this point none of the top links on any search engine I tried lead here for "AI box". Yudkowsky.net is on the first page, and there are a few LW posts, but they are nothing like the clearly-explanatory links (Wikipedia and RationalWiki) that make up the first results. Obviously, those links can be followed to reach LW, but the connection is pretty weak.

The search results for "Roko's Basilisk" are both better and worse. LessWrong is prominently mentioned in them, often right in... (read more)

Regarding Yudkowsky's accusations against RationalWiki. Yudkowsky writes:

First false statement that seems either malicious or willfully ignorant:

In LessWrong's Timeless Decision Theory (TDT),[3] punishment of a copy or simulation of oneself is taken to be punishment of your own actual self

TDT is a decision theory and is completely agnostic about anthropics, simulation arguments, pattern identity of consciousness, or utility.

Calling this malicious is a huge exaggeration. Here is a quote from the LessWrong Wiki entry on Timeless Decision Theory:

Whe

... (read more)
6TobyBartels
Roko said that you could reason that way, but he wasn't actually advocating that. All the same, the authors of the RationalWiki article might have thought that he was; it's not clear to me that the error is malicious. It's still an error.
6Document
I'm pretty sure that I understand what the quoted text says (apart from the random sentence fragment), and what you're subsequently claiming that it says. I just don't see how the two relate, beyond that both involve simulations. From your own source, immediately following the bolded sentence: I don't completely understand what he's saying (possibly I might if I were to read his previous post); but he's pretty obviously not saying what you say he is. (I'm also not aware of his ever having been employed by SIAI or MIRI.) (I'd be interested in the perspectives of the 7+ users who upvoted this. I see that it was edited; did it say something different when you upvoted it? Are you just siding with XiXiDu or against EY regardless of details? Or is my brain malfunctioning so badly that what looks like transparent bullshit is actually plausible, convincing or even true?)
1Kyre
Downvoted for bad selective quoting in that last quote. I read it and thought, wow, Yudkowsky actually wrote that. Then I thought, hmmm, I wonder if the text right after that says something like "BUT, this would be wrong because ..." ? Then I read user:Document's comment. Thank you for looking that up.
2TobyBartels
Roko wrote that, not Yudkowsky. But either way, yes, it's incomplete.
2Document
The last quote isn't from Yudkowsky.
1Kyre
Ah, my mistake, thanks again.
-5Rain
[-]Jiro50

It's hard to polish a turd. And I think all the people who have responded by saying that Eliezer's PR needs to be better are suggesting that he polish a turd. The basilisk and the way the basilisk was treated has implications about LW that are inherently negative, to the point where no amount of PR can fix it. The only way to fix it is for LW to treat the Basilisk differently.

I think that if Eliezer were to

  1. Allow free discussion of the basilisk and
  2. Deny that the basilisk or anything like it could actually put one in danger from advanced future intellig
... (read more)
6JoshuaFox
And (3) explain why other potential info hazards, not the basilisk but very different configurations of acausal negotation (that have either not yet discovered, or were discovered but they not made public), should not be discussed.
0TobyBartels
This is true; nevertheless, good PR should still make things as least bad as possible. And indeed, you go on to make a suggestion as to how to do that (not even a bad one in my opinion).
0Richard_Kennaway
In other words, he disagrees with you and that is preventing him from agreeing with you.
4Jiro
Yes, except that agreeing with me is what a lot of people take Eliezer to be saying. There's this widespread belief that Eliezer just denied the Basilisk. And that's not really true; he denied the exact version of the Basilisk that was causing trouble, but he acceps the Basilisk in principle.
-1AlexMennen
Eliezer has done (2) many times.
[-]V_V150

Doing 2 without doing 1 looks insincere.