All of Roland2's Comments + Replies

An African Folktale

I don't understand what is so shocking about this story. The lessons seems quite clear: the mouth that you feed today will bite you tomorrow. It's not as if we don't have this in western culture.

The Evolutionary-Cognitive Boundary

@Eliezer, you mention that "This hypothesis then has two natural subdivisions:" I suppose you consider the second correct and the first incorrect?

The Evolutionary-Cognitive Boundary

@Eliezer: is the following a real experiment that was actually made or are you hypothesizing?

The actual experiment which shows that parental grief correlates strongly to the expected reproductive potential of a child of that age in a hunter-gatherer society - not the different reproductive curve in a modern society - does not disturb me.

BHTV: Yudkowsky / Wilkinson

Thanks Vladimir. Maybe a game like chess could also be a model. When you start to see the patterns you start getting better. Btw, I noticed that in fact a lot of errors people make when playing chess can be attributed to biases.

BHTV: Yudkowsky / Wilkinson

Allan/Eliezer: sorry, I misheard that, my fault.

BHTV: Yudkowsky / Wilkinson

Eliezer, at 39:38 if I heard correctly you say:

"I have to say I'm the first person who actually ran to the opposite extreme and put the entire burden of rationality on system one fast perceptual intuitive judgement."

Correct me if I'm wrong, but wouldn't that make overcomingbias meaningless because what we do here is more on the side of deliberative reasoning? After all we can't change system one that much.

I think it would be interesting if you could make a posting contrasting the notion of rationality against verbality since a lot of people might fall into the latter trap.

BHTV: Yudkowsky / Wilkinson

Regarding Obama vs. Bush, I wonder why even rationalists seem to operate under the assumption that the president has the power to make all important decisions. Even if Obama wanted he probably couldn't go against the power elite that is operating behind the scenes. JFK tried it.

Nonsentient Optimizers

Eliezer, I don't understand the following:

probably via resolving some other problem in AI that turns out to hinge on the same reasoning process that's generating the confusion

If you use the same reasoning process again what help can that be? I would suppose that the confusion can be solved by a new reasoning process that provides a key insight.

As for the article one idea I had was that the AI could have empathy circuits like we have, yes, the modelling of humans would be restricted but good enough I hope. The thing is, would the AI have to be sentient in order for that to work?

Harmful Options


what would be the right thing to do regarding our own choices? Should we limit them? Somehow this seems related to the internet where you always have to choose when to click another link and when to stop reading. Timothy Ferris also recommends a low information diet. I'm just brainstorming a bit here.

Imaginary Positions

I write as an ex-christian. Common misconceptions:

-All christians believe in creationism(as opposed to evolution). -All christians believe that the Bible is 100% correct and the inerrant word of God.

Complex Novelty

This particular idea of fun(solving increasingly more complex problems) seems to reflect the audience's intellectual mindset. I wonder what people who hate mathematics would have to say. If you are a professional golf player how would your perfect world look like? Would it be an ever increasing golf game where you had to hit a small hole over astronomical distances?

High Challenge


Unlike Roland, who is obviously a puritan, I rather enjoy the occasional spot of idleness. For a non-trivial number of people, playing WoW for a couple of hours a day is more fun that playing real life. Rather than make thinly veiled moral judgements about folks for their unproductivity, perhaps he should consider what makes certain games so engaging.

I enjoyed playing games myself, so I know what you are talking about.

You mention idleness, which I agree is sometimes worthwhile. This is the package deal fallacy since there are other ways to achieve tha... (read more)

I defy this conclusion. online friends are maybe not as socially fulfilling as face-to-face friends, but I wouldn't count them as having zero positive benefits. Especially if you have an obscure hobby/interest only shared by some - or only by a few local people. Doubly-so if you happen not to live in a big city. I have both online and offline friends and both sets of them make up my complete social circle. If you also happen to travel frequently - offline contact with your old circle of friends/family may be the best way to keep in touch. As a member of a family who I can truthfully say the sun-never sets upon... this is a huge benefit.
High Challenge

@D. Alex: (For Robin Hanson: have you heard about the economic studies carried out in WoW setting?)

Since you mentioned economics, did you ever consider the opportunity cost of playing WoW?

High Challenge

@ D. Alex: Some important reasons why the game is so pleasurable seem to be:

a) the ultimate goals are pretty clear (so unlike real life...)

b) the "measures of progress" are likewise clear -

c) the rewards are clear -

This looks like real life without the hard parts. Sure, it makes it more fun, but at the end will you feel rewarded? If you look back now or in a few years to the time spent playing and consider what you could have achieved in real life if you invested the same time into real challenges how will you feel? From my own experience I can... (read more)

Sounds like WoW is optimized for System 1 pleasures, and you explicitly reject this. I think that brings up an important point: How can we build a society/world where there are strong optimization forces to enable people to choose System 2 preferences? Once such a world iterated on itself for a couple generations, what might it look like? I don’t think this would be a world with no WoW-like activities, because a world without any candy or simple pleasures strikes me as deeply lacking. My System 2 seems to place at least a little value on System 1 being happy. So I’d guess the world would just have many fewer of such activities, and be structured in such a way as to make it easy to avoid choices we’d regret the next day. If this turns out to a physically impossible problem to overcome for some reason, then I could imagine a world with no System 1 pleasures, but such a world would be deeply lacking, even if that loss was more than made up for by gains in our System 2 values. As a side note, it'd be an interesting question how much of the theoretical per capita maximum value falls into which categories. An easier question is how much of our currently actualized value is immediate gratifications. I'd expect that to be heavily biased toward System 1, since we suffer from Akrasia, but it might still be informative.
Chaotic Inversion

@haig: good point.

@Bryan Bishop: Your quote about inertia reminded me of EY's post Created Already In Motion.

Chaotic Inversion

What I'm successfully experimenting with at the moment is:

Prime the pump for action! Whatever you need to do, just get started. If instead you start thinking and analysing too much you will find 1000s of other things to do. A related advice: do your email and webbrowsing at the end of the day, not at the beginning or middle.

Failure By Analogy

spelling: like attaching beak => like attaching A beak

Whither OB?

@ Z. M. Davis

Yes, there is a forum but it's not an official one. The whole point is creating a central place on the net to discuss and collect rationality related stuff.

OB is great but it only has the top guns. Where is the place where the lower rank people can ask questions or publish inspiring new ideas? No, the comments section is not enough and the open thread neither. Stuff just gets buried in nowhere land. Try reaching the 2. or 3. page of the comments and you will know what I mean.

And I don't think we should underestimate the potential contributions... (read more)

Whither OB?

I second EY's idea. Yes, we need a community forum where everyone can post. Some where concerned with the quality of user generated posts to which I have to say that the point of the forum is not only to have good posts but also to ask questions and clarifications and spark new ideas. Nowadays those often get lost in the comments. Say you didn't understand what exactly a prior is, you can create a new posting asking for clarification. And there hopefully will be enough qualified people to answer it, freeing up time for EY. The ideal format for this would b... (read more)

Ask OB: Leaving the Fold

@ben: My rabbi friend still wanted to be a rabbi, still wanted to be a good wife, because these things brought her joy. She enjoyed knowing families and helping others, both in good times and bad, even if she wasn't doing those things on god's behalf. She was doing them for herself. And I'd say that makes her an even more selfless person.

Enjoying it contradicts selflessness. And if she is spreading wrong religious ideas she is leading others astray. Very wrong!

Ask OB: Leaving the Fold

My experience:

10 years of christianity, I was a member of a lot of churches always searching for the one that would bring me closer to the truth. Then I realized that most churches where just the blind leading the blind and finally that christianity itself was flawed, that's when I found freedom.

What made it easier for me is that I never was really deeply invested into it because from the very beginnings I always had doubts and never stopped asking questions. That's what helped me find the truth after all.

Now to your situation. Keep in mind that there is ... (read more)

Hanging Out My Speaker's Shingle

@Jack Christopher: what is H+?

Ethical Injunctions

@Tom McCabe: I would have answered "yes"; eg., I would have set off a bomb in Hitler's car in 1942, even if Hitler was surrounded by babies. This doesn't seem to be a case of corruption by unethical hardware; the benefit to me from setting off such a bomb is quite negative, as it greatly increases my chance of being tortured to death by the SS.

It's easy to talk now about it, harder if you actually lived in Germany at that time and had to really fear the SS. Are you american? If yes did you consider the fact that the actual political situation in ... (read more)

Indeed. I remember an IT project manager telling me the German people should have stood up to Hitler and stopped him. I pointed out that she was not even prepared to tell her manager the truth about the state of her project (running later than advertised of course). All she had at stake was the size of her end of year bonus. I remember reading about a man who voted against Hitler in the referendum to make him dictator. He was severely beaten, his house was burned down, and he wife and daughter were gang-raped.
There probably was a time when killing Hitler had a significant chance of ending the war by enabling peace talks (allowing some high-ranking German generals/politicians to seize power while plausibly denying having wanted this outcome). The window might have been short, and probably a bit after '42, though. I'd guess any time between the Battle of Stalingrad (where Germany stopped winning) and the Battle of Kursk (which made Soviet victory inevitable) should've worked - everyone involved should rationally prefer white peace to the very real possibility of a bloody stalemate. Before, Germany would not accept. Afterwards, the Soviets wouldn't.
Ethical Inhibitions


I don't understand this sentence:

"By and large, it seems to me a pretty fair generalization that people who achieve great good ends manage not to find excuses for all that much evil along the way."

I mean, if they really achieved great good ends and those ends have more positive utility than the negative utility of the evil along the way wouldn't this be a case where the end actually justifies the means?

Dark Side Epistemology


To drive home my earlier point. The whole idea of jedis vs. siths reflects a Manichaeistic worldview(good vs. bad). Isn't this a simplification?

Dark Side Epistemology

@Eliezer: Roland, these are the Sith masters.

Ok, got your point. One thing I worry though is how much those movie analogies end up inducing biases in you and others.

Dark Side Epistemology


I agree with you what regards people deceiving themselves. But I disagree regarding people that are deceiving others with purpose. Some of these people can be very smart and know very well what they are doing and on what biases they are playing. They have elevated the art of deception to a science, ohhh yes, read marketing books as an example. Otherwise a superintelligence would become stupid in the process of lying to the human operator with the intention to get out of the box.

Traditional Capitalist Values

the terrorists et al. probably do hate our freedom -- e.g., our freedom to watch DVDs of people having sex. This fact may not be particularly useful in keeping from being attacked again (I for one am not willing to give up the right to watch DVDs of people having sex).

A witchhunt tells us a lot about those doing it but little about those who are the target of it.

Why Does Power Corrupt?

Denis Bider: I agree with you.

Crisis of Faith

"How do I know if long-held belief X is false?"

Eliezer, I guess if you already are asking this question you are well on your way. The real problem arises when you didn't even manage to pinpoint the possibly false believe. And yes I was a religious person for many years before realizing that I was on the wrong way.

Why didn't I question my faith? Well, it was so obviously true to me. The thing is: did you ever question heliocentrism? No? Why not? When you ask the question "How do I know if Heliocentrism is false?" You are already on your ... (read more)

AIs and Gatekeepers Unite!

Addendum to my last post:

I forgot to emphasize: the marketing aspect might be more important then everything else. I guess a lot of people have no idea what the singularity institute is about, etc... So this experiment would be a great way to create awareness. And awareness means more donations. On the other hand I sometimes wonder if drawing too much attention on the subject of powerful AIs might backfire if the wrong people try to get hold of this technology for bad purposes.

AIs and Gatekeepers Unite!

Me, I'm out of the AI game, unless Larry Page wants to try it for a million dollars or something.


I think this is a great opportunity to get some funds and marketing for the singularity institute. How about collecting donations over the internet until a million is reached and then performing the experiment between you and an intelligent gatekeeper. Alternatively get the money in through marketing, maybe Google might be interested?

It could even be transmitted live over internet so all the interested parties could watch it.

Man this would be great news... (read more)

Shut up and do the impossible!

Addendum to my last comment:

I think another way to pinpoint the problem you are adressing is: You have to be able to live years with the strong feeling of uncertainty that comes from not really knowing the solution while still working on it. A patient enduring. Saying "it's impossible" or proposing a simple but incorrect solution is just an easy way out.

Doing the "extraordinary" effort doesn't work because people just fill in their cached thoughts about what constitutes extraordinary and then move on.

So my advice would be: embrace the uncertainty!

Shut up and do the impossible!

To accept this demand creates an awful tension in your mind, between the impossibility and the requirement to do it anyway. People will try to flee that awful tension.

This tension reminds me of need for closure. Most people hate ambiguity and so if a solution is not apparent it's easier to say "it's impossible" than to live with the tension of trying to solve it and not knowing if there is a solution at all.

Shut up and do the impossible!

People ask me how likely it is that humankind will survive, or how likely it is that anyone can build a Friendly AI, or how likely it is that I can build one. I really don't know how to answer.

Robin Hanson would disagree with you:

You Are Never Entitled to Your Opinion

Friedman's "Prediction vs. Explanation"

We believe the first(T1).

Why: Correctly predicted outcomes updates it's probability of being correct(Bayes).

The additional information available to the second theory is redundant since it was correctly predicted by T1.

The Level Above Mine

@Jef Allbright:

I suppose you could google "(arrogant OR arrogance OR modesty) eliezer yudkowsky" and have plenty to digest.

Well, I was asking you, not google. But it seems that you are not willing to stand behind your words, making claims then failing to provide evidence when asked. Refering to a third party is an evasive maneuver. Show us your cards!

That's as far as I'm willing to entertain this line of inquiry, which ostensibly neutral request for facts appears to belie an undercurrent of offense.

That's your supposition.

The Level Above Mine

@Jef Allbright:

Can you be concrete and specific about where Eliezer is or has been arrogant?

The Level Above Mine

First, same question as Douglas: what is it with the brick wall at 40?

Second: This is another great post, its rare for people to expose their thoughts about theirselves in such an open way. Congratulations!

Regarding your ability, I'm just a regular guy(studied Math in college) but your writings are the most inspiring I've ever read. So much self-reflection about intelligence and the thinking process. The insight about how certain mental processes feel is totally new to me. You have helped me a lot to identify my own blind spots and mistakes. Now I can look... (read more)

My Naturalistic Awakening

Eliezer, why do you call this awakening "naturalistic"? I don't see where your previous view was not "naturalistic".

The Sheer Folly of Callow Youth

We all know what happened to Donald Rumsfeld, when he went to war with the army he had, instead of the army he needed.

Sorry Eliezer, but when it comes to politics you are often wrong. AFAIK Donald Rumsfeld is doing fine and made a lot of money with the war, as did many others in power. Using your words: he is smiling from the top of a giant heap of utility. Do you really think he cares about the army or Iraq?

Existential Angst Factory


Ironically, based on what I know Psychiatry is perhaps the biggest Existential Angst Factory in existance.

Existential Angst Factory


I really enjoy your articles, but one petpeeve I have is that you sometimes seem to endorse certain medical treatments.

In this article you mentioned:

stomach reduction surgery Based on what I read this procedure is quite dangerous and based on statistics generally bad although doctors make good money with it. Google for it.

neuropharmaceutical interventions. Also highly controversial. Psychiatry has it's own set of problems. Google Thomas Szasz and anti psychiatry.

Eliezer never forget that for many readers you are an authority figure and so be careful with what you endorse.


Touching the Old

I can't believe no one wrote those two quotes yet:

"Those who don't learn from history are damned to repeat it."

"What we learn from history is that people don't learn from history."

My Kind of Reflection

michael vassar:

I would agree with you if there were no cognitive biases, but alas, there are and I think they are one of the main causes why reasoning errors happen. In fact this is why this blog exists.

When I look at my past such reasoning errors abound and they are the result of a biased human mind. According to your definition I don't think those where "honest" mistakes, at the same time I think it is unfair to label them "dishonest". The biases simply reflect the way human minds work.

My Kind of Reflection

michael vassar:

I would agree with you if there were no cognitive biases, but alas, there are and I think they are one of the main causes why reasoning errors happen. In fact this is why this blog exists.

When I look at my past such reasoning errors abound and they are the result of a biased human mind. According to your definition I don't think those where "honest" mistakes, at the same time I think it is unfair to label them "dishonest". The biases simply reflect the way human minds work.

My Kind of Reflection

@michael vassar: When you find someone who disagrees with you this is very strong evidence that either you or that other person or both HAVE NOT BEEN TRYING to reach true beliefs in the relevant domain.

How about the case where you have been trying hard but simply went down the wrong way because of undetected reasoning errors?

Will As Thou Wilt

Only the first interpretatio is correct!

  1. I would agree with the 4. before "because", which says the same thing as the first. After "because" there is an error of circular reasoning: our desires will determine our desires. Wrong! Our desires are the result of subconcious mental processing of our brain. Anyway Schopenhauer didn't say anything about the origin of our will or desire in this sentence so there is no point in interpreting it into it.
2 of 10, not 3 total

I agree with the above commenters. What we need is a decent forum! Why don't create one in google groups? This site is meant to be a blog, not a forum and TypePad sucks!

Load More