Eliezer,

I am very much inclined to analyze your articles because you are indeed very enthusiastic about your theories, which is a rarity these days. On the link there’s a Wordle tag cloud picture of your article.

As you can see, there’s a lot of “argument(s)”,”moral”, “abstract”, “fiction’, and a somewhat humble “experience”.

To the point – fiction in the literary domain is often a method for implying moral concepts. But fiction is, above all, an invitation to imagine. There is a catch. We can imagine a setting, a world, a relationship. But we sometimes cannot imagine that this setting, or world, or relationship, is morally justified, simply because we have an elaborate moral hierarchy to begin with.

Another aspect is that fiction is a theory we know is false, but useful. Beware, Yudkowski – fiction is typically useful when it relates to things that exist or could exist, but only if we are able to observe them one day.

Hence a problem with science fiction – it typically uses big time spans to make reading them books interesting. In the end, as we very well know, science fiction is either too far sighted or too short sighted, but not really useful.

So what the hell am I talking about? Eliezer, in your work there are many useful ideas. “Three Worlds Collide” is more like entertainment.

So I appologize, Vladimir for bringing this up again, but i'm sort of a newcomer :)

However, notice that even in "Building Something Smarter" Eliezer does NOT deny his underlying need for a religious foundation (he simply declines to comment, which, among other things denotes his own dissatisfaction with that, well, bias).

Vladimir, Kant once advised: "free yourself from the self-incurred tutelage of others".

I think that even if you consider Eliezer's Fun Theory as a somehow independent ethical construct (whatever that means), you still fail to accommodate for the lack of evidentialism in it. To me it appears as a mash-up of sporadic belief and wishful thinking, and definitely worth considering the ad hominem causality for it.

I don't know if this comment will get pass the political correctness criterion. May the webadmin have mercy on my soul :)

Eliezer, I am very much tempted to go into personal comments. I will do that on one premise only – that the title of this blog is “Overcoming Bias”. I would like to contribute to that purpose in good faith.

Having read some of Eliezer’s posts I was sure that he has been treated with a high dose of Orthodox Judaism. In this post he specifically points to that fact, thus confirming my analysis. To other readers: Orthodox Judaism requires that every action and thought be informed by divinely revealed law and ethics. It is one of the most sophisticated religious dogmas imaginable, and in its complexity and depth is comparable only to Buddhism.

Another important feature of Orthodox Judaism is its compartmentilization. This provides adherents of this religion with a very special belief system centered on indisputable sacredness of all things Jewish. It is so strong a system indeed that it sometimes leads to well-documented obsessive compulsive disorders.

Gladly, Eliezer has evaded the intricate chains of that belief system, it appears. My wild guess here is that he needs a substitute system. That is why he is so keen on Singularity. That is why he would like to have his Fun Theory – to counter his lack of security after he has left the warm house of Jahveh. So he is building a new, quite complex and evolutionary belief system that looks to me like a modern day חסידות.

I can only sympathize.

Jason, please see my comment in the next Eliezer post.

Dagon has made a point I referred to in the previous post: in the sentence “I have unlimited power” there are four unknown terms.

What is I? What does individuality include? How is it generated? Eliezer does not consider the evasive notion of self, because he is too focused on the highly hypothetical assumption of “self” that we adhere to in Western societies. However, should he take off the hat of ingenuity for a while, he would discover that the mere defining of “self” is extremely difficult, if not impossible.

“Unlimited” goes in the same basket as “perfect”. Both are human concepts that do not work well in a multidimensional reality. “Power” is another murky concept, because in social science it is the potential ability of one agent to influence another. However, in your post it seems we are talking about power as some universal quality to manipulate matter, energy, and time. One of the few things that quantum mechanics and relativity theory agree about, is that it is probably impossible to do it.

“I have unlimited power” implies total volitional control of a human being (presumably Robin Hanson), over the spacetime continuum This is much more ridiculous, because Robin is part of the system itself.

The notion of having such power, but being somehow separated from it is also highly dubious. Dagon rightly points to the transformative effect such “power” would have (only that it is impossible :) ). Going back to identity: things we have (or rather, things we THINK we have), do transform us. So Eliezer may want to unwind the argument. The canvas is flawed, methinks.