All of Venu's Comments + Replies

Are calibration and rational decisions mutually exclusive? (Part one)

I came to this post via a Google search (hence this late comment). The problem that Cyan's pointing out - the lack of calibration of Bayesian posteriors - is a real problem, and in fact something I'm facing in my own research currently. Upvoted for raising an important, and under-discussed, issue.

What I Think, If Not Why

The default case of FOOM is an unFriendly AI Before this, we also have: "The default case of an AI is to not FOOM at all, even if it's self-modifying (like a self-optimizing compiler)." Why not anti-predict that no AIs will FOOM at all?

This AI becomes able to improve itself in a haphazard way, makes various changes that are net improvements but may introduce value drift, and then gets smart enough to do guaranteed self-improvement, at which point its values freeze (forever). Given the tiny minority of AIs that will FOOM at all, what is the probability that an AI which has been designed for a purpose other than FOOMing, will instead FOOM?

Selling Nonapples

@Don: Eliezer says in his AI risks paper , criticising Bill Hibbard, that one cannot use supervised learning to specify the goal system for an AI. And although he doesn't say this in the AI risks paper (contra what I said in my previous comment), I remember him saying somewhere (was it in a mailing list?) that supervised learning as such is not a reliable component to include in a Friendly AI. (I may be wrong in attributing this to him however.) I feel this criticism is misguided as any viable proposal for (Friendly or not) AI will have to be built out of ... (read more)

Isn't supervised learning the current method of achieving friendly natural intelligence? (Most insane psychopaths had bad parents, didn't they?)
Selling Nonapples

I don't get this post. There is no big mystery to asynchronous communication - a process looks for messages whenever it is convenient for it to do so, very much like we check our mail-boxes when it is convenient for us. Although it is not clear to me how asynchronous communication helps in building an AI, I don't see any underspecification here. And if people (including Brooks) have actually used the architecture for building robots, that at least must be clear proof that there is a real architecture here.

Btw, from my understanding, Thrun's team made heavy use of supervised learning - the same paradigm that Eliezer knocked down as being unFriendly in his AI risks paper.

I DO get this post - I understand, and agree with the general concept, but I think Venu has a point that asynchronous programming is a bad example... although it LITERALLY means only "non-synchronous", in practice it refers to a pretty specific type of alternative programming methodology... much more particular than just the set of all programming methodologies that aren't synchronous.

The Weighted Majority Algorithm

I am interested in what Scott Aaronson says to this.

I am unconvinced, and I agree with both the commenters g and R above. I would say Eliezer is underestimating the number of problems where the environment gives you correlated data and where the correlation is essentially a distraction. Hash functions are, e.g., widely used in programming tasks and not just by cryptographers. Randomized algorithms often are based on non-trivial insights into the problem at hand. For example, the insight in hashing and related approaches is that "two (different) object... (read more)

Inner Goodness

To me it seems like this post evades what is to me the hard question of morality. If my own welfare often comes in conflict with the welfare of others, then how much weight should I attach to my own utility in comparison to the utility of other humans? This post seems to say I should look into the mirror to get my answer - but that answer is too crude - in the sense that I know I should care for others, but how much?

I think there is definitely a role for external influence here. My reading OB for the last year or more has made me consciously think of myse... (read more)

I'd agree with that. However, I doubt Eliezer would agree that you should only look in the mirror. Perhaps we can steel-man the concept: Look in the mirror, holding up the evidence you've gone out into the world to collect. At that point, if you see two pieces of paper with possible answers to your question float by, it will be you who chooses which answer is better. Even if one of the pieces of paper just says "Do whatever your parents tell you to", it would still be you who chose to listen to that piece of paper rather than another one. (Eliezer makes this analogy somewhere (and he does a better job than I did), but I couldn't find it; otherwise, I would have cited it.)
Inner Goodness

Ayn Rand? Aleister Crowley? How exactly do you get there? What Rubicons do you cross? It's not the justifications I'm interested in, but the critical moments of thought.

My guess is that Ayn Rand at least applied a "reversed stupidity = intelligence" heuristic. She saw examples of ostensible altruists committing great evil - and from there generalized to the opposite extreme - since altruism leads to evil, the only good must come from selfishness.

(Just to be clear, I am not defending Rand here.)

Could Anything Be Right?

"There are no-free-lunch theorems in computer science - in a maxentropy universe, no plan is better on average than any other. " I don't think this is correct - in this form, the theorem is of no value, since we know the universe is not max-entropy. No-free-lunch theorems say that no plan is better on average than any other, when we consider all utility functions. Hence, we cannot design an intelligence that will maximize all utility functions/moralities.

Whither Moral Progress?

@billswift: I do not want to divert the thread onto the topic of animal rights. It was only an example in any case. See Paul Gowder's comment previous to mine for a more detailed (and different) example of how empirical knowledge can affect our moral judgements.

Whither Moral Progress?

A few processes to explain moral progress (but probably not all of it): a) Acquiring new knowledge (e.g. the knowledge that chimps and humans are, on an evolutionary scale, close relatives), which leads us to throw away moral judgements that make assumptions which are inconsistent with such knowledge. b) Morality is only one of the many ends that we pursue, and as an end it becomes easier to pursue once you are amply fed, watered and clothed. In other words, improvements in material conditions enable improvements in morality. c) Conquest of one culture by ... (read more)

Moral Complexities

@Richard I agree with you, of course. I meant there exists no objective, built-into-the-fabric-of-the-universe morality which we can compute using an idealised philosopher program (without programming in our own intuitions that is).

Moral Complexities

I share neither of those intuitions. Why not stick with the obvious option of morality as the set of evolved (and evolving) norms? This is it, looking for the "ideal" morality would be passing the recursive buck.

This does not compel me to abandon the notion of moral progress though; one of our deepest moral intuitions is that our morality should be (internally) consistent, and moral progress, in my view, consists of better reasoning to make our morality more and more consistent.

The Tragedy of Group Selectionism

"Rationalisation of a predetermined bottom-line" is not always be a bad thing. It is common enough in Mathematics that you intuitively feel a result is right, and you work backwards from the result to see how you can prove it. The real mistake is if you do not take care in working it out backwards, and make wrong inferential steps in the chain. You may (legitimately) point out failures of this strategy, but there are also successes that you need to acknowledge.

Priming and Contamination

"Yet the most fearsome aspect of contamination is that it serves as yet another of the thousand faces of confirmation bias. Once an idea gets into your head, it primes information compatible with it - and thereby ensures its continued existence."

I am not sure I understand this. Once an idea gets into my head, my brain should prime all information related to the idea, not just information that is compatible with the idea. I am of course not denying the existence of confirmation bias, just trying to understand how priming in particular can promote it.

Recommended Rationalist Reading

Eliezer:'Probability theory and the structure of the real world exploited by tractable cognitive algorithms: Judea Pearl, "Probabilistic Reasoning in Intelligent Systems"'

Is the use of the phrase "cognitive algorithms" intended to mean that these algorithms are plausibly implemented in our own brains?

Why is the Future So Absurd?

".. but technological change feeds on itself, and therefore has a positive second derivative."

Nitpick: If technological progress were merely quadratic in time, then too it would have a positive second derivative. Kurzweil of course claims something much stronger - that technological progress is exponential in time, which means the first derivative and all succeeding derivatives are also exponential.

Suggested Posts

Why does statistical hypothesis testing continue to be used in many research fields despite its very many flaws? Are there biases at work here in that the widespread rejection of hypothesis testing would lead to the trashing of many senior researchers' works? How skeptical should we be of science in general if such shaky methodology is so widely adopted?