All of Joseph_Knecht's Comments + Replies

Moral Complexities
Why do people seem to mean different things by "I want the pie" and "It is right that I should get the pie"? Why are the two propositions argued in different ways?

"I want the pie" is something that nobody else is affected by and thus nobody else has an interest in. "I should get the pie" is something that anybody else interested in the pie has an interest in. In this sense, the moral preferences are those that other moral beings have a stake in, those that affect other moral beings. I think some kind of a distinct... (read more)

2 of 10, not 3 total

I've voiced my annoyance with the commenting system in the past, in particular that it is non-threaded and so often very difficult to figure out what someone is responding to if they don't include context (which they often don't), so I won't give details again.

On the topic of the 2 of 10 rule, if it's to prevent one person dominating a thread, shouldn't the rule be "no more than 2 of last 10 should be by the same person in the same thread" (so 3 posts by the same person would be fine as long as they are in 3 different threads)?

The Bedrock of Fairness
Optimistically, I would say that if the murderer perfectly knew all the relevant facts, including the victim's experience, ve wouldn't do it

The murderer may have all the facts, understand exactly what ve is doing and what the experience of the other will be, and just decide that ve doesn't care. Which fact is ve not aware of? Ve may understand all the pain and suffering it will cause, ve may understand that ve is wiping out a future for the other person and doing something that ve would prefer not to be on the receiving end of, may realize that it is beh... (read more)

What Would You Do Without Morality?
But if I understand you, you are saying that human morality is human and does not apply to all sentient beings. However, as long as all we are talking about and all we really deal with is humans, then there is no difference in practice between a morality that is specific to humans and a universal morality applicable to all sentient beings, and so the argument about universality seems academic, of no import at least until First Contact is achieved.

What I am really saying is that the notion of "morality" is so hopelessly contaminated with notions... (read more)

The Moral Void

Laura ABJ: To expand on the text you quoted, I think that killing babies is ugly, and therefore would not do it without sufficient reason, which I don't think the scenario provides. The ugliness of killing babies doesn't need a moral explanation, and the moral explanation just builds on (and adds nothing but a more convenient way of speaking about) the foundation of aversion, no matter how it's dressed up and made to look like something else.

The idea is not compelling to me and so would not haunt me forever, because like I said, I'm not yet convinced that ... (read more)

The Moral Void

Hal: as an amoralist, I wouldn't do it. If there is not enough time to explain to me why it is necessary and convince me that it is necessary, no deal. Even if I thought it probably would substantially increase the future happiness of humanity, I still wouldn't do it without a complete explanation. Not because I think there is a moral fabric to the universe that says killing babies is wrong, but because I am hardwired to have an extremely strong aversion to like killing babies. Even if I actually was convinced that it would increase happiness, I still migh... (read more)

What Would You Do Without Morality?

Traditional notions of morality are confused, and observation of the way people act does show that they are poor explanations, so I think we are in perfect agreement there. (I do mean "notion" among thinkers, not among average people who haven't given much though to such things.) Your second paragraph isn't in conflict with my statement that morality is traditionally understood to be in some sense objectively true and objectively binding on us, and that it would be just as true and just as binding if we had evolved very differently.

It's a differe... (read more)

What Would You Do Without Morality?

Constant: I basically agree with the gist of your rephrasing it in terms of being relative to the species rather than independent of the species, but I would emphasize that what you end up with is not a "moral system" in anything like the traditional sense, since it is fundamental to traditional notions of morality that THE ONE TRUE WAY does not depend on human beings and the quirks of our evolutionary history and that it is privileged from the point of view of reality (because its edicts were written in stone by God or because the one true speci... (read more)

What Would You Do Without Morality?

I agree with mtraven's last post that morality is an innate functionality of the human brain that can't be "disproved", and yet I have said again and again that I don't believe in morality, so let me explain.

Morality is just a certain innate functionality in our brains as it expresses itself based on our life experiences. This is entirely consistent with the assertion that what most people mean by morality -- an objective standard of conduct that is written into the fabric of reality itself -- does not exist: there is no such thing!

A lot of confu... (read more)

What Would You Do Without Morality?

mtraven: many of the posters in this thread -- myself included -- have said that they don't believe in morality (meaning morality and not "values" or "motivation"), and yet I very highly doubt that many of us are clinically psychopaths.

Not believing in morality does not mean doing what those who believe in morality consider to be immoral. Psychopathy is not "not believing in morality": it entails certain kinds of behaviors, which naive analyses of attribute to "lack of morality", but which I would argue are a result of aberrant preferences that manifest as aberrant behavior and can be explained without recourse to the concept of morality.

What Would You Do Without Morality?

Unknown: of course it would make a difference, just as my behavior would be different if I had billions of dollars rather than next to nothing or if I were immortal rather than mortal. It doesn't have anything to do with "morality" though.

For example, if I had the power of invisibility (and immateriality) and were able to plant a listening device in the oval office with no chance of getting caught, I would do it in order to publicly expose the lies and manipulations of the Bush administration and give proof of the willful stupidity and rampant di... (read more)

What Would You Do Without Morality?

On the topic of vegetarianism, I originally became a vegetarian 15 years ago because I thought it was "wrong" to cause unnecessary pain and suffering of conscious beings, but I am still a vegetarian even though I no longer think it is "wrong" (in anything like the ordinary sense).

Now that I no longer think that the concept of "morality" makes much sense at all (except as a fancy and unnecessary name for certain evolved tendencies that are purely a result of what worked for my ancestors in their environments (as they have expre... (read more)

What Would You Do Without Morality?

Like many others here, I don't believe that there is anything like a moral truth that exists independently of thinking beings (or even dependently on thinking beings in anything like an objective sense), so I already live in something like that hypothetical. Thus my behavior would not be altered in the slightest.

Possibility and Could-ness

@Richard: I think that's a valid reduction. It explains non-negative integers reductively in terms of an isomorphism between two groups of things without appealing to numbers or number concepts.

@constant: regardless of the label, you still have 2 sets of things, those which it is possible to label fizzbin (following the rules) and those which it is not. Possibility is still there. So what does it mean that it is possible to label a node fizzbin? Does that mean that in order to understand the algorithm, which relies on possibility of labelling nodes "f... (read more)

Possibility and Could-ness

To be clear, there are two different but related points that I've tried to make here in the last few posts.

Point 1 is a minor point about the Rationalist's Taboo game:

With regard to this point, as I've stated already, the task was to give a reductive explanation of the concept of possibility by explaining it in terms of more fundamental concepts (i.e., concepts which have nothing to do with possibility or associated concepts, even implicitly). I think that Eliezer failed that by sneaking the concept of "possibile to be reached" (i.e., "rea... (read more)

Possibility and Could-ness

Cyan: I think your quibble misses the point. Eliezer's directions were to play the rationality taboo game and talk about possibility without using any of the forbidden concepts. His explanation failed that task, regardless of whether either he or Brandon were referring to the map or the territory. (Note: this point is completely unrelated to the specifics of the planning algorithm.)

I'll summarize my other points later. (But to reiterate the point that Eliezer doesn't get and pre-empt anybody else telling me yet again that the label is irrelevant, I realize the label is irrelevant, and I am not talking about character strings or labels at all.)

Possibility and Could-ness

Eliezer, my point was that you dedicated an entire follow-up post to chiding Brandon, in part for using realizable in his explanation since it implicitly refers to the same concept as could, and that you committed the same mistake in using reachable.

Anyway, I guess I misunderstood the purpose of this post. I thought you were trying to give a reductive explanation of possibility without using concepts such a "can", "could", and "able". If I've understood you correctly now, that wasn't the purpose at all: you were just trying to describe what people generally mean by possibility.

Possibility and Could-ness

Can you talk about "could" without using synonyms like "can" and "possible"? .... Can you describe the corresponding state of the world without "could", "possible", "choose", "free", "will", "decide", "can", "able", or "alternative"?

My point being that you set out to explain "could" without "able" and you do it by way of elaboration on a state being "able to be reached".

What you decide to label the concept do... (read more)

Possibility and Could-ness

I don't think reachable is permissible in the game, since reach-able means able to be reached, or possible to be reached.

Possibility is synonymous with to be able, so any term suffixed with -able should be forbidden.

The reachability explanation of possibility is also just one instance of possibility among many. to be able (without specifying in what manner) is the general type, and able to be reached is one particular (sub-) type of possibility. The more traditional understanding of possibility is able to exist, but others have been used too, such as able... (read more)

Possibility and Could-ness

Robin Z:

I think it is not only a clash of intuitions, since the success rates of pre-scientific theory and folk psychology are poor. This should urge caution in keeping concepts that seem to give rise to much confusion. I would argue that the default attitude towards pre-scientific concepts that have been shrouded in confusion for thousands of years, with still no clarity in sight, should be to avoid them when possible.

When you say that you haven't seen evidence that puts "soul" on shaky grounds, do you mean that assuming determinism and what we... (read more)

Possibility and Could-ness

Nick, your example confuses more than it clarifies. What exactly is the choice? Brain processes that occur in 0 < t < 1? Brain processes occurring in that slice that have certain causal relations with future actions? Conscious brain processes occurring such that...? Conscious brain processes occurring such that ... which are initiated by (certain) other brain processes?

You speak as if "choice" means something obvious that everybody understands, but it only has such a meaning in the sense that everybody knows what is meant by "soul" (which refers to a non-existent thing that means something different to practically everybody who uses it and usually results in more confusion than clarification).

Possibility and Could-ness


by "constraints", I meant that Eliezer specified only that some particular processes happening in the brain are sufficient for choice occurring, which my example refuted, to which you added the ideas that it is not mere happening in the brain but also the additional constraints entailed by concepts of Eliezer-the-person and body-shell-of-Eliezer and that the former can be destroyed while the latter remains, which changes ownership of the choice, etc.

Anyway, I understand what you're saying about choice as a higher-level convenience term, ... (read more)

Possibility and Could-ness

Jadagul: I'm saying that Eliezer's explanation of what a choice is is not a sufficient condition. You suggested some additional constraints, which I would argue may be necessary but are still not sufficient conditions for a choice occurring.

My key point, though, as Schizo noted, was that I don't think the concept should be salvaged, any more than phlogiston or caloric should have been salvaged.

Possibility and Could-ness

Robin, I don't think "whose brain it is" is really a meaningful and coherent concept, but that is another topic.

My general point was that Eliezer seemed to be saying that certain things occurring in his brain are sufficient for us to say that he made a choice and is morally responsible for the choice. My example was intended to show that while that may be a necessary condition, it is not sufficient.

As for what I actually believe, I think that while the notions of choice and moral responsibility may have made sense in the context in which they aro... (read more)

Possibility and Could-ness

Eliezer said:

I am not saying that choice is an illusion. I am pointing to something and saying: "There! Right there! You see that? That's a choice, just as much as a calculator is adding numbers! It doesn't matter if it's deterministic! It doesn't matter if someone else predicted you'd do it or designed you to do it! It doesn't matter if it's made of parts and caused by the dynamics of those parts! It doesn't matter if it's physically impossible for you to have finally arrived at any other decision after all your agonizing! It's still a choice!"

-... (read more)

Causality and Moral Responsibility

Nick: note that I said "conscious thoughts" and not "thoughts", and I specified that the individual is not aware of the inputs/outputs from/to the actuators/sensors and has no control over them.

Causality and Moral Responsibility

Nick, I don't understand what you mean by random. There is nothing in the slightest random (as I understand the term) in the scenario I gave. The primary difference between the two cases is that in one case you believe you are effecting action via your conscious thoughts (but you aren't) and in the other you do not believe you are effecting action via your conscious thoughts. In both cases, your actions are fully determined by what is going on in your brain; it's just that your conscious thoughts are irrelevant to what happens (and you are deluded about this in one of the two scenarios).

Causality and Moral Responsibility

Eliezer: why do you say John-1 (the "coward") is morally responsible if under your scenario it was physically impossible for him to act as John-2 did given his initial physical conditions? (If it were not impossible, then his actions wouldn't have been fully determined by his initial physical condition.)

To possibly confuse matters a little more, here's a thought experiment that occurred to me for some reason. I'd be curious to hear what anybody who says that determinism does not undermine moral responsibility, or who makes the even stronger claim... (read more)

Bloggingheads: Yudkowsky and Horgan

AI researchers of previous eras made predictions that were wildly wrong. Therefore, human-level AI (since it is a goal of some current AI researchers) cannot happen in the foreseeable future. They were wrong before, so they must be wrong now. And dawg-garn it, it seems like some kind of strange weirdo religious faith-based thingamajiggy to me, so it must be wrong.

Thanks for a good laugh, Mr. Horgan! Keep up the good work.

The Rhythm of Disagreement

In the video link that komponisto gave, the relevant section of the video starts at 49:00 or so. He doesn't argue there though that consciousness -- he uses the term sentience -- will necessarily remain a mystery but only that this might turn out to be the case. He makes the analogy of trying to think about time before the big bang and that there is some kind of conceptual category-type error going on there that gives it its subjective feeling of being mysterious and unknowable, and states that this may be the case with sentience/consciousness, free-will, etc.

A Premature Word on AI

Robin, I agree that the charitable interpretation is possible. But when he said "It seemed a safe answer since no one could ever tell me I was wrong", I took him to be saying that whether he could be shown to be wrong or not was a major factor in saying what he did.

In that case, if he cared more about saying something that he couldn't be held responsible for if wrong than about having a justified and likely-to-be-correct position, I'd say that is at least as bad.

A Premature Word on AI

Robin, the issue is not "having made a mistake years ago", which we all have done; it is "knowingly being intellectually sloppy/dishonest because you know you can get away with it for a while".

Einstein's Superpowers

Caledonian, the childish "I have a secret that I'm not going to tell you, but here's a hint" bs is very annoying and discourages interacting with you. If you're not willing to spell it out, just don't say it in the first place. Nobody cares to play guessing games with you.

Einstein's Superpowers

Caledonian: were you trolling, or are you going to explain the "gaping hole" and "false equivalence" you mentioned?

Einstein's Superpowers

What "giant hole"? What "false equivalence"?

Einstein's Superpowers

Personally, I don't there is a trick, and I don't think he's keeping it private for those reasons. I think his method, if something so obvious (which is not to say easy) can be called a method, is to discuss the issue and interact with the person long enough to build up a model of the person, what he values and fears most, and then probe for weaknesses & biases where that individual seems most susceptible, and follow those weaknesses -- again and again.

I think most, perhaps all, of us, unless we put our fingers in our ears and refuse to honestly engage... (read more)

Einstein's Superpowers

@DaveInNYC: what you can and can't assume is not relevant to whether the transcripts should be private or not. If they were public, anybody predisposed to explanations like "they must have been more simple-minded than me" could just as easily find another equally "compelling" explanation, like "I didn't think of that 'trick', but now that I know it, I'm certain I couldn't be convinced!"

I personally think they should remain private, as frustrating as it is to not know how Eliezer convinced them. Not knowing how Eliezer did it nicely mirrors the reality of our not knowing how a much smarter AGI might go about it.

Einstein's Superpowers

Eliezer: if you're going to point to the AI Box page, shouldn't you update it to include more recent experiments (like the ones from 2005 where the gatekeeper did not let the AI out)?

Einstein's Superpowers

When did "genius" (as in "just another Jewish genius") as a term become acceptable to use in the sense of mere "exceptional ability" without regard to accomplishment/influence or after-the-fact eminence? I know it is commonly (mis-)used in this sense, but it seems to me that "unaccomplished genius" should be an oxymoron, and I'm somewhat surprised to see it used in this sense so much in this thread (and on this forum).

I have always considered the term to refer (after the fact) to those individuals who shaped the inte... (read more)

Reductive Reference

Eliezer: no comment on my point that 'single-levelness' is an attribute of your model of reality rather than of reality itself? And that saying "reality is single-level" is therefore misleading.

2Ramana Kumar13y
Indeed, what would it mean for reality to be multi-leveled? (This would address tcpkac's comment too.)
Reductive Reference

I prefer to think of the distinction between reality and our models of it not in terms of single-level versus multi-level but rather in terms of the thing itself versus descriptions of the thing: conceptual systems have levels, reality does not.

Thinking in terms of levels is applicable to conceptual systems, but not applicable to reality — applicable to conceptual systems about reality, but not to reality itself.

The notion of level is just a mental construct that is helpful in describing how we think and how we represent. It is a function of how we carve u... (read more)

Joy in Discovery

Caledonian, that's mere sophistry to say "mathematics is physics because it is performed by a brain or analogous physical device".

According to that definition, no matter what you study at university, you are really doing physics. Every single human being that has ever earned a university degree earned a physics degree (since English is Physics, Art History is Physics, etc.), and every individual whose work involves use of her brain (even if only for respiration and basic metabolic processes) is a physicist.

I think I'll stick with the understanding of physics that the rest of the world uses.

Joy in Discovery

Me: Physics could at some point be completely solved, which is to say that at some point, there would be no further knowledge that would ever allow us to do anything new, to make any better a prediction, to do anything more efficiently, etc.

Tobbic: It seems to me that ppl have a tendency to overstate their knowledge. What does a slug know about physics? Respectively, what does a human know about (possible) 101th dimension or travel through time or any of the stuff some posthuman might do "physics" about.

I didn't claim that it is a fact that physi... (read more)

Joy in Discovery

Caledonian: Every mathematical statement is a claim about the behavior of the physical world.

Please interpret the following statements for me in terms of the behavior of the physical world, and tell me which branch of physics deals with the behavior of each:

  • The cardinality of the set of real numbers is greater than the cardinality of the set of natural numbers.

  • The continuum hypothesis is independent of ZF and ZFC set theory.

  • There are no solutions to the equation a^n + b^n = c^n for non-zero integers a, b, and c and integer n > 2.

Joy in Discovery

The mathematics of physics is just an infinitesimal part of all of mathematics.

Physics could at some point be completely solved, which is to say that at some point, there would be no further knowledge that would ever allow us to do anything new, to make any better a prediction, to do anything more efficiently, etc.

There is no such limit to mathematics though, because mathematics, unlike physics, is not constrained by reality. It only needs to be self consistent (under perhaps limitless different conceptions of consistency) given a particular starting point... (read more)

Joy in Discovery

I think the shot of adrenaline to the ego is what gives the sense of high in most cases, and what motivates most scientists. And it probably is almost entirely the source of the high of the non-world changing and minor discoveries.

Having said that, I do think that in some cases, very few, there is perhaps a stronger element of what Eliezer briefly touched on towards the end of the essay: that one has just added to the sum total of humanity's knowledge, and inched us toward the perfect understanding of the world around us that science constantly seeks.

To th... (read more)

Words as Mental Paintbrush Handles

Caledonian: you said "Your visualizations include such details? As the description didn't include such details, they're necessarily undefined - so why did you define them out of their uncertainty?"

I understood from your statement that you expressed surprise that the reported visualization contained such details as "which side of the street the person is walking down". This implied to me that you believe it is possible to visualize a man walking down a street, but not be either walking down the left or right side or in the street itself, etc.

Words as Mental Paintbrush Handles

I personally saw the man walking away from me on the left side of the street, and my persective was just to the left of the curb on that side of the street and slightly higher than the man, who was a short distance from me. I saw him turn left into a drugstore for a split second, and then when I realized the joke briefly saw him morph into a drugstore on the sidewalk.

To the people who say that they visualized the scene but, for example, didn't see the person walking towards or away from you, or didn't see the man on one side of the street or the other: how... (read more)

The Second Law of Thermodynamics, and Engines of Cognition


I don't see how your response addresses my concern that saying accurate belief requires observation implies unacceptable consequences for the man on the street, such as that his correct belief that the Giants would win on Sunday is nevertheless not an accurate belief.

Load More