Sailor Vulcan

1. On the deontology/virtue ethics vs consequentialism thing, you're right I don't know how I missed that, thanks!

1a. I'll have to think about that a bit more.

2. Well, if we were just going off of the four moralities I described, then I already named two examples where two of those moralities are unable to converge: a pure flourishing maximizer wouldn't want to mercy kill the human species, but a pure suffering minimizer would. A pure flourishing maximizer would be willing to have one person tortured forever if that was a necessary prerequisite for uplifting the rest of the human species into a transhumanist utopia. A suffering minimizer would not. Even if the... (read more)

Replying toWhy the AI Alignment Problem Might be Unsolvable?

1a. Deontology/virtue ethics is a special case of consequentialism. The reason for following deontological rules is because the consequences that result from following deontological rules almost always tend to be better than the consequences of not following deontological rules. The exceptions where it is wiser to not follow deontological rules are generally rare.

1b. Those are social mores, not morals. If a human is brainwashed into shutting down the forces of empathy and caring within themselves, then they can be argued into treating any social more as a moral rule.

2. Sorry I should have started that paragraph by repeating what you said, just to make it clear what I was responding to.... (read 400 more words →)

Replying toWhy the AI Alignment Problem Might be Unsolvable?

1. Ask yourself, what sorts of things do we humans typically refer to as "morality" and what things do we NOT refer to as "morality"? There are clearly things that do not go in the morality bucket, like your favorite flavor of ice cream. But okay, what other things do you think go in the morality bucket and why?

2. Because a) the same sorts of arguments can be made in reverse. Just as Minnie or Maxie might come to accept Eye for an Eye on pragmatic grounds because it makes society as a whole better/less bad, Goldie might accept Maximize Flourishing and/or Minimize Suffering on the grounds that it helps create the... (read more)

Replying toWhy the AI Alignment Problem Might be Unsolvable?

Except that for humans, life is a journey, not a destination. If you make a maximize flourishing optimizer you would need to rigorously define what you meant by flourishing, which requires a rigorous definition of a general human utility function, which doesnt and cannot exist. Human values are instrumental all the way down. Some values are just more instrumental than others--that is the mechanism which allows for human values to be over 4d experiences rather than 3d states. I mean, what other mechanism could result in that for a human mind? This is a natural implication of "adaptation executors not fitness maximizers".

And I will note that humans tend to care a lot

good point, I missed that, will fix later. more likely that effect would result from programming the AI with the overlap between those utility functions, but I'm not totally sure so I'll have to think about it. I don't think that point is actually necessary for the crux of my argument, though. Like I said, I'll have to think about it. Right now it's almost 4am and Im really sick now.

A Tale of Four Moralities

Author's note 1:

The following is a chapter from the story I've been writing which contains, well, it contains what I think is probably a proof that the value alignment problem is unsolvable. I know it sounds crazy, but as far as I can tell the proof seems to be correct. There are further supporting details which I can explain if anyone asks, but I didn't want to overload you guys with too much information at once, since a lot of those additional supporting details would require articles of their own to explain.

One of my friends, who I shall not name, came up with what we think is also a proof, but it's... (read 1839 more words →)

Author's note: This is a children's story I wrote a while back, which teaches a very important life lesson that none of us got to learn as kids. That lesson is extremely important, so all the adults here should pay attention too. I'll explain more of the nitty gritty details of the underlying theory behind it later.

----------

Ivan was very angry.
His teddy was stolen.

Ivan decided.
He would catch the thief and steal from them.

"This will pay them back," said Ivan. "Serves them right."

Goldie was very happy.
It was her birthday.
Her papa gave her a teddy.

Goldie decided.
She would give a gift to her papa in return.

"It was nice of him to give me a... (read 904 more words →)

Replying toNo Really, Why Aren't Rationalists Winning?

No Really, Why Aren't Rationalists Winning?

In other words, people who win at offline life spend less time on the internet because they're devoting more time offline. And since rationalists are largely an online community rather than offline at least outside of the bay area, this results in rationalists dropping out of the conversation when they start winning. That's a surprisingly plausible alternative explanation. I'll have to think about this.

No Really, Why Aren't Rationalists Winning?

Reply to Extreme Rationality: It's Not That Great, Extreme Rationality: It could Be Great, the Craft and the Community and Why Don't Rationalists Win?

I’m going to say something which might be extremely obvious in hindsight:

If LessWrong had originally been targeted at and introduced to an audience of competent business people and self-improvement health buffs instead of an audience of STEM specialists and Harry Potter fans, things would have been drastically different. Rationalists would be winning.

Right now, rationalists aren’t winning. Rationality helps us choose which charities to donate to, and as Scott Alexander pointed out in 2009 it gives clarity of mind benefits. However, as he also pointed out in the same article,... (read 1434 more words →)

Replying toWhy do we like stories?

Why do we like stories?

So everything we do in life is problem solving and therefore storytelling was originally a form of problem solving, and this explains the origin of storytelling how? This seems like saying "the sky is made of quarks, all matter is made of quarks. Therefore this explains the origins of the sky." But just saying "quarks!" doesn't tell you where the quarks are and where they're going and how far away they all are from each other in what directions. And the positions of all the many quarks involved are too many to keep track of them all individually with a human level intelligence in any reasonable time frame. Sure, in theory you

... (read more)

-3

Replying toAnti-social Punishment

Anti-social Punishment

In some societies it might not be considered socially acceptable to want to punish someone merely because what they are doing will raise their social status. That sort of thing is dishonest because social status is reputational and meant to be earned. If someone tries to punish you for doing something to earn status, they probably did not come by their social status by honest means.

In societies where people think like that, I imagine no one would want to say "this act of altruism will increase their status and so should be punished", because that is a low status motive and expressing it out loud will lower their own status. So instead

Extreme Rationality: It's Not That Great

This. If less wrong had been introduced to an audience of self-improvement health buffs and business people instead of nerdy booksmart Harry Potter fans, things would have been drastically different. it is possible to become more effective at optimizing for other goals besides just truth. People here seem to naively assume so as long as they have enough sufficiently accurate information everything else will simply fall into place and they'll do everything else right automatically without needing to really practice or develop any other skills. I will be speaking more on this later.

Subjunctive Tenses Unnecessary for Rationalists?

Do any Spanish speaking rationalists use subjunctive tenses? I've had the sense for a while that it's necessary for when you're expressing unfocused uncertainty or a strong emotional attachment to a belief or opinion, but that most of the time you can get by just fine with indicative tenses only. You can express uncertainty as a factual statement about how certain you are of a claim's accuracy. You can express strong emotions by making a factual statement about how you feel about something.

Then again, maybe it takes more words to do it that way. But surely emotions can be gleaned well enough from context and tone without using subjunctive tenses. Other languages

... (read more)

Replying toThe Case For Gods

The Case For Gods

Except that you're using "useful to believe" as a criteria for determining whether something is true or not. Also, if you had developed the skills, qualities, attitudes, and habits necessary to handle the truth in a sane and healthy manner, you wouldn't need to believe in a God, because you would know how to live with the knowledge that there is no God and not be broken by it. If you truly had developed the ability to handle the truth safely, it wouldn't matter what the truth was, you'd be able to handle it regardless. That is to say, if a God does not exist, you would be able to handle that

... (read more)

-1

I am Sailor Vulcan!