nigerweiss

Replying toSteelmanning Young Earth Creationism

It's going to be really hard to come up with any models that don't run deeply and profoundly afoul of the Occam prior.

Replying to[LINK] Slashdot interview with David Ettinger of the Cryonics Institute

[LINK] Slashdot interview with David Ettinger of the Cryonics Institute

When asked a simple question about broad and controversial assertions, it is rude to link to outside resources tangentially related to the issue without providing (at minimum) a brief explanation of what those resources are intended to indicate.

Replying toHarry Potter and the Methods of Rationality discussion thread, part 25, chapter 96

nigerweiss13y

Harry Potter and the Methods of Rationality discussion thread, part 25, chapter 96

I don't speak Old English, unfortunately. Could someone who does please provide me with a rough translation of the provided passage?

Replying toFor FAI: Is "Molecular Nanotechnology" putting our best foot forward?

nigerweiss13y

For FAI: Is "Molecular Nanotechnology" putting our best foot forward?

It isn't the sort of bad argument that gets refuted. The best someone can do is point out that there's no guarantee that MNT is possible. In which case, the response is 'Are you prepared to bet the human species on that? Besides, it doesn't actually matter, because [insert more sophisticated argument about optimization power here].' It doesn't hurt you, and with the overwhelming majority of semi-literate audiences, it helps.

Replying toFor FAI: Is "Molecular Nanotechnology" putting our best foot forward?

nigerweiss13y

For FAI: Is "Molecular Nanotechnology" putting our best foot forward?

Of course there is. For starters, most of the good arguments are much more difficult to concisely explain, or invite more arguments from flawed intuitions. Remember, we're not trying to feel smug in our rational superiority here; we're trying to save the world.

Replying toFor FAI: Is "Molecular Nanotechnology" putting our best foot forward?

nigerweiss13y

For FAI: Is "Molecular Nanotechnology" putting our best foot forward?

That's... not a strong criticism. There are compelling reasons not to believe that God is going to be a major force in steering the direction the future takes. The exact opposite is true for MNT - I'd bet at better-than-even odds that MNT will be a major factor in how things play out basically no matter what happens.

All we're doing is providing people with a plausible scenario that contradicts flawed intuitions that they might have, in an effort to get them to revisit those intuitions and reconsider them. There's nothing wrong with that. Would we need to do it if people were rational agents? No - but, as you may be aware, we definitely don't live in that universe.

Replying toFor FAI: Is "Molecular Nanotechnology" putting our best foot forward?

nigerweiss13y

For FAI: Is "Molecular Nanotechnology" putting our best foot forward?

I don't have an issue bringing up MNT in these discussions, because our goal is to convince people that incautiously designed machine intelligence is a problem, and a major failure mode for people is that they say really stupid things like 'well, the machine won't be able to do anything on its own because it's just a computer - it'll need humanity, therefore, it'll never kill us all." Even if MNT is impossible, that's still true - but bringing up MNT provides people with an obvious intuitive path to the apocalypse. It isn't guaranteed to happen, but it's also not unlikely, and it's a powerful educational tool for showing people the sorts of things that strong AI may be capable of.

Replying toMahatma Armstrong: CEVed to death.

nigerweiss13y

Mahatma Armstrong: CEVed to death.

There's a deeper question here: ideally, we would like our CEV to make choices for us that aren't our choices. We would like our CEV to give us the potential for growth, and not to burden us with a powerful optimization engine driven by our childish foolishness.

One obvious way to solve the problem you raise is to treat 'modifying your current value approximation'' as an object-level action by the AI, and one that requires it to compute your current EV - meaning that, if the logical consequences of the change (including all the future changes that the AI predicts will result from that change) don't look palatable to you,... (read more)

Replying toRationality Quotes June 2013

nigerweiss13y

Rationality Quotes June 2013

I've read some of Dennet's essays on the subject (though not the book in question), and I found that, for me,his ideas did help to make consciousness a good deal less mysterious. What actually did it for me was doing some of my own reasoning about how a 'noisy quorum' model of conscious experience might be structured, and realizing that, when you get right down to it, the fact that I feel as though I have subjective experience isn't actually that surprising. It'd be hard to design to a human-style system that didn't have a similar internal behavior that it could talk about.

Replying toWho thinks quantum computing will be necessary for AI?

nigerweiss13y

Who thinks quantum computing will be necessary for AI?

Yeah, The glia seem to serve some pretty crucial functions as information-carriers and network support infrastructure - and if you don't track hormonal regulation properly, you're going to be in for a world of hurt. Still, I think the point stands.

How should negative externalities be handled? (Warning: politics)

nigerweiss

13y

Politics ahead! Read at your own risk, mind killers, etc. Let all caveats be well and thoroughly emptored.

It seems reasonably clear to me that, from a computational perspective, functional central planning is not practically possible. Resource allocation among many agents looks an awful lot like an exponential time problem, and the world market is quite an efficient approximation. In the real world, markets, regulated to preclude blackmail, theft, and slavery, will tend to provide a better approximation of "correct" resource allocation between free agents than a central resource allocation algorithm could plausibly achieve without a tremendous, invasive amount of information about the desires of every market participant, and quite a lot of... (read 701 more words →)

132

-14

A Series of Increasingly Perverse and Destructive Games

nigerweiss

13y

Related to: Higher Than the Most High

The linked post describes a game in which (I fudge a little), Omega comes to you and two other people, and ask you to tell him an integer. The person who names the largest integer is allowed to leave. The other two are killed.

This got me thinking about variations on the same concept, and here's what I've come up, taking that game to be GAME₀. The results are sort of a fun time-waster, and bring up some interesting issues. For your enjoyment...

THE GAMES:

GAME1: Omega takes you and two strangers (all competent programmers), and kidnaps and sedates you. You awake in three rooms with instructions printed on the... (read 832 more words →)

Inferring Values from Imperfect Optimizers

nigerweiss

13y

One approach to constructing a Friendly artificial intelligence is to create a piece of software that looks at large amounts of evidence about humans, and attempts to infer their values. I've been doing some thinking about this problem, and I'm going to talk about some approaches and problems that have occurred to me.

In a naive approach, we might define the problem like this: take some unknown utility function, U, and plug it into a mathematically clean optimization process (like AIXI) O. Then, look at your data set and take the information about the inputs and outputs of humans, and find the simplest U that best explains human behavior.

Unfortunately, this won't work. The best possible... (read 604 more words →)

AI "Boxing" and Utility Functions

nigerweiss

13y

So, I had this idea the other day when I was thinking about how to safely conduct research on potentially-FOOM-capable AI software. I'd like to sketch it out briefly and then get feedback on it.

So, this started out with the idea that an AI based on AIXI is, in some sense, safer than a fully functional AI, due to the existence of the anvil problem. Because AIXI can't conceive of its own nonexistence, it has no preference ordering over its own mortality, and won't (shouldn't) resist any attempt to shut it down. In other words, if AIXI starts to FOOM undesirably out of control, you actually can go pull the plug on... (read 395 more words →)

LESSWRONG
LW

LESSWRONG
LW

A Series of Increasingly Perverse and Destructive Games

Inferring Values from Imperfect Optimizers

AI "Boxing" and Utility Functions

How should negative externalities be handled? (Warning: politics)

nigerweiss

How should negative externalities be handled? (Warning: politics)

A Series of Increasingly Perverse and Destructive Games

Inferring Values from Imperfect Optimizers

AI "Boxing" and Utility Functions

nigerweiss

A Series of Increasingly Perverse and Destructive Games

Inferring Values from Imperfect Optimizers

AI "Boxing" and Utility Functions

How should negative externalities be handled? (Warning: politics)

nigerweiss

How should negative externalities be handled? (Warning: politics)

A Series of Increasingly Perverse and Destructive Games

Inferring Values from Imperfect Optimizers

AI "Boxing" and Utility Functions