RedFishBlueFish

Crossposting from my substack.

I figured I would crosspost to LessWrong since electoral systems seem like the type of thing rationalists would be interested in.

This is the latest post in my constitutional design series. It is the quintessential topic of constitutional design, the one that everyone loves to talk about. I hope to show and have shown that constitutional design is far more than just electoral systems, but it’s obviously an important topic and worth discussing. First I’ll discuss electoral systems for electing a single person, and then I’ll discuss multi-member electoral systems for electing a legislature. But before that…

Social Choice Theory Background

Maximizing Societal Utility

If you want a prescription for who objectively “should”... (read 3918 more words →)

RedFishBlueFish2y

Yeah so I think Tomasik has written basically what I’m saying https://reducing-suffering.org/dissolving-confusion-about-consciousness/

RedFishBlueFish2y

I can think of two interpretations of consciousness being "causally connected" to physical systems:

1. consciousness is the result of physical phenomena like brain states, but it does not cause any. So it has an in-edge coming from the physical world, but not an out-edge to the physical world. Again, this implies that consciousness cannot be what causes me to think about consciousness.

2. consciousness causes things in the physical world. Which, again, I believe, necessitates a consciousness variable in the laws of the universe.

Note that I am not trying to get at what Eliezer was arguing, I am asking about the consequences of his arguments, even ones that he may not have intended.

RedFishBlueFish2y

So does this suppose that there is some “consciousness” variable in the laws of the universe? If consciousness causes me to think X, and thinking X can be traced back to a set of physical laws that govern the neurons in my brain, then there must be some consciousness variable somewhere in these physical laws, no? Otherwise it has to be that consciousness corresponds to some physical phenomenon, and it is that phenomenon - not the consciousness - that caused you to think about it. If there were no consciousness attached to that physical phenomenon, you would go along just the same way, thinking the exact same thing.

RedFishBlueFish2yQuick Take

I was reading this old Eliezer piece arguing against the conceivability of p-zombies. https://www.lesswrong.com/posts/7DmA3yWwa6AT5jFXt/zombies-redacted

And to me this feels like a more general argument against the existence of consciousness itself, in any form similar to how we normally think about it, not just against p-zombies.

Eliezer says that consciousness itself cannot be what causes me to think about consciousness, that causes philosophers to write papers about consciousness, thus it must be the physical system that such consciousness corresponds to that causes these things. But then…that seems to discredit the ability to use your own consciousness as evidence for anything. If my own conscious experience cannot be evidence for me thinking about consciousness, why should we think consciousness exists?

Am I confused about something here?

RedFishBlueFish's Shortform

RedFishBlueFish

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

Replying toDoes AI care about reality or just its own perception?

RedFishBlueFish2y

Does AI care about reality or just its own perception?

That's not a simple problem.First you have to specify "not killing everyone" robustly (outer alignment) and then you have to train the AI to have this goal and not an approximation of it (inner alignment).

See my other comment for the response.

Anyway, the rest of your response is spent talking about the case where AI cares about its perception of the paperclips rather than the paperclips themselves. I'm not sure how severity level 1 would come about, given that the AI should only care about its reward score. Once you admit that the AI cares about worldly things like "am I turned on", it seems pretty natural that the AI would care about... (read more)

Replying toDoes AI care about reality or just its own perception?

RedFishBlueFish2y

Does AI care about reality or just its own perception?

We don't know how to represent "do not kill everyone"

I think this goes to Matthew Barnett’s recent article of actually yes we do. And regardless I don’t think this point is a big part of Eliezer’s argument. https://www.lesswrong.com/posts/i5kijcjFJD6bn7dwq/evaluating-the-historical-value-misspecification-argument

We don't know how to pick which quantity would be maximized by a would-be strong consequentialist maximizer

Yeah so I think this is the crux of it. My point is that if we find some training approach that leads to a model that cares about the world itself rather than hacking some reward function, that’s a sign that we can in fact guide the model in important ways and there’s a good chance this includes being... (read more)

Replying toDoes AI care about reality or just its own perception?

RedFishBlueFish2y

Does AI care about reality or just its own perception?

I also do not think the responses to this question are satisfying enough to be refuting. I don’t even think they are satisfying enough to make me confident I haven’t just found a hole in AI risk arguments.This is not a simple case of “you just misunderstand something simple”.

Replying toDoes AI care about reality or just its own perception?

RedFishBlueFish2y

Does AI care about reality or just its own perception?

I don’t care that much but if LessWrong is going to downvote sincere questions because it finds them dumb or whatever this will make for a site very unwelcoming to newcomers

Does AI care about reality or just its own perception?

RedFishBlueFish

Does a paperclip-maximizing AI care about the actual number of paperclips being made, or does it just care about its perception of paperclips?

If the latter, I feel like this contradicts some of the AI doom stories: each AI shouldn’t care about what future AIs do (and thus there is no incentive to fake alignment for the benefit of future AIs), and the AIs also shouldn’t care much about being shut down (the AI is optimizing for its own perception; when it’s shut off, there is nothing to optimize for).

If the former, I think this makes alignment much easier. As long as you can reasonably represent “do not kill everyone”, you can make this a goal of the AI, and then it will literally care about not killing everyone, it won’t just care about hacking its reward system so that it will not perceive everyone being dead.

-6

Replying toThe Dictatorship Problem

RedFishBlueFish3y

The Dictatorship Problem

I do indeed agree this is a major problem even if I'm not sure if I agree with the main claim. The rise of fascism in the last decade and expectation that it will continue is extremely evident; its consequences for democracy are a lot less clear.

The major wrinkle in all of this is in assessing anti-democratic behavior. Democracy indices not a great way of assessing democracy for much the same reason that the Doomsday Clock is a bad way of assessing nuclear risk: they're subjective metrics by (probably increasingly) left-leaning academics and tend to measure a lot of things that I wouldn't classify as democracy (eg rights of women/LGBT people/minorities). This... (read more)

Replying toAI scares and changing public beliefs

RedFishBlueFish3y

AI scares and changing public beliefs

I'm going to quote this from an EA Forum post I just made for why simply repeated exposure to AI Safety (through eg media coverage) will probably do a lot to persuade people:

[T]he more people hear about AI Safety, the more seriously people will take the issue. This seems to be true even if the coverage is purporting to debunk the issue (which as I will discuss later I think will be fairly rare) - a phenomenon called the illusory truth effect. I also think this effect will be especially strong for AI Safety. Right now, in EA-adjacent circles, the argument over AI Safety is mostly a war of vibes. There is

... (read more)

Keep Making AI Safety News

RedFishBlueFish

Crossposted from the EA Forum

AI Safety is hot right now.

The FLI letter was the catalyst for most of this, but even before that there was the Ezra Klein OpEd piece in the NYTimes. (Also general shoutout to Ezra for helping bring EA ideas to the mainstream - he's great!).

Since the FLI letter, there was there was this CBS interview with Geoffrey Hinton. There was this WSJ Op-Ed. Eliezer's Time OpEd and Lex Fridman interview led to Bezos following him on Twitter. Most remarkably to me, Fox News reporter Peter Doocey asked a question in the White House press briefing, which got a serious (albeit vague) response. The president of the United States,... (read 277 more words →)

Why does advanced AI want not to be shut down?

RedFishBlueFish

I've always been pretty confused about this.

The standard AI risk scenarios usually (though I think not always) suppose that advanced AI wants not to be shut down. As commonly framed, the AI will fool humanity into believing it is aligned so as not to be turned off, until - all at once - it destroys humanity and gains control over all earth's resources.

But why does the AI want not to be shut down?

The motivation behind a human wanting not to die comes from evolution. You die before reproduction age, and you won't be able to pass on your not-afraid-of-death-before-reproduction-age genes. You die after reproduction age, and you won't be able to take... (read 270 more words →)

LESSWRONG
LW

LESSWRONG
LW