capybaralet

capybaralet's Comments

What I talk about when I talk about AI x-risk: 3 core claims I want machine learning researchers to address.

I'm definitely interested in hearing other ways of splitting it up! This is one of the points of making this post. I'm also interested in what you think of the ways I've done the breakdown! Since you proposed an alternative, I guess you might have some thoughts on why it could be better :)

I see your points as being directed more at increasing ML researchers respect for AI x-risk work and their likelihood of doing relevant work. Maybe that should in fact be the goal. It seems to be a more common goal.

I would describe my goal (with this post, at least, and probably with most conversations I have with ML people about Xrisk) as something more like: "get them to understand the AI safety mindset, and where I'm coming from; get them to really think about the problem and engage with it". I expect a lot of people here would reason in a very narrow and myopic consequentialist way that this is not as good a goal, but I'm unconvinced.

What I talk about when I talk about AI x-risk: 3 core claims I want machine learning researchers to address.

TBC, it's an unconference, so it wasn't really a talk (although I did end up talking a lot :P).

How sure are you that the people who showed up were objecting out of deeply-held disagreements, and not out of a sense that objections are good?

Seems like a false dichotomy. I'd say people were mostly disagreeing out of not-very-deeply-held-at-all disagreements :)

A list of good heuristics that the case for AI x-risk fails

Another important improvement I should make: rephrase these to have the type signature of "heuristic"!

What I talk about when I talk about AI x-risk: 3 core claims I want machine learning researchers to address.

No, my goal is to:

  • Identify a small set of beliefs to focus discussions around.
  • Figure out how to make the case for these beliefs quickly, clearly, persuasively, and honestly.

And yes, I did mean >1%, but I just put that number there to give people a sense of what I mean, since "non-trivial" can mean very different things to different people.


A list of good heuristics that the case for AI x-risk fails

Oh sure, in some special cases. I don't this this experience was particularly representative.

A list of good heuristics that the case for AI x-risk fails

Yeah I've had conversations with people who shot down a long list of concerned experts, e.g.:

  • Stuart Russell is GOFAI ==> out-of-touch
  • Shane Legg doesn't do DL, does he even do research? ==> out-of-touch
  • Ilya Sutskever (and everyone at OpenAI) is crazy, they think AGI is 5 years away ==> out-of-touch
  • Anyone at DeepMind is just marketing their B.S. "AGI" story or drank the koolaid ==> out-of-touch

But then, even the big 5 of deep learning have all said things that can be used to support the case....

So it kind of seems like there should be a compendium of quotes somewhere, or something.

Clarifying some key hypotheses in AI alignment

Nice chart!

A few questions and comments:

  • Why the arrow from "agentive AI" to "humans are economically outcompeted"? The explanation makes it sounds like it should point to "target loading fails"??
  • Suggestion: make the blue boxes without parents more apparent? e.g. a different shade of blue? Or all sitting above the other ones? (e.g. "broad basin of corrigibility" could be moved up and left).
A list of good heuristics that the case for AI x-risk fails

I pushed this post out since I think it's good to link to it in this other post. But there are at least 2 improvements I'd like to make and would appreciate help with:

LessWrong anti-kibitzer (hides comment authors and vote counts)

The link to user preferences is broken. Is there still this feature built-in? Or does the firefox thing still work?

Load More