All of WilliamKiely's Comments + Replies

2Yair Halberstadt8mo
Search here for spoiler for instructions on how to use spoiler tags: https://www.lesswrong.com/posts/2rWKkWuPrgTMpLRbp/lesswrong-faq

Retracted due to spoilers and not knowing how to use spoiler tags.

2Yair Halberstadt8mo
Consider hiding your answer using spoiler tags.

Received $400 worth of bitcoin. I confirm the bet.

@RatsWrongAboutUAP I'm willing to risk up to $20k at 50:1 odds (i.e. If you give me $400 now, I'll owe you $20k in 5 years if you win the bet) conditional on (1) you not being privy to any non-public information about UFOs/UAP and (2) you being okay with forfeiting any potential winnings in the unlikely event that I die before bet resolution.

Re (1): Could you state clearly whether you do or do not have non-public information pertaining to the bet?

Re (2): FYI The odds of me dying in the next 5 years are less than 3% by SSA base rates, and my credence is eve... (read more)

2RatsWrongAboutUAP8mo
1. I do not have any non-public information about ufos/uap. 2. Sure Please verify reception of funds and confirm our bet https://blockstream.info/tx/ab7173abec208a6eda17bdf1b75668bc5e6efe46356f40109125a42962bfb9e2

IMO the largest trade-offs of being vegan for most people aren't health trade-offs, but they're other things like the increased time/attention cost of identifying non-vegan foods. Living in a place where there's a ton of non-vegan food available at grocery stores and restaurants makes it more of a pain to get food at stores and restaurants than it is if you're not paying that close attention to what's in your food. (I'm someone without any food allergies, and I imagine being vegan is about as annoying as having certain food allergies).

That being said, it also seems to me that the vast majority of people's diets are not well optimized for health. Most people care about convenience, cost, taste, and other factors as well. My intuition is that if we took a random person and said "hey, you have to go vegan, lets try to find a vegan diet that's healthier than your current diet" that we'd succeed the vast majority of the time simply because most people don't eat very healthily. That said, the random person would probably prefer a vegan diet optimized for things other than just health more than a vegan diet optimized for just health.

I only read the title, not the post, but just wanted to leave a quick comment to say I agree that veganism entails trade-offs, and that health is one of the axes. Also note that I've been vegan since May 2019 and lacto-vegetarian since October 2017, for ethical reasons, not environmental or health or other preferences reasons.

It's long (since before I changed my diet) been obvious to me that your title statement is true since a prior it seems very unlikely that the optimal diet for health is one that contains exactly zero animal products, given that humans are omnivores. One doesn't need to be informed about nutrition to make that inference.

3orthonormal9mo
In this and your comments below, you recapitulate points Elizabeth made pretty exactly- so it looks like you didn't need to read it after all!
1Portia9mo
Mh, I think we need to distinguish two different things here. Is veganism healthier than all other diets? - Doubt it. While reducing meat consumption has demonstrable benefits, I don't think you get any particular advantage from removing all animal products entirely to the last tiniest piece, despite the known harms many of them entail. I doubt a little bit of animal products now and then does much harm, plus there are some nutrient forms you get more easily that way to outweigh that. I would not universally recommend going vegan for health reasons. But can you live a very healthy, long life while vegan, just like an omni? - With rare exceptions (severe food intolerances or food access issues), it seems like the answer is yes; people have been raised vegan from birth, been vegan for decades, and they do perfectly fine. So I do not think I will die earlier, or later, because I am vegan. And that is good enough for me. I care a lot about animal rights, but I would not make myself sick over them. (E.g. I take non-vegan medications, because I actually need those to be healthy. I wish they existed in vegan, and push for it, but until they do, I will take them as is.) But the mere inconvenience of reading food labels or not being able to eat the cake in a traditional restaurant, that I am fine with. Ironically, I think being vegan was healthier 1,5 decades ago. At the time, there were no fancy fake replacement products, so going vegan effectively entailed quitting practically all chocolate, candy, cake, takeout, etc. and having to cook most everything oneself from whole foods from scratch. Nowadays, I can buy the exact same crap in vegan, and the sugar content is just as bad, and I doubt the vegan options are healthier at all.
2WilliamKiely9mo
IMO the largest trade-offs of being vegan for most people aren't health trade-offs, but they're other things like the increased time/attention cost of identifying non-vegan foods. Living in a place where there's a ton of non-vegan food available at grocery stores and restaurants makes it more of a pain to get food at stores and restaurants than it is if you're not paying that close attention to what's in your food. (I'm someone without any food allergies, and I imagine being vegan is about as annoying as having certain food allergies).
2WilliamKiely9mo
That being said, it also seems to me that the vast majority of people's diets are not well optimized for health. Most people care about convenience, cost, taste, and other factors as well. My intuition is that if we took a random person and said "hey, you have to go vegan, lets try to find a vegan diet that's healthier than your current diet" that we'd succeed the vast majority of the time simply because most people don't eat very healthily. That said, the random person would probably prefer a vegan diet optimized for things other than just health more than a vegan diet optimized for just health.

Probability that most humans die because of an AI takeover: 11%

This 11% is for "within 10 years" as well, right?

Probability that the AI we build doesn’t take over, but that it builds even smarter AI and there is a takeover some day further down the line: 7%

Does "further down the line" here mean "further down the line, but still within 10 years of building powerful AI"? Or do you mean it unqualified?

4paulfchristiano10mo
I think almost all the cumulative takeover probability is within 10 years of building powerful AI. Didn't draw the distinction here, but my views aren't precise enough to distinguish.

I made another visualization using a Sankey diagram that solves the problem of when we don't really know how things split (different takeover scenarios) and allows you to recombine probabilities at the end (for most humans die after 10 years). 

More geometric (but less faithful):

But to my mind, such a scenario is implausible (much less than one percent probability overall) because it stacks up too many unlikely assumptions in terms of our prior experiences with related systems.

You mentioned 5-6 assumptions. I think at least one isn't needed (that the goal changes as it self-improves), and disagree that the others are (all) unlikely. E.g. Agentic, non-tool AIs are already here and more will be coming (foolishly). Taking a point I just heard from Tegmark on his latest Lex Fridman podcast interview, once companies add APIs to systems... (read more)

This is the fear of “foom,”

I think the popular answer to this survey also includes many slow takeoff, no-foom scenarios.

And then, when humans are worth more to the advance of this AI’s radically changed goals as mere atoms than for all the things we can do, it simply kills us all.

I agree with this, though again I think the "changed" can be ommitted.

Secondly, I also think it's possible that rather than the unaligned superintelligence killing us all in the same second like EY often says, that it may kill us off in a manner like how humans kill off other species (i.e. we know we are doing it, but it doesn't look like a war.)

Re my last point, see Ben Weinstein-Raun's vision her... (read more)

Furthermore, the goals of this agent AI change radically over this growth period.

Noting that this part doesn't seem necessary to me. The agent may be misaligned before the capability gain.

1WilliamKiely10mo
I agree with this, though again I think the "changed" can be ommitted. Secondly, I also think it's possible that rather than the unaligned superintelligence killing us all in the same second like EY often says, that it may kill us off in a manner like how humans kill off other species (i.e. we know we are doing it, but it doesn't look like a war.) Re my last point, see Ben Weinstein-Raun's vision here: https://twitter.com/benwr/status/1646685868940460032

Plausibly, such “ems” may long remain more cost-effective than AIs on many important tasks.

"Plausibly" (i.e. 'maybe') is not enough here to make the fear irrational ("Many of these AI fears are driven by the expectation that AIs would be cheaper, more productive, and/or more intelligent than humans.")

In other words, while it's reasonable to say "maybe the fears will all be for nothing", that doesn't mean it's not reasonable to be fearful and concerned due to the stakes involved and the nontrivial chance that things do go extremely badly.

And yes, even if AIs behave predictably in ordinary situations, they might act weird in unusual situations, and act deceptively when they can get away with it. But the same applies to humans, which is why we test in unusual situations, especially for deception, and monitor more closely when context changes rapidly.

"But the same applies to humans" doesn't seem like an adequate response when the AI system is superintelligent or past the "sharp left turn" capabilities threshold. Solutions that work for unaligned deceptive humans won't save us from a sufficiently intelligent/capable unaligned deceptive entity.

If we like where we are and can’t be very confident of where we may go, maybe we shouldn’t take the risk and just stop changing. Or at least create central powers sufficient to control change worldwide, and only allow changes that are widely approved. This may be a proposal worth considering, but AI isn’t the fundamental problem here either.

I'm curious what you (Hanson) think(s) *is* the fundamental problem here if not AI?

Context: It seems to me that Toby Ord is right that the largest existential risks (AI being number one) are all anthropormphic risks, ra... (read more)

Doomers worry about AIs developing “misaligned” values. But in this scenario, the “values” implicit in AI actions are roughly chosen by the organisations who make them and by the customers who use them.

There is reason to think "roughly" aligned isn't enough in the case of a sufficiently capable system.

Second, Robin's statement seems to ignore (or contradict without making an argument) the fact that even if it is true for systems not as smart as humans, there may be a "sharp left turn" at some point where, in Nate Soares' words, "as systems start to work re... (read more)

Yudkowsky and others might give different reasons why waiting until later to gain more information about the future systems doesn't make sense, including pointing out that that may lead us to missing our first "critical try."

Robin, I know you must have heard these points before--I believe you are more familiar with e.g. Eliezer's views than I am. But if that's the case I don't understand why you would write a sentence like last one in the quotation above. It sounds like a cheap rhetorical trick to say "but instead of waiting to deal with such problems when... (read more)

To clarify explicitly, people like Stuart Russell would point out that if future AIs are still built according to the "standard model" (a phrase I borrow from Russell) like the systems of today, then they will continue to be predictably misaligned.

1WilliamKiely10mo
Yudkowsky and others might give different reasons why waiting until later to gain more information about the future systems doesn't make sense, including pointing out that that may lead us to missing our first "critical try." Robin, I know you must have heard these points before--I believe you are more familiar with e.g. Eliezer's views than I am. But if that's the case I don't understand why you would write a sentence like last one in the quotation above. It sounds like a cheap rhetorical trick to say "but instead of waiting to deal with such problems when we understand them better and can envision them more concretely" especially without saying why people who don't think we should wait don't think that's a good enough reason to wait / think there are pressing reasons to work on the problems now despite our relative state of ignorance compared to future AI researchers.

This part doesn't seem to pass the ideological Turing test:

At the moment, AIs are not powerful enough to cause us harm, and we hardly know anything about the structures and uses of future AIs that might cause bigger problems. But instead of waiting to deal with such problems when we understand them better and can envision them more concretely, AI “doomers” want stronger guarantees now.

1WilliamKiely10mo
To clarify explicitly, people like Stuart Russell would point out that if future AIs are still built according to the "standard model" (a phrase I borrow from Russell) like the systems of today, then they will continue to be predictably misaligned.

I strongly agree with this request.

If companies don't want to be the first to issue such a statement then I suggest they coordinate and share draft statements with each other privately before publishing simultaneously.

Question: Are innerly-misaligned (superintelligent) AI systems supposed to necessarily be squiggle maximizers, or are squiggle maximizers supposed to only be one class of innerly-misaligned systems?

It'd be nice if Hassabis made another public statement about his views on pausing AI development and thoughts on the FLI petition. If now's not the right time in his view, when is? And what can he do to help with coordination of the industry?

On the subject of DeemMind and pausing AI development, I'd like to highlight Demis Hassabis's remark on this topic in a DeepMind podcast interview a year ago:

'Avengers assembled' for AI Safety: Pause AI development to prove things mathematically

Hannah Fry (17:07):

You said you've got this sort of 20-year prediction and then simultaneously where society is in terms of understanding and grappling with these ideas. Do you think that DeepMind has a responsibility to hit pause at any point?

Demis Hassabis (17:24):

Potentially. I always imagine that as we got close

... (read more)

It'd be nice if Hassabis made another public statement about his views on pausing AI development and thoughts on the FLI petition. If now's not the right time in his view, when is? And what can he do to help with coordination of the industry?

Demis Hassabis didn't sign the letter, but has previously said that DeepMind potentially has a responsibility to hit pause at some point:

'Avengers assembled' for AI Safety: Pause AI development to prove things mathematically

Hannah Fry (17:07):

You said you've got this sort of 20-year prediction and then simultaneously where society is in terms of understanding and grappling with these ideas. Do you think that DeepMind has a responsibility to hit pause at any point?

Demis Hassabis (17:24):

Potentially. I always imagine that as we got closer to the sort of gray

... (read more)

Feedback on the title: I don't like the title because it is binary.

Saying X is "good" or "bad" at something isn't very informative.

There are many degrees of goodness. Was it worse than you thought it would be before you played around with it a bit more? Was it worse than some popular article or tweet made you think? Was it worse than some relevant standard?

Loved the first paragraph:

In 2022, over 700 top academics and researchers behind the leading artificial intelligence companies were asked in a survey about future A.I. risk. Half of those surveyed stated that there was a 10 percent or greater chance of human extinction (or similarly permanent and severe disempowerment) from future AI systems. Technology companies building today’s large language models are caught in a race to put all of humanity on that plane.

Idea: Run a competition to come up with other such first paragraphs people can use in similar op ed... (read more)

For general public, the Youtube posting is now up—it has 80 comments so far. There are also likely other news articles citing this interview that may have comment sections.

I'm not John, but if you interpret "epsilon precautions" as meaning "a few precautions" and "pre-galaxy-brained" as "before reading Zvi's Galaxy Brained Take interpretation of the film" I agree with his comment.

1M. Y. Zuo1y
I thought 'epsilon' was euphemism for 'practically but not literally zero'. But then it wouldn't seem to make sense for John to recommend it as great, since that seems to reinforce the point the character was holding an 'idiot ball', hence my question. At least that's the impression I got from this. If the character took a 'few precautions' in the sense of a limited number out of the full range, for whatever reasons, then the required suspension of disbelief might not be a dealbreaker.

I just thought of a flaw in my analysis, which is that if it's intractable to make AI alignment more or less likely (and intractable to make the development of transformative AI more or less safe), then accelerating AI timelines actually seems good because the benefits to people post-AGI if it goes well (utopian civilization for longer) seem to outweigh the harms to people pre-AGI if goes badly (everyone on Earth dies sooner). Will think about this more.

Curious if you ever watched M3GAN?

I can't stand it, and I struggle to suspend my disbelief after lazy writing mistakes like this.

FWIW this sort of thing bothers me in movies a ton, but I was able to really enjoy M3GAN when going into it wanting it to be good and believing it might be due to reading Zvi's Mostly Spoiler-Free Review In Brief.

Yes, it's implausible that Gemma is able to build the protype at home in a week. The writer explains that she's using data from the company's past toys, but this still doesn't explain why a similar AGI hasn't been built ... (read more)

Dumb characters really put me off in most movies, but in this case I think it was fine. Gemma and her assistant's jobs are both on the line if M3GAN doesn't pan out, so they have an incentive to turn a blind eye to that. Also, their suspicions that M3GAN was dangerous weren't blatantly obvious such that people who lacked security mindsets (as some people do in real life) couldn't miss them.

I was thinking the characters were all being very stupid taking big risks when they created this generally intelligent agentic protype M3GAN, but given that we live in a... (read more)

2johnlawrenceaspden1y
Yes, it's really hard to believe that people are that stupid, even when you're surrounded by very bright people being exactly that stupid.  And the characters in the film haven't even had people screaming at them about their obvious mistakes morning til night for the last twenty years. At that point the film would fail willing suspension of disbelief so hard it would be unwatchable.

This review of M3GAN didn't get the attention it deserves!

I only just came across your review a few hours ago and decided to stop and watch the movie immediately after reading your Mostly Spoiler-Free Review In Brief section, before reading Aaronson's review and the rest of yours.

  • In my opinion, the most valuable part of this review is your articulation of how the film illustrates ~10 AI safety-related problems (in the Don’t-Kill-Everyoneism Problems section).
  • This is now my favorite post of yours, Zvi, thanks to the above and your amazing Galaxy Brained Tak
... (read more)

Thanks for sharing. FWIW I would have preferred to read the A Way to Be Okay section first, and only reference the other sections if things didn't make sense (though I think I would have understood it all fine without those five sections) (though I didn't know this in advance so I just read the essay from beginning to end).

The main benefits of the project are presumably known to the engineer engaging in it. It was the harm of the project (specifically the harm arising from how the project accelerates AI timelines) that the engineer was skeptical was significant that I wanted to look at more closely to determine whether it was large enough to make it questionable whether engaging in the project was good for the world.

Given my finding that a 400-hour ML project (I stipulated the project takes 0.2 years of FTE work) would, via its effects on shortening AI timelines, shorten the... (read more)

0weverka1y
You said nothing about positive contributions.  When you throw away the positives, everything is negative.  

Thanks for the response and for the concern. To be clear, the purpose of this post was to explore how much a typical, small AI project would affect AI timelines and AI risk in expectation. It was not intended as a response to the ML engineer, and as such I did not send it or any of its contents to him, nor comment on the quoted thread. I understand how inappropriate it would be to reply to the engineer's polite acknowledgment of the concerns with my long analysis of how many additional people will die in expectation due to the project accelerating AI timel... (read more)

IIRC Linch estimated in an EA Forum post that we should spend up to ~$100M to reduce x-risk by 1 basis point, i.e. ~$1M per microdoom. Maybe nanodooms would be a better unit.

2Lone Pine1y
If your efforts improve the situation by 1 nanodoom, you've saved 8 people alive today.

Re: 1: Do Dane's Guestimate models ever yield >1 microdoom estimates for solo research projects? That sounds like a lot.

2WilliamKiely1y
IIRC Linch estimated in an EA Forum post that we should spend up to ~$100M to reduce x-risk by 1 basis point, i.e. ~$1M per microdoom. Maybe nanodooms would be a better unit.

my unconditional median TAI timeline is now something like 2047, with a mode around 2035, defined by the first year we get >30% yearly GWP growth as measured from a prior peak, or an event of comparable significance.

Given it's about to be 2023, this means your mode is 12 years away and your median is 24 years away. I'd expect your mode to be nearer than your median, but probably not that much nearer.

I haven't forecasted when we might get >30% yearly GWP growth or an event of comparable significance (e.g. x-risk) specifically, but naively I'd guess th... (read more)

5Matthew Barnett1y
These numbers were based on the TAI timelines model I built, which produced a highly skewed distribution. I also added several years to the timeline due to anticipated delays and unrelated catastrophes, and some chance that the model is totally wrong. My inside view prediction given no delays is more like a median of 2037 with a mode of 2029. I agree it appears the mode is much too near, but I encourage you to build a model yourself. I think you might be surprised at how much sooner the mode can be compared to the median.

Also every one of the organizations you named is a capabilities company which brands itself based on the small team they have working on alignment off on the side.

I'm not sure whether OpenAI was one of the organizations named, but if so, this reminded me of something Scott Aaronson said on this topic in the Q&A of his recent talk "Scott Aaronson Talks AI Safety":

Maybe the one useful thing I can say is that, in my experience, which is admittedly very limited—working at OpenAI for all of five months—I’ve found my colleagues there to be extremely serious

... (read more)

That's it, thank you!

How is...

I'm short on AGI timelines (50% by 2030)

...consistent with...

An AI paradigm as performance-enhancing as transformers is discovered by AI search. 30% by 2030

...?

Doesn't AGI imply the latter?

1Lao Mein1y
I'm using a weird definition of AI here, basically "an AI that can do my job". I'm imagining a cobbled-together system of transformers that individually automates everything I can do, thereby replacing most of the information jobs like coding, scientific research, and advertising. So in a lot of the worlds where AGI happens, there's no hard takeoff. AIs are helping do AI research, and maybe labor isn't a major limiting factor in AI development anymore. But there isn't a >1 OOM increase in AI research output from AI. This also means that I think in most of the 30% there is no hard takeoff. Some low-hanging fruit is picked by machines, but not enough for a FOOM. Thanks for bringing up the contradiction, though. I really need to go back and clarify a lot of my statements.

To add some more emphasis to my point, because I think it deserves more emphasis:

Quoting the interview Jacy linked to:

Your paper also says that “[w]ithout being overly alarmist, this should serve as a wake-up call for our colleagues” — what is it that you want your colleagues to wake up to? And what do you think that being overly alarmist would look like?

We just want more researchers to acknowledge and be aware of potential misuse. When you start working in the chemistry space, you do get informed about misuse of chemistry, and you’re sort of responsible f

... (read more)

Risk of misuse

The thought had never previously struck us.

This seems like one of the most tractable things to address to reduce AI risk.

If 5 years from now anyone developing AI or biotechnology is still not thinking (early and seriously) about ways their work could cause harm that other people have been talking about for years, I think we should consider ourselves to have failed.

To add some more emphasis to my point, because I think it deserves more emphasis:

Quoting the interview Jacy linked to:

Your paper also says that “[w]ithout being overly alarmist, this should serve as a wake-up call for our colleagues” — what is it that you want your colleagues to wake up to? And what do you think that being overly alarmist would look like?

We just want more researchers to acknowledge and be aware of potential misuse. When you start working in the chemistry space, you do get informed about misuse of chemistry, and you’re sort of responsible f

... (read more)

FWIW when I first saw the title (on the EA Forum) my reaction was to interpret it with an implicit "[I think that] We must be very clear: fraud in the service of effective altruism is unacceptable".

Things generally don't just become true because people assert them to be--surely people on LW know that. I think habryka's concern that not including "I think" in the title is a big deal is overblown. Dropping "I think" from the title is reasonable IMO to make the title more concise; I don't anticipate it degrading the culture of LW. I also don't see how it "byp... (read more)

Load More