This work has been done in the context of SaferAI’s work on risk assessment. Equal contribution by Eli and Joel. I'm sharing this writeup in the form of a Google Doc and reproducing the summary below. Disclaimer: this writeup is context for upcoming experiments, not complete work. As such it...
Summary From the piece: > Earlier this year I decided to take a few weeks to figure out what I think about the existential risk from Artificial Superintelligence (ASI xrisk). It turned out to be much more difficult than I thought. After several months of reading, thinking, and talking with...
TL;DR We want to hear ideas for physical engineering projects - that is, hardware prototypes moving around atoms (not just bits) - from the EA and/or longtermist community. Submit here! Introducing Reslab There has been a lot of talk about EA needing to take more large-scale action recently (e.g. Longtermist...
I think it doesn't -- our dev effects are so, so noisy!
We definitely do not claim that AI broadly slows down programmers! See tweet thread about this.
I think all the points you raised are an important part of the story -- we additionally go through some other factors that we think might explain the surprising result.
FWIW: this is my qualitative sense for other devs too.
We made this design decision because we wanted to max out on external validity, following the task length work which had fewer internal validity/more external validity concerns.
I (study author) responded to some of Ruby's points on twitter. Delighted for devs including Ruby to discuss their experience publicly, I think it's helpful for people to get a richer sense!
Found this helpful, thanks!
This work has been done in the context of SaferAI’s work on risk assessment. Equal contribution by Eli and Joel. I'm sharing this writeup in the form of a Google Doc and reproducing the summary below.
Disclaimer: this writeup is context for upcoming experiments, not complete work. As such it contains a lot of (not always well-justified) guess-work and untidy conceptual choices. We are publishing now despite this to get feedback.
If you are interested in this work — perhaps as a future collaborator or funder, or because this work could provide helpful input into e.g. risk assessments or RSPs — please get in touch with us at joel@qallys.com and/or simeon@safer-ai.org.
From the piece:
... (read 1693 more words →)Earlier this year I decided to take a few weeks to figure out what I think about the existential risk from Artificial Superintelligence (ASI xrisk). It turned out to be much more difficult than I thought. After several months of reading, thinking, and talking with people, what follows is a discussion of a few observations arising during this exploration, including:
- Three ASI xrisk persuasion paradoxes, which make it intrinsically difficult to present strong evidence either for or against ASI xrisk. The lack of such compelling evidence is part of the reason there is such strong disagreement about ASI xrisk, with people often (understandably) relying instead on prior beliefs, self-interest, and tribal
To "there were various suspicious/bad things Drew did," I would reply:
I have this opposing consideration. [...] I gather that part of the reason Alice and Chloe feel this way is that Drew did try to be helpful with respect to their concerns, at least to whatever degree was required for them to ask for him to be shielded from professional consequences.
and, to "the choice he's made to kinda hang around Emerson and Kat for this long," I would reply:
To the extent you believe that Nonlinear has been a disfunctional environment, in significant part due to domineering characteristics of senior staff, I think that you should also believe that a junior family member beginning to work in this environment is going to have a hard time reasoning through and pushing back against it.
Repost from EA forum:
Thank you very much for sharing, Chloe.
Ben, Kat, Emerson, and readers of the original post have all noticed that the nature of Ben's process leads to selection against positive observations about Nonlinear. I encourage readers to notice that the reverse might also be true. Examples of selection against negative information include:
I have this opposing consideration. I think it does speak to your point -- I gather that part of the reason Alice and Chloe feel this way is that Drew did try to be helpful with respect to their concerns, at least to whatever degree was required for them to ask for him to be shielded from professional consequences.
Here's another (in my view weaker, but perhaps more directly relevant to your point) consideration. To the extent you believe that Nonlinear has been a disfunctional environment, in significant part due to domineering characteristics of senior staff, I think that you should also believe that a junior family member beginning to work in this environment is going to have a hard time reasoning through and pushing back against it. Happy to expand.
We want to hear ideas for physical engineering projects - that is, hardware prototypes moving around atoms (not just bits) - from the EA and/or longtermist community. Submit here!
There has been a lot of talk about EA needing to take more large-scale action recently (e.g. Longtermist EA needs more Phase 2 work). And there likely exist cases where it would be helpful for action plans to start with small-scale hardware prototypes/MVPs. We came across such a case recently: as participants on SHELTER Weekend, we realised that the smallest scale MVPs for civilizational shelters often required technical implementation and tests.
We are starting Resilience Lab (or ‘Reslab’) to build capacity for prototyping hardware ideas... (read 180 more words →)
I think "many people" is doing a lot of work here -- I've generally found the public reception to be very nuanced, moreso than I was expecting. See e.g. Gary Marcus' post