I think we should not be hesitant to talk about this in public. I used to be of the opposite opinion, believing-as-if there was a benevolent conspiracy that figured out which conversations could/couldn’t nudge AI politics in useful ways, whose upsides were more important than the upsides of LWers/etc. knowing what’s up. I now both believe less in such a conspiracy, and believe more that we need public fora in which to reason because we do not have functional private fora with memory (in the way that a LW comment thread has memory) that span across organizations.
It’s possible I’m still missing something, but if so it would be nice to have it spelled out publicly what exactly I am missing.
I agree with Lincoln Quirk’s comment that things could turn into a kind of culture war, and that this would be harmful. It seems to me it’s worth responding to this by trying unusually hard (on this or other easily politicizable topics) to avoid treating arguments like soldiers. But it doesn’t seem worthwhile to me to refrain from honest attempts to think in public.
How open do we think OpenAI would be to additional research showing the dangers of AGI? If OpenAI is pursuing a perilous course, perhaps this community should prioritize doing the kind of research that would persuade them to slow down. Sam Altman came across to me at the two SSC talks he gave as being highly rational as this community would define rational.
If this is the correct path, we would benefit from people who have worked at OpenAI explaining what kind of evidence would be needed to influence them towards Eliezer's view of AGI.
I just saw this, but this feels like a better-late-than-never situation. I think hard conversations about the possibilities of increasing existential risk should happen.
I work at OpenAI. I have worked at OpenAI for over five years.
I think we definitely should be willing and able to have these sorts of conversations in public, mostly for the reasons other people have listed. I think AnnaSalamon is the answer I agree with most.
I want to also add this has made me deeply uncomfortable and anxious countless times over the past few years. It can be a difficult thing to navigate well or navigate efficiently. I feel like I've gotten better at it, and better at knowing/managing myself. I see newer colleagues also suffering from this. I try to help them when I can.
I'm not sure this is the right answer for all context, but I am optimistic for this one. I've found the rationality community and the lesswrong community to be much better than average at dealing with bad faith arguments, and for cutting towards the truth. I think there are communities where it would go poorly enough that it could be net-negative to have the conversation.
Side note: I really don't have a lot of context about the Elon Musk connection, and the guy has not really been involved for years. I think the "what things (including things OpenAI is doing) might increase existential risk" is an important conversation to have when analyzing forecasts, predictions, and research plans. I am less optimistic about "what tech executives think about other tech executives" going well.
I'm skeptical of OpenAI's net impact on the spirit of cooperation because I'm skeptical about the counterfactual prospects of cooperation in the last 6 years had OpenAI not been founded.
The 2000s and early 2010s centralized and intermediated a lot of stuff online, where we trusted centralized parties to be neutral arbiters. We are now experiencing the after effects of that naivete, where Reddit, Twitter and Facebook are censoring certain parties on social media, and otherwise neutral infrastructure like AWS or Cloudflare kick off disfavored parties. I am at the point where I am scared of centralized infrastructure and try to minimize my reliance on it because it allows third parties to apply pressure on me.
These general trends are much larger than Elon Musk and would have happened without him. I'm uncertain to what extent Elon was just reacting to this trend and was ahead of the curve.
Darkly humorous, but OpenAI's recent destruction of otherwise entirely benign users of GPT-3 is teaching people to not rely on centralized AI. Now that OpenAI is large, the Blue Egregore can apply media pressure on the tech company to extract submission, in this case censorship demands. The downstream effects of this demand to think of the non-existent children was the destruction of trust in AI Dungeon leading people to mostly switch to various GPT-J alternatives (including local setups) and the shutdown of the Samantha, which Mr. Rohrer is taking hard.
Releasing GPT-3 non-trivially increased the odds of doomsday. So yeah they are not good actors.
Eliezer Yudkowsky writes on twitter:
I've found myself repeatedly uncertain about what to make of OpenAI and their impact. The most recent LessWrong discussion that I'm aware of has happened on the post Will OpenAI-s work unintentionally increase existential risk, but the arguments for negative impact are different from the thing Eliezer named.
I'm also not entirely sure whether publicly debating sensitive questions like whether a person or organization accidentally increased existential risk is a good idea in the first place. However, spontaneously bringing up the issue on a twitter thread is unlikely to be optimal. At the very least, it should be beneficial to discuss the meta question, i.e., how we should or shouldn't talk about this. With that in mind, here are three things I would like to understand better:
And -- conditional on discussing them being a good idea:
What is the more detailed story of how the "nascent spirit of cooperation" has degraded or changed since the inception of OpenAI?
What interventions are possible here, if any? (Is it really that difficult to organize some kind of outreach to Elon to try and reverse some of the effects? Naively speaking, my impression has been that our community is sufficiently well connected to do this, and that Elon is amenable to arguments.)
I'm less interested in estimating the total impact of any specific person.