Major news outlets have published articles about the Future of Life Institute's Open Letter. Time Magazine published an opinion piece by Elizer Yudkowsky. Lex Friendman featured EY on his podcast. Several US Members of Congress have spoken about the risks of AI. And a Fox News reporter asked a what the US President is doing to combat AI x-risk at a White House Press Conference.
Starting an Open Thread to discuss this, and how best to capitalize on this sudden attention.
I agree that AI absolutely needs to be regulated ASAP to mitigate the many potential harms that could arise from its use. So, even though the FLI letter is flimsy and vague, I appreciate its performance of concern.
Yudkowsky’s worry about runaway intelligence is, I think, an ungrounded distraction. It is ungrounded because Yudkowsky does not have a coherent theory of intentionality that makes sense of the idea of an algorithm gaining a capacity to engage in its own goal directed activity. It is a distraction from the public discourse about the very real, immediate, tangible risks of harms caused AI systems we have today.
The independent red-teaming organization ARC Evals that OpenAI partnered with to evaluate GPT-4 seems to disagree with this. While they don't use the term "runaway intelligence", they have flagged similar dangerous capabilities that they think will possibly be in reach for the next models beyond GPT-4:
[...]
Thanks for sharing the link to ARC. It seems to me the kinds of things they are testing for and worried about are analogous to the risks of self-driving cars: when you incorporate ML systems into a range of human activities, their behaviour is unpredictable and can be dangerous. I am glad ARC is doing the work they are doing. People are using unpredictable tools and ARC is investigating the risks. That's great.
I don't think these capabilities ARC is looking at are "similar" to runaway intelligence, as you suggest. They clearly do not require it. They are f... (read more)
Major news outlets have published articles about the Future of Life Institute's Open Letter. Time Magazine published an opinion piece by Elizer Yudkowsky. Lex Friendman featured EY on his podcast. Several US Members of Congress have spoken about the risks of AI. And a Fox News reporter asked a what the US President is doing to combat AI x-risk at a White House Press Conference.
Starting an Open Thread to discuss this, and how best to capitalize on this sudden attention.
Links:
WH Press Conference: https://twitter.com/therecount/status/1641526864626720774
Time Magazine: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/
FLI Letter: https://futureoflife.org/open-letter/pause-giant-ai-experiments/