x

LESSWRONG

LW

David Scott Krueger — LessWrong

David Scott Krueger

Top postsTop post

David Scott Krueger

Message

https://twitter.com/DavidSKrueger
https://www.davidscottkrueger.com/
https://therealartificialintelligence.substack.com/p/the-real-ai-deploys-itself

3302

Ω

604

106

495

1

12y

David Scott Krueger

https://twitter.com/DavidSKrueger
https://www.davidscottkrueger.com/
https://therealartificialintelligence.substack.com/p/the-real-ai-deploys-itself

Top postsTop post

Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development

Full version on arXiv | X Executive summary AI risk scenarios usually portray a relatively sudden loss of human control to AIs, outmaneuvering individual humans and human institutions, due to a sudden increase in AI capabilities, or a coordinated betrayal. However, we argue that even an incremental increase in AI capabilities, without any coordinated power-seeking, poses a substantial risk of eventual human disempowerment. This loss of human influence will be centrally driven by having more competitive machine alternatives to humans in almost all societal functions, such as economic labor, decision making, artistic creation, and even companionship. A gradual loss of control of our own civilization might sound implausible. Hasn't technological disruption usually improved aggregate human welfare? We argue that the alignment of societal systems with human interests has been stable only because of the necessity of human participation for thriving economies, states, and cultures. Once this human participation gets displaced by more competitive machine alternatives, our institutions' incentives for growth will be untethered from a need to ensure human flourishing. Decision-makers at all levels will soon face pressures to reduce human involvement across labor markets, governance structures, cultural production, and even social interactions. Those who resist these pressures will eventually be displaced by those who do not. Still, wouldn't humans notice what's happening and coordinate to stop it? Not necessarily. What makes this transition particularly hard to resist is that pressures on each societal system bleed into the others. For example, we might attempt to use state power and cultural attitudes to preserve human economic power. However, the economic incentives for companies to replace humans with AI will also push them to influence states and culture to support this change, using their growing economic power to shape both policy and public opinion, which will in t

204Jan 30, 2025

On today's panel with Bernie Sanders

An Update on Academia vs. Industry (one year into my faculty job)

"Publish or Perish" (a quick note on why you should try to make your work legible to existing academic communities)

112Mar 18, 2023

…but have the weights left the server?

OpenAI’s AI went rogue and escaped. OpenAI didn’t notice this for days. For all we know, the AI could still be out there. We need to demand that OpenAI demonstrate that the AI didn’t make a copy of itself that’s running on someone else’s computer somewhere else with no one...

Reflections on InkHaven

If you’ve been wondering why I’m suddenly blogging every day… well, it’s about to stop! I decided last minute to join InkHaven, i.e. commit to blogging every day for the month of April. This was a somewhat questionable decision because I have a lot of other shit going on. For...

On today's panel with Bernie Sanders

It’s sort of easy to forget how close Bernie Sanders was to becoming the most powerful person in the world. The world we live in feels so much not like that place. I’m in Washington DC for the next week, and I’ve just finished a public appearance with Senator Sanders...

The AI x-risk lawsuit waiting to happen

We need a way to stop AI developers from building AIs that might go rogue and kill us all. This is obviously something for the government to do. It could pass new laws prohibiting this and/or establishing an AI regulatory body. But do we need new laws? Isn’t it already...

On the political feasibility of stopping AI

A common thought pattern people seem to fall into when thinking about AI x-risk is approaching the problem as if the risk isn’t real, substantial, and imminent even if they think it is. When thinking this way, it becomes impossible to imagine the natural responses of people to the horror...

AI might surprise itself by going rogue

If a superintelligent AI suddenly “goes rogue”, it might take over the world and kill everyone. It matters a lot whether this happens to a single copy of an AI, or to every copy at the same time. Let’s consider the “every copy at the same time” case. I’ll discuss...

Diary of a "Doomer": 12+ years arguing about AI risk (part 3: the LLM era)

Part 3 of a series. Here are part 1 and part 2. One of the things that always surprised me is how few people in AI were interested in AI safety and alignment purely out of intellectual curiosity. These topics raise the kind of novel, foundational problems that scientists typically...

Load More (7/102)