If you haven't read the letters from The Future of Life Institute and Eliezer Yudkowsky, do that first:

https://futureoflife.org/open-letter/pause-giant-ai-experiments/

https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/

Is AI going to kill us all?

Quite possibly. We shouldn't panic, but even a 1% risk of ending the world deserves a massive intervention. The risk is much higher than 1%, but experts disagree, and there's no way to know who is right. More on this later.

Should we, therefore, pause large-model AI research?

Yes, if it were possible.

Is it possible?

Absolutely not. The cat is out of the bag, anyone who understands game theory should grasp that.

Should we try to pause research anyway?

A poorly executed pause may do more harm than good. If the "good guys" stop AI research, we risk letting "bad guys" have superpowers, or just very dangerous toys.

So are we doomed?

Probably not. There are huge unknowns, and the apocalypse is far from certain.

Why would a research pause not work?

1: Large models are overhyped, yet they have proven extremely valuable, and their value is likely to increase in improved versions. Imagine a machine that prints gold. If most people refrain from improving their machines, those who improve their printers in secret obtain even more value. Any person or nation which refuses to go along with the ban obtains tremendous value. Even a global police state cannot stop people from running code on every single computer. It's preferable for leading public entities to dominate research over million of unknowns

2: While GPT-4 requires massive resources, existing open-source models can be run on a consumer GPU. Improvements in AI models and hardware will continue to narrow this gap. A ban on large model AI research is a massive stimulus to optimizing models to run with fewer resources.

Are there any reasons to be optimistic?

Yes. I believe our options are either quick, global death or a (literally) unimaginable explosion in human flourishing. There is no in-between. Skynet won't be hunting us down, so at least the end, if it comes, will be quick.

While no one is suggesting that AI is likely to be malevolent, there are reasons to think that AI trained on comprehensive large language models may be benevolent. While humans have no idea how to teach AI ethics or values, the models work by rewarding AI for being "helpful." The nature of these m...

Posts

Wikitag Contributions

Comments