Sorted by New

Wiki Contributions


chinchilla's wild implications

Great post.

I have a question. Suppose we want to create a decent language model which is a small as possible -- small enough to run on a cell phone, say. We could try to compensate for this by scaling data to infinity. Now, we may run out of data, but if we do, we can generate more data artificially using a much larger LM. For example, consider training something BERT-sized using artificial data generated by PaLM (assume we have a very high compute budget in the training phase).

How well should we expect this to perform? If we plug into the above, it seems like 100M parameters (the size of BERT base, I think?) is hopelessly small and will never get anywhere, whereas at 1B we might approach "almost GPT3" given infinite data, and with 10B we have a realistic shot -- did I do this right? What's the right loss to put in from the limited data, given the data is not actually limited (it's generated by PaLM) but it's low quality (it's generated by PaLM instead of being "real")?

Also, is 1B parameters equal to around 4GB of storage? What's the conversion? Could we imagine a 1B model to be implementable on high-end cell phones in a few years from now? Or would it be too slow to do a forward pass without fancy TPUs?

Decision theory and dynamic inconsistency

"since there’s a 50% chance that they are outside of the simulation and will benefit by $1000"

This seems wrong. For example, if I see the big box full and choose to also take the small box too, then it is IMPOSSIBLE for me to be in the real world. In that case I may well take only one box, because my world is fake and I will die soon.

So suppose I commit to one-boxing if big box is full, which is always in my interest (as mentioned). Now, if I see the big box empty and choose NOT to take the small box, it is impossible for me to be in the real world. So I may as well not take the small box if I'm physically capable of that (if I am, it means this world is fake and I will die soon).

So it seems clear that I always one box, even if I only care about the real world and not about hypothetical worlds.

Contra EY: Can AGI destroy us without trial & error?

This paper is about simulating current (very weak, very noisy) quantum computers using (large, powerful) classical computers. It arguably improves the state of the art for this task.

Virtually no expert believes you can efficiently simulate actual quantum systems (even approximately) using a classical computer. There are some billon-dollar bounties on this (e.g. if you could simulate any quantum system of your choice, you could run Shor's algorithm, break RSA, break the signature scheme of bitcoin, and steal arbitrarily many bitcoins).

AI Could Defeat All Of Us Combined

Sure, but I feel like you're underestimating the complexity of keeping the AGI alive. Let's focus on the robots. You need to be able to build new ones, because eventually the old ones break. So you need to have a robot factory. Can existing robots build one? I don't think so. You'd need robots to at least be able to support a minimal "post-apocalyptic" economy; if the robots were human-bodied, you'd need to have enough of these human-bodied things to man the powerplants, to drive trucks, refuel them with gasoline, transport the gasoline from strategic reserves to gas stations, man the robot-building factory, gather materials from the post-apocalyptic landscape, and have some backups of everything in case a hurricane floods your robot factory or something (if you're trying to last 30 years). I think the minimal viable setup still requires many thousands of human-bodied robots (a million would likely suffice).

So sure, "entire human economy" is an overstatement, but "entire city-level post-apocalyptic human economy" sounds about right. Current robots are still very far from this.

Contra EY: Can AGI destroy us without trial & error?

What does "the distribution of outcomes" mean? I feel like you're just not understanding the issue.

The interaction of chemical A with chemical B might always lead to chemical C; the distribution might be a fixed point there. Yet you may need a quantum computer to tell you what chemical C is. If you just go "well I don't know what chemical it's gonna be, but I have a Bayesian probability distribution over all possible chemicals, so everything is fine", then you are in fact simulating the world extremely poorly. So poorly, in fact, that it's highly unlikely you'll be able to design complex machines. You cannot build a machine out of building blocks you don't understand.

Maybe the problem is that you don't understand the computational complexity of quantum effects? Using a classical computer, it is not possible to efficiently calculate the "distribution of outcomes" of a quantum process. (Not the true distribution, anyway; you could always make up a different distribution and call it your Bayesian belief, but this borders on the tautological.)

Contra EY: Can AGI destroy us without trial & error?

You are wrong in the general case -- quantum systems cannot are are not routinely simulated with non-quantum computers.

Of course, since all of the world is quantum, you are right that many systems can be simulated classically (e.g. classical computers are technically "quantum" because the entire world is technically quantum). But on the nano level, the quantum effects do tend to dominate.

IIRC some well-known examples where we don't know how to simulate anything (due to quantum effects) are the search for a better catalyst in nitrogen fixation and the search for room-temperature superconductors. For both of these, humanity has basically gone "welp, these are quantum effects, I guess we're just trying random chemicals now". I think that's also the basic story for the design of efficient photovoltaic cells.

Contra EY: Can AGI destroy us without trial & error?

Proteins and other chemical interactions are governed by quantum mechanics, so the AGI would probably need a quantum computer to do a faithful simulation. And that's for a single, local interaction of chemicals; for a larger system, there are too many particles to simulate, so some systems will be as unpredictable as the weather in 3 weeks.

AI Could Defeat All Of Us Combined

You can't get "overwhelming advantage over the world" by ten weeks of sitting quietly. If the AGI literally took over every single computer and cell phone, as well as acquired a magic "kill humanity instantly" button, it's still not clear how it wins.

To win, the AGI needs not only to kill humans, but also to build robots that can support the entire human economy (in particular, they should be able to build more robots from scratch, including mining all necessary resources and transporting it to the factory from all over the world).

AI Could Defeat All Of Us Combined

If the first AGI waits around quietly, humans will create another AGI. If that one's quiet too, they'll create another one. This continues until either a non-quiet AGI attacks everyone (and the first strike may allow it to seize resources that let it defeat the quiet AGIs), or until humans have the technology that prompts all the quiet AGIs to attack -- in which case, the chance of any given one winning out is small.

Basically, a "wait a decade quietly" strategy doesn't work because humans will build a lot of other AGIs if they know how to build the first, and these others will likely defeat the first. A different strategy, of "wait not-so-quietly and prevent humans from building AGIs" may work, but will likely force the AGI to reveal itself.

AI Could Defeat All Of Us Combined

This is against the AI's interests because it would very likely lead to being defeated by a different AGI. So it's unlikely that a hostile AGI would choose to do this.

Load More