The Fatal AGI Hardware Gap

jrincayc

Restricting computer hardware has been proposed as a solution to preventing the dangers of building superintelligent AI. For example, MIRI suggests restricting hardware fabricated on 28 nanometer process node processes or smaller or hardware with 15,840 TFLOP/s or greater computing capacity ^[1]

There is a catch with restricting hardware however, that I have not seen discussed elsewhere. Essentially, there is a minimum level of hardware required to create an AGI which we can call . If the hardware is below this level (which will actually be a frontier depending on multiple things like memory, compute speed and available I/O), the hardware will be incapable of AGI, if it is below the limit. If it is above, the hardware is capable of AGI.

Now, let us suppose that the AGI is ethical, in that it can correctly choose actions that do not harm other beings in the universe. We can list another limit, for the minimum level of hardware required to create an ethical AGI.

Now compare them. Imagine we have a minimum ethical AGI. We can make the AGI smaller and simpler by deleting things like how to figure out which atoms are currently being used by other beings, and instead just grab whatever atoms we can. So it seems quite likely that . ^[2] This means that there is a gap between the two, and if a hardware limit is set in this gap, the probability that an AGI is unethical is 100% ^[3] , because ethical AGIs are not possible below .

In short, if we try and prevent AGI by using hardware limits, we need to make sure we are not in the gap between and because that would actually move into a region where any AGI is always fatal. ^[4] Above $L_{EAGI} the probability of an AGI being unethical is less, but I am not sure what direction the probability goes with increasing computational power (increases, decreases, or remains constant seem possible to me).

https://ifanyonebuildsit.com/treaty ↩︎
seems impossible, since an ethical AGI is an AGI. might be possible if all (or at least the lowest resource using) AGI's figure out ethics more or less automatically. might also be possible if ethics are easy to add an existing AGI. ↩︎
At least if the hardware limit is completely enforced. ↩︎
Note that the MIRI limits seem more likely to be set to prevent superintelligence, rather than AGI. Eliezer Yudkowsky, Steve Byrnes, and myself have all estimated that current high end personal computers (or lower) probably can do AGI: https://intelligence.org/2022/03/01/ngo-and-yudkowsky-on-scientific-reasoning-and-pivotal-acts/ , https://www.lesswrong.com/posts/LY7rovMiJ4FhHxmH5/thoughts-on-hardware-compute-requirements-for-agi , https://www.researchgate.net/publication/388398902_Memory_and_FLOPS_Hardware_limits_to_Prevent_AGI ↩︎

^{^}

Or impractically large, like simulating literal human brains. But MIRI didn't mention such a pathway, opting for calls to make the humans smarter.

^{^}

Or impractically large, like simulating literal human brains. But MIRI didn't mention such a pathway, opting for calls to make the humans smarter.

[-]Davidmanheim6h30

I don't understand the argument. If the limit is above that needed for ASI, we're dead. The fact that it could also be below or above the level of aligned ASI doesn't change anything.

[-]StanislavKrym6h20

The main problem is that rapidly DECREASES over time. As far as I understand MIRI's position, MIRI believesthat current tech has an infinite^[1] because nobody can (yet?) make the AGI ethical. Suppose, alternatively, that current tech is such that is a thousand times more than , while the developments of mechinterp allow us to have reach just twice . Then a blanket ban wouldn't be THAT useless.

^{^}
Or impractically large, like simulating literal human brains. But MIRI didn't mention such a pathway, opting for calls to make the humans smarter.