I’m considering whether to work on accelerating automated scientific labs.
My main concern: if the time until the next future AI escape is roughly fixed (say like, hypothetically, 6 months), then faster progress on automated labs means there will be more scientific infrastructure available for that AI to exploit when it escapes, thereby increasing the chance of catastrophic risk.
Is that reasoning sound? Many people are still willing to accelerate automated labs, so I’m wondering if I’m missing something.
A few responses I’ve encountered, but am not convinced by:
Safe development is feasible. Automated labs could be designed so that even a rogue AI has essentially no chance of taking control. I’m skeptical of this after the Hugging Face incident. Unless access ultimately depends on something like biometric authorization, it seems hard to guarantee that sufficiently capable AI couldn’t bypass the safeguards.
Benefits outweigh the added risk. Faster progress in medicine, materials, etc. may justify the increase in catastrophic risk. I’m fairly pessimistic about this tradeoff, so this personally doesn’t persuade me.
“Warning-shot” argument. Perhaps meaningful regulation will only follow a serious AI incident. Accelerating automated labs could make an earlier escape severe enough to trigger action while overall capabilities are still lower. In contrast, slowing lab automation might make the first incident too mild to prompt regulation, delaying action until a later, more dangerous second escape. This seems quite tenuous.
TL;DR: Are there compelling reasons to reject the argument that we should avoid accelerating automated labs because doing so gives a future escaped AI more infrastructure to exploit, thereby increasing catastrophic risk? If there aren’t any, though, happy to not work on progressing automated labs.
I’m considering whether to work on accelerating automated scientific labs.
My main concern: if the time until the next future AI escape is roughly fixed (say like, hypothetically, 6 months), then faster progress on automated labs means there will be more scientific infrastructure available for that AI to exploit when it escapes, thereby increasing the chance of catastrophic risk.
Is that reasoning sound? Many people are still willing to accelerate automated labs, so I’m wondering if I’m missing something.
A few responses I’ve encountered, but am not convinced by:
TL;DR: Are there compelling reasons to reject the argument that we should avoid accelerating automated labs because doing so gives a future escaped AI more infrastructure to exploit, thereby increasing catastrophic risk? If there aren’t any, though, happy to not work on progressing automated labs.