Yes, sorry, when I say "we'll have to see if such things emerge," I mean "it's not obvious to me that this is a big threat. I don't know if there are likely to be non-superintelligent rogue systems attempting to deploy bioweapons. It's not obvious to me that the reasons the rogue AIs commit cyberattacks would cause them to engage in bioattacks, or that there would be other reasons that misaligned AIs would use bioweapons not as part of a takeover scenario. AI-generated plagues may be a capability the AIs would have and not use before superintelligence." I agree that there's enough uncertainty, and the results would be bad enough, that it is worth investing in. Our policy shouldn't be wait and see.

If the humans get relatively corrigible AIs to design supervirsuses and then fail to safely create and store those bioweapons, causing them to leak and spread unrelated to AI malfeasance, that is also a big potential problem. I don't know enough about factors like how much restraint major powers show with regards to bioweapons research already, how much restraint they would show if additional research showed novel useful capabilities not previously possible (eg. a virus that spreads harmlessly through the population but is lethal in one individual), and to what extent intelligence lowers the barrier to entry (can you set up a bioweapons program in your garage if you're smart enough, or do you need particular exotic ingredients?). But again the risk is sufficiently bad that we should take it seriously and do what we can to decrease it.

I'm not sure the exact intervention you're proposing is. Who are the few hundred researchers who can easily be stopped? Are those the AI researchers creating AIs that can do virology research, or are they the bioweapons researchers who would use AIs to create novel viruses? Both these approaches seem like they have their difficulties, and I don't think saying "what you're doing is dangerous, please stop" is likely to be effective on either group (these seem to be professions that attract people who are either very confident in their abilities or figure someone else will do it anyway so they might as well get paid). It would be great to do something about these risks; I'm interested in hearing more about what could be done and how it'd be effective. Work through the scenarios.