I think it would be good for the labs to invest in building air gapped datacenters for training^[1] and evaluating models. This is good for cybersecurity as recent events have shown, and it also makes it harder for the models to self-exfiltrate or for an adversary to steal the weights.

1. ^{^}

   Though phases prior to RL rollouts (e.g. pretraining, midtraining) can be done normally.