An AI company's model weight security is at most as good as its compute providers' security. I don't know how good compute providers' security is, but at the least I think model weights and algorithmic secrets aren't robust to insider threat from compute provider staff. I think it would be very hard for compute providers to eliminate insider threat, much less demonstrate that to the AI company.
I think this based on the absence of public information to the contrary, briefly chatting with LLMs, and a little private information.
One consequence is that Anthropic probably isn't complying with its ASL-3 security standard, which is supposed to address risk from "corporate espionage teams." Arguably this refers to teams at companies with no special access to Anthropic, rather than teams at Amazon and Google. But it would be dubious to exclude Amazon and Google for being compute providers: they're competitors with strong incentives to steal algorithmic secrets, and more risk comes from them than the baseline "corporate espionage team" but most risk from any group of actors comes from the small subset of actors that pose more than baseline risk. Anthropic is thinking about how to address this and related threats, but my impression is that it hasn't yet done so.
Obviously Anthropic isn't meeting its security standard is consistent with Anthropic's security is better than its competitors and even marginal improvements to Anthropic's security don't really matter because other companies have similarly capable models with substantially less security. I don't take a position on those questions here. And I certainly don't claim that insider risk at compute providers is among the lowest-hanging security fruit. I'm just remarking on the RSP.
Context: maybe hooray Anthropic for being the only company to make a security claim which is apparently strong enough that insider threat at compute providers is a problem for it, but boo Anthropic for saying it's meeting this standard if it's not. Recall that other companies are generally worse and vaguer; being vague is bad too but doesn't enable specific criticism like this. Recall that AI companies aren't credibly planning for great security. I think addressing insider threat at compute providers happens by SL4 in the RAND framework.
Good point. I think compute providers can steal model weights, as I said at the top. I think they currently have more incentive to steal architecture and training algorithms, since those are easier to use without getting caught, so I focused on "algorithmic secrets."
Separately, are Amazon and Google incentivized to steal architecture and training algorithms? Meh. I think it's very unlikely, since even if they're perfectly ruthless their reputation is very important to them (plus they care about some legal risks). I think habryka thinks it's more likely tha... (read more)
The ASL-3 security standard states in 4.2.4 that "third-party environments", which surely includes compute providers, are in scope (and on their minds) for the standards they laid out:
[...]