Companies are building confidential computing architectures (“private AI clouds”) to run inference in a way that is private and inaccessible to the companies hosting the infrastructure. Apple, Google, and Meta all have versions of this in production today, and I think OpenAI and Anthropic are likely building this. Private AI clouds have real privacy benefits for users and security benefits for model weights, but they don’t provide true guarantees in the same way as end-to-end encryption. Users still need to place trust in the hardware manufacturer, third-party network operators, and abuse monitoring systems, among others.

A lawsuit between the New York Times and OpenAI has compelled OpenAI to produce 20 million user chats as part of the copyright claim. In OpenAI’s current architecture, they store user content, and it’s subject to subpoenas or emergency requests.

OpenAI wants to prevent similar overreaches in the future. In a blog post outlining their actions fighting the order, they wrote:

Our long-term roadmap includes advanced security features designed to keep your data private, including client-side encryption for your messages with ChatGPT. We believe these features will help keep your private conversations private and inaccessible to anyone else, even OpenAI.

OpenAI is describing a private AI cloud — where user data is processed inside a trusted execution environment (TEE) and is inaccessible to the provider, even if that provider has root access to the servers. This is a very different threat model than encryption at rest or in transit. A TEE aims to protect and encrypt data while it’s being processed. Users can send their data to the cloud without the cloud providers being able to see it.

OpenAI isn’t the first company to build a private cloud. Apple released Private Cloud Compute last June, where intensive Apple Intelligence tasks are offloaded to their data centers. Meta shipped Private Processing for WhatsApp earlier this year, which they describe as enabling “you to use Meta AI for processing messages off-device in a confidential and secure environment where no one, not even Meta or WhatsApp, can read or access your personal message”. Google announced Private AI Compute last month, which is built on TPUs.

I think it’s likely that Anthropic is also building its own version of this, and I predict that both OpenAI and Anthropic will launch private AI clo...

Posts

Wikitag Contributions

Comments