Anthropic has an unconventional governance mechanism: an independent "Long-Term Benefit Trust" elects some of its board. Anthropic sometimes emphasizes that the Trust is an experiment, but mostly points to it to argue that Anthropic will be able to promote safety and benefit-sharing over profit.[1]
But the Trust's details have not been published and some information Anthropic has shared is concerning. In particular, Anthropic's stockholders can apparently overrule, modify, or abrogate the Trust, and the details are unclear.
Anthropic has not publicly demonstrated that the Trust would be able to actually do anything that stockholders don't like.
The facts
There are three sources of public information on the Trust:
They say there's a new class of stock, held by the Trust/Trustees. This stock allows the Trust to elect some board members and will allow them to elect a majority of the board by 2027.
But:
Morley et al.: "the Trust Agreement also authorizes the Trust to be enforced by the company and by groups of the company’s stockholders who have held a sufficient percentage of the company’s equity for a sufficient period of time," rather than the Trustees.
I don't know what this means.
Morley et al.: the Trust and its powers can be amended "by a supermajority of stockholders. . . . [This] operates as a kind of failsafe against the actions of the Voting Trustees and safeguards the interests of stockholders." Anthropic: "the Trust and its powers [can be changed] without the consent of the Trustees if sufficiently large supermajorities of the stockholders agree."
It's impossible to assess this "failsafe" without knowing the thresholds for these "supermajorities." Also, a small number of investors—currently, perhaps Amazon and Google—may control a large fraction of shares. It may be easy for profit-motivated investors to reach a supermajority.
Maybe there are other issues with the Trust Agreement — we can't see it and so can't know.
Vox: the Trust "will elect a fifth member of the board this fall," viz. Fall 2023.
Anthropic has not said whether that happened nor who is on the board these days (nor who is on the Trust these days).
Conclusion
Public information is consistent with the Trust being quite subordinate to stockholders, likely to lose their powers if they do anything stockholders dislike. (Even if stockholders' formal powers over the Trust are never used, that threat could prevent the Trust from acting contrary to the stockholders' interests.)
Anthropic knows this and has decided not to share the information that the public needs to evaluate the Trust. This suggests that Anthropic benefits from ambiguity because the details would be seen as bad. I basically fail to imagine a scenario where publishing the Trust Agreement is very costly to Anthropic—especially just sharing certain details (like sharing percentages rather than saying "a supermajority")—except that the details are weak and would make Anthropic look bad.[2]
Maybe it would suffice to let an auditor see the Trust Agreement and publish their impression of it. But I don't see why Anthropic won't publish it.
Maybe the Trust gives Anthropic strong independent accountability — or rather, maybe it will by default after (unspecified) time- and funding-based milestones. But only if Anthropic's board and stockholders have substantially less power over it than they might—or if they will exercise great restraint in using their power—and the Trust knows this.
Unless I'm missing something, Anthropic should publish the Trust Agreement (and other documents if relevant) and say whether and when the Trust has elected board members. Especially vital is (1) publishing information about how the Trust or its powers can change, (2) committing to publicly announce changes, and (3) clarifying what's going on with the Trust now.
Note: I don't claim that maximizing the Trust's power is correct. Maybe one or more other groups should have power over the Trust, whether to intervene if the Trust collapses or does something illegitimate or just to appease investors. I just object to the secrecy.
Thanks to Buck Shlegeris for suggestions. He doesn't necessarily endorse this post.
The LTBT, whose members have no equity in the company, currently elects one out of the board’s five members. But that number will rise to two out of five this July, and then to three out of five this November.
This is encouraging and makes me not care anymore about seeing the "milestones." (And it explains why OP/investors/whoever didn't bother to replace Luke.) My concerns about investors' power over the Trust remain.
Also:
The LTBT’s first five members were picked by Anthropic’s executives for their expertise in three fields that the company’s co-founders felt were important to its mission: AI safety, national security, and social enterprise. Among those selected were Jason Matheny, CEO of the RAND corporation, Kanika Bahl, CEO of development nonprofit Evidence Action, and AI safety researcher Paul Christiano. [The other two were Neil Buddy Shah of the Clinton Health Access Initiative and formerly GiveWell and Zach Robinson of CEA and EV] (Christiano resigned from the LTBT prior to taking a new role in April leading the U.S. government’s new AI Safety Institute, he said in an email. His seat has yet to be filled.)
From this we can infer that the other four Trustees remain on the Trust, which is weak good news. [Edit: nope, Matheny left months ago due to potential (appearance of) conflict of interest. It's odd that this article doesn't mention that. As of May 31, Christiano and Matheny have not yet been replaced. It is maybe quite concerning if they—the two AI safety experts—are gone and not replaced by AI safety experts, and the Trust is putting non-AI-safety people on the board. Also I'm disappointed that Anthropic didn't cause me to know this before.]
Also:
Amazon and Google, he says, do not own voting shares in Anthropic, meaning they cannot elect board members and their votes would not be counted in any supermajority required to rewrite the rules governing the LTBT. (Holders of Anthropic’s Series B stock, much of which was initially bought by the defunct cryptocurrency exchange FTX, also do not have voting rights, Israel says.)
Google and Amazon each own less than 15% of Anthropic, according to a person familiar with the matter.
So then the question is: who does own voting shares; how are voting shares distributed? and how can this change in the future?
(Also we just got twomore examples of Anthropic taking credit for the Trust, and one of the articles even incorrectly says "power ultimately lies with a small, unaccountable group.")
Crossposted from AI Lab Watch. Subscribe on Substack.
Introduction
Anthropic has an unconventional governance mechanism: an independent "Long-Term Benefit Trust" elects some of its board. Anthropic sometimes emphasizes that the Trust is an experiment, but mostly points to it to argue that Anthropic will be able to promote safety and benefit-sharing over profit.[1]
But the Trust's details have not been published and some information Anthropic has shared is concerning. In particular, Anthropic's stockholders can apparently overrule, modify, or abrogate the Trust, and the details are unclear.
Anthropic has not publicly demonstrated that the Trust would be able to actually do anything that stockholders don't like.
The facts
There are three sources of public information on the Trust:
They say there's a new class of stock, held by the Trust/Trustees. This stock allows the Trust to elect some board members and will allow them to elect a majority of the board by 2027.
But:
Conclusion
Public information is consistent with the Trust being quite subordinate to stockholders, likely to lose their powers if they do anything stockholders dislike. (Even if stockholders' formal powers over the Trust are never used, that threat could prevent the Trust from acting contrary to the stockholders' interests.)
Anthropic knows this and has decided not to share the information that the public needs to evaluate the Trust. This suggests that Anthropic benefits from ambiguity because the details would be seen as bad. I basically fail to imagine a scenario where publishing the Trust Agreement is very costly to Anthropic—especially just sharing certain details (like sharing percentages rather than saying "a supermajority")—except that the details are weak and would make Anthropic look bad.[2]
Maybe it would suffice to let an auditor see the Trust Agreement and publish their impression of it. But I don't see why Anthropic won't publish it.
Maybe the Trust gives Anthropic strong independent accountability — or rather, maybe it will by default after (unspecified) time- and funding-based milestones. But only if Anthropic's board and stockholders have substantially less power over it than they might—or if they will exercise great restraint in using their power—and the Trust knows this.
Unless I'm missing something, Anthropic should publish the Trust Agreement (and other documents if relevant) and say whether and when the Trust has elected board members. Especially vital is (1) publishing information about how the Trust or its powers can change, (2) committing to publicly announce changes, and (3) clarifying what's going on with the Trust now.
Note: I don't claim that maximizing the Trust's power is correct. Maybe one or more other groups should have power over the Trust, whether to intervene if the Trust collapses or does something illegitimate or just to appease investors. I just object to the secrecy.
Thanks to Buck Shlegeris for suggestions. He doesn't necessarily endorse this post.
E.g. 1, 2, and 3, and almost every time Anthropic people talk about Anthropic's governance.
Unlike with some other policies, the text of the Trust Agreement is crucial; it is a legal document that dictates actors' powers over each other.