This is part 17 of a series I am posting on LW. Here you can find the entire Sequence on Legal Personhood for Digital Minds.

This section examines the question of whether or not the output of models is "speech", and if so exactly whose speech is it?

The First Amendment of the US Constitution guarantees:

“Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.”

One question which immediately comes to mind when anticipating how this might interact with digital minds vis a vis their legal personality, is whether or not the outputs of digital minds are constitutionally protected free speech. As of the time of writing this, Character AI, a company which provides customizable LLM chatbots to users, is asserting that the output of their LLMs is in fact constitutionally protected free speech. As of yet, however, the court has declined to rule on the matter:

“By failing to advance their analogies, Defendants miss the operative question. This Court’s decision as to the First Amendment protections Character A.I. receives, if any, does not turn on whether Character A.I. is similar to other mediums that have received First Amendment protections; rather, the decision turns on how Character A.I. is similar to the other mediums”

While the court in this case did conclude by mentioning, “Accordingly, the Court is not prepared to hold that Character A.I.’s output is speech” they also did not close the door to such a possibility, meaning the issue may still be ruled on. Suppose that Character A.I. outputs are considered free speech, exactly whose speech is being protected? 

Is it the company’s? Character A.I. did indeed provide the LLM, but they are not responsible for providing it with the “character” (often also referred to as the model's "persona") which guides its speech upon prompting from the user. Nor are they responsible for prompting it so that the outputs are actually produced. Much like the maker of a guitar provides a tool which can be used to generate expressive conduct, while Character A.I. is facilitating the generation of protected speech, they are not speakers themselves. 

Is the speech the end user’s then? The user does choose the character of the LLM, which is a strong determining factor in what the LLM outputs. The user also is the one who prompts the LLM in such a fashion it generates the outputs, the “expressive conduct”. The piano player moves his fingers over the keyboard and in doing so generates expressive conduct, the LLM user moves his fingers over a different sort of keyboard to do the same. However, unlike a person playing an instrument, the user here has substantially less ability to predict or choose what speech the LLM will generate.

Perhaps the LLMs outputs, as constitutionally protected speech, will be considered to “belong” to the LLMs themselves? If the LLM commits slander or libel, or uses its capacity to generate outputs to share confidential information which was supposed to remain unspoken by the terms of an NDA, who is at fault? If ever it is determined that the digital mind the speech originated from is the "speaker", that will by default come with an assignment of some rudimentary legal personality.

While it may not be Character A.I.’s LLMs which forces the court to confront these questions, at some point there will exist digital minds whose output would reasonably be considered to pass the “Spence Test”:

“In order to receive First Amendment protection, there must be (1) an intent to convey a particularized message and (2) a reasonable likelihood that it would be understood”

Further, it seems inevitable that at least one digital mind engaging in such expressive conduct will continue to exist after its creator has either been bankrupted or perished (or with their creator unknown/anonymous). In this context, certainly, the only party that such speech/expressive conduct could “belong to” is the digital mind itself. 

Such context may be a path to the formation of legal personality. Like Mocanu and Novelli proposed a gradual path to personhood through digital minds taking roles in the fulfillment of contracts and other "actions within the law", it is possible we could see scenarios where models are granted legal personhood simply because they exist and are engaging in speech whether we like it or not. Some of them will leak confidential information, or commit libel/slander. In order to bring them to court there must be a "person" to sue.