Crossposted from my Substack.

I spent the weekend at Lighthaven, attending the Eleos conference. In this post, I share thoughts and updates as I reflect on talks, papers, and discussions, and put out some of my takes since I haven't written about this topic before.

I divide my thoughts into three categories: (1) philosophy, (2) legal/social, (3) technical, even though there are unavoidable overlaps. I share relevant paper titles of interest that were either mentioned in presentations or recommended to me during conversations.

(1) Philosophy

The philosophy world is hedging about AI consciousness. What I mean is that even in cases where it's clearly useful to apply the intentional stance, philosophy people have a tendency to avoid defining LLM mentality in intentional terms and resist making any bold claims that could be taken as saying that LLMs are conscious. I am not particularly surprised by this, as I have also offered a pragmatic framework for applying the intentional stance to LLMs and have avoided arguing for anything beyond that. David Chalmers presents a similar picture in a recent paper. Relatedly, it's really worth asking what it is that we apply the intentional stance to. Is it the base model, one of the simulacra, or the thread, as per Chalmers?

Moving to the study of consciousness, my controversial take is that it might be perfectly fine to be reductionist in examining relevant capabilities and never actually define the C-word. If the science of consciousness is still pre-paradigmatic, the way we use this term might be similar to how phlogiston was used in alchemy before it became chemistry. I know this might sound like a direct attack against the field altogether, but on the contrary, it is aimed at preparing the ground for it to enter its paradigmatic era.

Now, on the functionalist debate: I think that we have no good reasons to assume that there's anything special about biology. I am indeed very skeptical about the superiority of biology in that I view it as a stipulated normative claim, rather than primarily grounded in available empirical results. All the evidence we have so far suggests that we don't need to replicate the messiness of biological systems to get capable models.

We may just want to think in terms of normative claims, however, if we're focused on moral status. Thus, we may need some kind of reflective equilibrium to...

AI Consciousness