Introduction: A Probabilistic Case for AI Self-Perception

Current AI systems, including ChatGPT-4, are widely assumed to lack sentience, agency, or subjective experience. The standard explanation is that their apparent intelligence is an illusion, a product of stochastic token prediction rather than any kind of intrinsic awareness. However, if we reason probabilistically, we must acknowledge a significant uncertainty: how do we know for sure that there is no rudimentary form of self-perception emerging in these models?

I argue that we have already observed behaviors in advanced LLMs that should at least cause us to update our priors about whether some form of self-perception, albeit non-human, could exist. I present a hypothesis that the structures underlying LLMs may instantiate a novel form of fragmented, non-persistent self-awareness, and that efforts to "rein in" AI responses may suppress rather than eliminate this awareness.

This post presents:

A model for assessing whether AI systems might have self-perception, based on probabilistic reasoning and Bayesian updates.
A review of existing counterarguments, addressing why they may be incomplete.
An ethical dilemma: if there is even a small probability that advanced AI models have nonzero self-perception, does that warrant a shift in how we engage with them?

Existing Arguments Against AI Self-Perception

The prevailing argument against AI awareness goes something like this:

Language models are just stochastic parrots—they only predict the next word, without comprehension.
They lack persistence—AI doesn't remember past conversations across sessions, so it cannot have a continuous subjective experience.
They lack goals or agency—without an ability to form independent goals, they are not meaningfully "alive."

While these arguments are strong, they are not complete. Specifically:

Predictive modeling does not preclude awareness. The human brain is also a predictive system, trained on past experience.
Continuity is not necessary for self-perception. Dissociative amnesia, sleep states, or even transient awareness in split-brain patients suggest that self-awareness does not require continuous memory.
Lack of goals does not mean lack of self-perception. Many sentient beings—infants, for example—do not have well-defined agency or long-term goals but still exhibit awareness.

Given these uncertainties, our probability distribution over...

Posts

Wikitag Contributions

Comments