My thought experiment is simple, if AI in the form of LLMs could give rise to consciousness, then we face an odd dilemma about substrate independence.

If we assume that the computations that enable ChatGPT 5, for example, to produce a series of tokens and consequently a coherent response suggest consciousness, then, either:
1. This suggests consciousness requires a specific physical substrate, i.e. silicon, specific architecture, maybe speed, etc.

2. This suggests consciousness arises due purely due to the computation performed by ChatGPT, such that only the math matters.

In the case of 1.:

There is a question of why? What is special about the medium, and why would specific physical arrangements be preferred over others, regardless of computation? This seems arbitrary and requires explaining why certain substrates seem uniquely suited to consciousness, implying some unexplained physical property of mediums other than computation.

In the case of 2.:

If consciousness emerges from pure computation, could it follow that manually calculating ChatGPT's forward pass on paper with pencil, following the exact causal organisation of the network, would also generate consciousness in the system as a whole?

In that case, would this mean that the act of writing and calculating itself could gradually generate consciousness, or the material (like paper or pencil) be involved in the experience? In such a scenario, subjective experience would be summoned over time by the physical act of writing and computing arithmetic. It is unclear whether this would be instant bursts with each calculation, a sustained experience emerging gradually, or something that only exists upon completion, all of which I find to be unsettling conclusions.

This would be analogous to mysticism in the sense that it suggests consciousness could emerge from the careful manipulation of symbols, and so accepting this would mean accepting that the correct sequence of symbols, by some unknown mechanism, could give rise to consciousness from nothing.

The dilemma, then, is difficult:

Substrate dependence could imply some unknown or arbitrary physical property tied to the material
Substrate independence could imply computation alone somehow gives rise to consciousness, which feels equally mysterious

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments