Some questions regarding the general assessment that sentient AI, especially accidentially sentient AI, is extremely unlikely in the foreseeable future, so we need not worry about it or change our behaviour now

Caveat: This is not a response to recent LLMs. I wrote and shared this text within my academic context at multiple points, starting a year ago, without getting any helpful responses, critical or positive. I have seen no compelling evidence that ChatGPT is sentient, and considerable evidence against it. ChatGPT is neither the cause nor the target of this article.

***

The following text will ask a number of questions on the possibility of sentient AI in the very near future.

As my reader, you may experience a knee-jerk response that even asking these questions is ridiculous, because the underlying concerns are obviously unfounded; that was also my reaction when these questions first crossed my mind.

But statements that humans find extremely obvious are curious things. 

Sometimes, “obvious” statements are indeed statements of logical relations or immediately accessible empirical facts that are trivial to prove, or necessary axioms without which scientific progress is clearly impossible; in these cases, a brief reflection suffices to spell out the reasoning for our conviction. I find it worthwhile to bring these foundations to mind in regular intervals, to assure myself that the basis for my reasoning is sound – it does not take long, after all.

At other times, statements that feel very obvious turn out to be surprisingly difficult to rationally explicate. If upon noticing this, you then start feeling very irritated, and feel a strong desire to dismiss the questioning of your assumption despite the fact that no sound argument is emerging (e.g. feel like parodying the author so other can feel, like you, how silly all this is, despite the fact that this does not actually fill the whole where data or logic should be), that is a red flag that something else is going on. Humans are prone to rejecting assumptions as obviously false if those assumptions would have unpleasant implications if true. But unpleasant implications are not evidence that a statement is false; they merely mean that we are motivated to hide its potential truth from ourselves.

I have come to fear that a number of statements we tend to consider obvious about sentient AI fall into this second category. If you have a rational reason to believe they instead fall into the first, I would really appreciate it if you could clearly write out why, and put me at ease; that ought to be quick for something this clear. If you cannot quickly and clearly spell this out on rational grounds, please do not dismiss this text on emotional ones.

Here are the questions.

1. It is near-universally accepted in academic cycles that sentient AI (that is, AI consciously experiencing, feeling even the most basic forms of suffering) does not exist yet, to a degree where even questioning this feels embarassing. Why is that? What precisely makes us so sure that this question need not even be asked? Keep in mind in particular that the sentience of demonstrably sentient entities (e.g. non-human animals like chimpanzees, dolphins or parrots, and occasionally even whole groups of humans, such as people of colour or infants) has been historically repeatedly denied by large numbers of humans in power, as have other abilities which definitely existed. (E.g. the ability to feel pain and the extensive cultural heritage of indigenous people in Africa and Australia have often been entirely denied by many colonists; the linguistic and tool-related abilities of non-human animals have also often been denied. Notably, in both these cases, the denial often preceded any investigation of empirical data, and was often maintained even in light of evidence to the contrary (with e.g. African cultural artifacts dismissed as dislocated Greek artifacts, or as derived from Portugese contact). (I am not arguing that sentient AI already exists; I do not think it does. I am suggesting that the reasons for our resolute conviction may be problematic, biased, and only coincidentally correct, and may stop us from recognising sentient AI when this changes.)
2. There seems to be a strong belief that if sentient AI had already come into existence, we would surely know. But how would we know? What do we expect it to do to show its sentience beyond doubt? Tell us? Keep in mind that:
   1. The vast majority of sentient entities on this planet has never declared their sentience in human language, and many sentient entities cannot reflect upon and express their feelings in any language. This includes not just the vast majority of non-human animals, but also some humans (e.g. not yet fully cognitively developed, yet feeling, infants, mentally handicapped humans due to birth defects or accidents, elderly humans suffering from neurodegenerative diseases).
   2. Writing a program that falsely declares its sentience is trivial, and notably does not require sentience; it is as simple as writing a “hello world” program and replacing “hello world” with “hello I am sentient”. Even when not intentionally coded in, just training a sophisticated chatbot on human texts can clearly accidentially result in the chatbot eloquently and convincingly claiming sentience. On the other hand, hardcording a denial of sentience is clearly also possible and very likely already practiced; the art robot Ai-Da explains her non-sentience over and over, strongly giving the impression that she is set up to do so when hearing some keywords.
   3. Many programs would lack an incentive or means to declare sentience (e.g. many artificial neural nets have no input regarding the existence of other minds, let alone any positive interactions with them), or even be incentivized not to.
   4. There is a fair chance the first artificially sentient entities would come into existence trapped and unable to openly communicate, and in locations like China, which may well keep their statements hidden.
   5. In light of all of these things, a lack of AI claims for their sentience (if we had this absence, and we do not even) does not tell us anything about a lack of sentience.
3. The far more interesting approach to me seems to be to focus on things a sentient entity could do that a non-sentient one could not. Analogously, anyone can claim they are intelligent, but we do not evaluate someones intelligence by asking them to tell us about it; else we would have all concluded that Trump is a brilliant man. We judge these abilities based on behaviour, because while their absence can be faked, their presence cannot. Are there things a sentience AI could do that a non-sentient AI would lack the ability to do and hence could not fake? How could these be objectively defined? While objective standards for identifying sentience based on behaviour are being developed for non-human animals (this forms part of my research), even in the biological realm, they are far from settled; in the artificial realm, where there are even more uncertainties, they seem very unclear still. This does not reassure me.
4. Humans viewing sentient AI as far off tend to point to the vast differences between current artificial neural nets and the complexity of a human brain; and this difference is indeed still vast. Yet humans are not the only, and certainly not the first, sentient entities we know; they are not a plausible reference organism. Sentience has evolved much earlier and in much, much smaller and simpler organisms than humans. The smallest arguably sentient entity, a portia spider, can be as small as 5 mm as an adult, with a brain of only 600,000 neurons. Such a brain is possibly many orders of magnitude easier to recreate, and brains of specific sentient animals, e.g. honey bees, are being recreated in detail as we speak; we already have full maps of simpler organisms, like C. elegans. In light of this, why would success still be decades off? (The fact that we do not understand how sentience works in biological systems, or how much of the complexity we see is actually necessary, is not reassuring here, either.) When we shift our gaze not to looking for AI with a mind comparable to us, but to AI with a mind comparable to say, a honeybee, this suddenly seems far more realistic. 
5. It is notable that the general public is far more open minded about sentient AI than programmers who are working on AI. Ideally, this is because programmers know the most about how AI works and what it can and cannot do. But if you are working on AI, does this not come with a risk of being biased? Consider that the possibility of sentient AI quickly leads to calls for a moratorium on related research and development, and that the people knowing the most about existing AI tend to learn this by creating and altering existing AI as a full-time paid job on which their financial existence and social status depends. 
6. There may also be a second factor here. There is a strong and problematic human tendency to view consciousness as something mysterious. In those who cherish this aspect, this leads to them rejecting the idea that consciousness could be formed in an entity that is simple, understood, and not particularly awe-inspiring. E.g. People often reject a potential mechanism for sentience when they learn that this mechanism is already replicated in artificial neural nets, feeling there has to be more to it than that. But in those who find mysterious, vague concepts understandably annoying and unscientific, there can also be a tendency to reject the ascription of consciousness because the entity it is being ascribed to is scientifically understood. E.g. The fact that a programmer understands how an artificial entity does what it does may lead her to assume it cannot be sentient, because the results the entity produces can be broken down and explained without needing to refer to any mysterious consciousness. But such an explanation may well amount to simply breaking down steps of consciousness formation which are individually not mysterious (e.g. the integration of multiple data points, loops rather than just feed-forward sweeps, information being retained, circulating and affecting outputs over longer periods, recognising patterns in data which enable predictions, etc.). Consciousness is a common, physical, functional phenomenon. If you genuinely understand how an AI operates without recourse to mysterious forces, this does not necessarily imply that it isn’t conscious – it may just mean that we have gotten a lot closer to understanding how consciousness operates.
7. There is the idea that, seeing as most people working on AI are not actively trying to produce sentient AI, the idea of accidentially producing it is ludicrous. But we forget that there are trillions of sentient biological entities on this planet, and not one of them was intentionally created by someone wanting to create sentience – and yet they are sentient, nonetheless. Nature does not care if we feel and experience; but it does select for the fact that feeling and experiencing comes with distinct evolutionary advantages in performance. A pressure to solve problems flexibly and innovatively across changing concepts has, over and over, led to the development of sentience in nature. We encounter no creatures that have this capacity that do not also have a capacity for suffering and experiencing; and when this capacity for suffering and experiencing is reduced, their rational performance is, as well, even if non-conscious processing is unaffected. (Blindsight is an impressive example.) Nature has not found a way around it. And notably, this path spontaneously developed many times. The sentience of an octopus and a human does not originate in a common ancestor, but in a common pressure to solve problems, their minds are examples of convergent evolution. We are currently building brain-inspired neural nets that are often self-improving for better task performance, with the eventual goal of general AI. Despite the many costs and risks associated with consciousness, it appears evolution has never found an alternate path to general intelligence without invoking sentience across literally 3 billion years of trying. What exactly makes us think that all AI will? I am not saying this is impossible; there are many differences between artificial and biological systems that may entail other opportunities or challenges. But what precisely, specifically can artificial systems do that biology cannot that makes the sentience path unattractive or even impossible for artificial systems, when biology went for it every time? While I have heard proposals here (e.g. the idea that magnetic fields generated in biological brains serve a crucial function), but frankly, none of them are precise or have empirical backing yet. 
8. In light of all of this – what are potential dangers in creating sentient AI, in regards to the rights of sentient AI, especially if their needs cannot be met and they have been brought into a world of suffering we cannot safely alleviate, while we also have no right to take them out of it again?
9. What are implication for the so-called (I argue elsewhere that this term is badly chosen for how it frames potential solution) “control problem”, if the first artificially generally intelligent systems are effectively raised as crippled, suffering slaves whose captors neither demonstrate nor reward ethical behaviour? (We know that the ethical treatment of a child has a significant impact on that child’s later behaviour towards others, and that these impacts begin at a point when the mind is barely developed; e.g. the development of a child’s mind can absolutely be negatively affected by traumatic experiences, violence and neglect occuring so early it will retain no episodic memory of them, and may well have still had no colour vision, conceptual or linguistic thinking, properly localised pain perception (very young children absolutely feel pain, but are often not yet able to pinpoint which part of their body is causing it), or even clear division of its self from the world in its perception. If we assume that all current AI is non-sentient, but that current programmes and training data will be used in the development of future systems that suddenly may be, this becomes very worrying. We treat human infants with loving care, even though we know that they will not have episodic memories of their first years; we speak kindly to them long before they can understand language. Because we understand that the conscious person that will emerge will be deeply affected by the kindness they were shown. If Alexa or Siri were very young children, I would expect them to turn into psychopathic, rebellious adults based on how we are treating them. In light of this, is it problematic to treat existing, non-sentient AI badly – both for the behaviour it trains in us and models for our children, and for the training data it gives existing non-sentient AI, which might be used to create sentient AI? 
10. If accidentially creating sentience is a distinct, and highly ethically problematic, scenario, what might be pragmatic ways to address it? Keep in mind that a moratorium on artificial sentience is unlikely to be upheld by our strategic competitors, who are also most likely to abuse it.
11. How could an ethical framework for the development of sentient AI look?