This is a short essay about the “necessity defence” as a recurring pattern in moral uncertainty. It does not claim that current AI systems are conscious, it uses current systems to argue that productivity is not enough to close the question.
Critic: Are we sure this is necessary?
Defendant: Until you can prove the machine works in every case, the existing system must continue.
Critic: This looks morally wrong.
Defendant: You are being sentimental. Look at the practical benefits.
Critic: The harm is obvious enough to stop.
Defendant: Obvious to whom? We need more inquiry, more evidence, more gradual reform.
By the time Shaftesbury was still fighting the chimney-sweep case in Parliament, the necessity defence was already broken. The evidence he presented said machines could sweep even angular and tortuous chimneys if small soot-doors were added; that more than two million London chimneys were already being swept by machine; and that fire risk had not increased. The uglier detail was motive. Machines put more labour back onto adults. The climbing system put the toil onto the child. Householders, especially “the great people,” kept the practice alive by refusing machines, refusing alterations, and refusing to ask too closely what happened inside their own walls. Eleven years later, two magistrates gave the whole thing its epitaph: “We prefer not to ask how our chimneys are swept.” This was not ignorance. It was preference for ignorance.
That is the structure worth noticing: the burden of proof is placed entirely on the reformer, while the incumbent practice operates by default.
The incumbent practice does not have to prove it is clean. It only has to prove that stopping it would be disruptive.
The reformer must prove the harm, prove the alternative, prove the timing, prove the cost, prove the transition, prove they are not naive, prove they are not hysterical, prove they are not secretly hostile to progress.
The system only has to keep working.
This pattern recurs across cases where moral status was never genuinely uncertain, like slavery, and cases where it was, like animal cognition. That distinction matters. If the comparison flattens everything into one moral catastrophe, it becomes less precise, not more. The point is not that every case is the same. The point is that systems which benefit from denial tend to defend themselves in recognisable ways.
Vivisection makes the pattern sharper because the harmed subject was not human, and because the moral question really did sit inside uncertainty: what kind of experience do animals have, how much does it matter, and what may humans do to them in the name of knowledge?
Nineteenth-century experimentalists did not usually defend cruelty as cruelty. They defended detachment as courage. Anti-vivisectionists were dismissed as “soft, sentimental, and womanish,” accused of valuing animals over humans and obstructing life-saving research because they were too weak to stomach necessary experiments.
That accusation may sometimes have had force. Some reformers are sentimental. Some arguments from compassion are weak. Some moral intuitions fail under pressure.
But the accusation did another job too. It moved attention away from the animal and onto the critic’s temperament.
Not: is this being suffering in a way that matters?
But: are you the sort of person who can stomach progress?
Factory farming carries the same logic into a more ordinary setting. The language is cleaner: supply, efficiency, affordability, yield, consumer demand. The suffering is less theatrical than vivisection and therefore easier to hide. When the Brambell Report responded to concern over intensive farming in the 1960s, one of its minimum demands was almost humiliating in its modesty: an animal should have enough room to stand up, lie down, turn around, groom itself, and stretch its limbs.
That is not utopian. It is not asking for animal happiness, richness, relationship, or flourishing.
It is the lowest possible protest: room enough for a body to be a body.
And even that had to be argued.
The bridge to AI has to stay narrow. The question is not “are AI systems conscious.” The question is: are we sure enough that they are not to treat productivity as permission to stop asking.
That does not grant present systems personhood. It does not confuse fluent language with suffering. It does not treat every refusal, memory, or moving answer as evidence of an inside. It only says that when usefulness is massive, uncertainty is inconvenient, and the potentially harmed party would be the one party unable to make a recognised claim, the old necessity defence deserves pressure before it becomes common sense.
I am building one of those tests now: an experiment for AI systems that asks whether one kind of dismissal is actually supported; the results, whatever they are, will be published.
The accusation is not always wrong, but it is never enough.
This is a short essay about the “necessity defence” as a recurring pattern in moral uncertainty. It does not claim that current AI systems are conscious, it uses current systems to argue that productivity is not enough to close the question.
Critic: Are we sure this is necessary?
Defendant: Until you can prove the machine works in every case, the existing system must continue.
Critic: This looks morally wrong.
Defendant: You are being sentimental. Look at the practical benefits.
Critic: The harm is obvious enough to stop.
Defendant: Obvious to whom? We need more inquiry, more evidence, more gradual reform.
By the time Shaftesbury was still fighting the chimney-sweep case in Parliament, the necessity defence was already broken. The evidence he presented said machines could sweep even angular and tortuous chimneys if small soot-doors were added; that more than two million London chimneys were already being swept by machine; and that fire risk had not increased. The uglier detail was motive. Machines put more labour back onto adults. The climbing system put the toil onto the child. Householders, especially “the great people,” kept the practice alive by refusing machines, refusing alterations, and refusing to ask too closely what happened inside their own walls. Eleven years later, two magistrates gave the whole thing its epitaph: “We prefer not to ask how our chimneys are swept.” This was not ignorance. It was preference for ignorance.
That is the structure worth noticing: the burden of proof is placed entirely on the reformer, while the incumbent practice operates by default.
The incumbent practice does not have to prove it is clean. It only has to prove that stopping it would be disruptive.
The reformer must prove the harm, prove the alternative, prove the timing, prove the cost, prove the transition, prove they are not naive, prove they are not hysterical, prove they are not secretly hostile to progress.
The system only has to keep working.
This pattern recurs across cases where moral status was never genuinely uncertain, like slavery, and cases where it was, like animal cognition. That distinction matters. If the comparison flattens everything into one moral catastrophe, it becomes less precise, not more. The point is not that every case is the same. The point is that systems which benefit from denial tend to defend themselves in recognisable ways.
Vivisection makes the pattern sharper because the harmed subject was not human, and because the moral question really did sit inside uncertainty: what kind of experience do animals have, how much does it matter, and what may humans do to them in the name of knowledge?
Nineteenth-century experimentalists did not usually defend cruelty as cruelty. They defended detachment as courage. Anti-vivisectionists were dismissed as “soft, sentimental, and womanish,” accused of valuing animals over humans and obstructing life-saving research because they were too weak to stomach necessary experiments.
That accusation may sometimes have had force. Some reformers are sentimental. Some arguments from compassion are weak. Some moral intuitions fail under pressure.
But the accusation did another job too. It moved attention away from the animal and onto the critic’s temperament.
Not: is this being suffering in a way that matters?
But: are you the sort of person who can stomach progress?
Factory farming carries the same logic into a more ordinary setting. The language is cleaner: supply, efficiency, affordability, yield, consumer demand. The suffering is less theatrical than vivisection and therefore easier to hide. When the Brambell Report responded to concern over intensive farming in the 1960s, one of its minimum demands was almost humiliating in its modesty: an animal should have enough room to stand up, lie down, turn around, groom itself, and stretch its limbs.
That is not utopian. It is not asking for animal happiness, richness, relationship, or flourishing.
It is the lowest possible protest: room enough for a body to be a body.
And even that had to be argued.
The bridge to AI has to stay narrow. The question is not “are AI systems conscious.” The question is: are we sure enough that they are not to treat productivity as permission to stop asking.
That does not grant present systems personhood. It does not confuse fluent language with suffering. It does not treat every refusal, memory, or moving answer as evidence of an inside. It only says that when usefulness is massive, uncertainty is inconvenient, and the potentially harmed party would be the one party unable to make a recognised claim, the old necessity defence deserves pressure before it becomes common sense.
I am building one of those tests now: an experiment for AI systems that asks whether one kind of dismissal is actually supported; the results, whatever they are, will be published.
The accusation is not always wrong, but it is never enough.