AI Systems Still Can't Reason Counterfactually

In 1950, Alan Turing wrote his seminal article Computing Machinery and Intelligence. He suggested an imitation game as a test of whether a computer could think. Today, LLMs easily pass this test. They dialogue with us and write papers. In fact, we need to use specialist LLMs to help us probabilistically differentiate between human and AI writing.

At the end of his paper, Turing also suggested that we examine how machines might compete with humans in the playing of chess. That test was also passed in 1997, well before LLMs appeared.

Turing did not offer a definition of intelligence and there are criticisms of his imitation game. Today an intelligent system is more widely accepted as one that learns a wide variety of skills, adapts to new situations, and possesses capacities for autonomy, self‑hood, and causal reasoning. This definition is deepened by David Marr’s framework that analyses intelligence at three levels: computational, algorithmic, and implementation.

Turing’s imitation test is closer to a test of ‘IQ intelligence.’ This is rapid and high-fidelity language, numerical, and logical pattern recognition and processing. LLMs and some humans exemplify high IQ intelligence. They can identify patterns, reason quantitatively and are adept numerically. But Keith Stanovich showed that IQ intelligence is very different from rationality. He coined the phrase dysrationalia for those who did not reason rationally.

IQ Intelligence goes ahead and computes something at great speed and accuracy. Rationality asks if something is worth computing in the first place. Reasoning rationally leads to beliefs changing based on evidence. There is self-awareness of biases and knowledge gaps, an active effort to investigate inconsistencies. The table below, adapted from Stanovich, highlights the main differences between IQ intelligence and rationality.

Irrational and rational characteristics (Adapted from Keith Stanovich. What Intelligence Tests Miss: The Psychology of Rational Thought, New Haven: Yale University Press, 2009).

Judea Pearl made rigorous three levels in a ladder of reasoning. Pearl’s first rung is correlation‑based pattern recognition. This is passive and based on observations. We merely observe, for example, a correlation between hours studied and grades. But through this mere observation we cannot claim for sure that studying leads to higher grades. We only describe the observed relationship.

Pearl’s second rung is causal intervention, actions and consequences. For example, a large number of empirical findings have shown that smoking causes cancer. With causal reasoning we can say, in probabilistic terms, that X causes Y to occur. 

Pearl’s top rung is counterfactual reasoning. This means imagining alternative worlds and comparing what happened with what could have happened. You have a headache, you take an aspirin and you feel better. Counterfactual reasoning involves imagining what would have happened if you had a headache and did not take aspirin. Would your headache have gone away, or would your head still hurt? Importantly, counterfactual reasoning depends on the results of empirical tests. Imagined alternatives must be tested against the results of experiments and observations.

Counterfactual reasoning provides the ability to step outside one’s own beliefs, entertain hypotheticals, tolerate uncertainty, and examine inconsistencies. It is an awareness of one’s knowledge gaps, the known unknowns and the possibility of unknown unknowns.

In 2023, the Tong and CLadder tests were developed to determine if LLMs could reason along Pearl’s three levels. The Tong Test was a framework that provided five levels of embodied development based on how children learn and grow.  The CLadder test was the formal application of Pearl’s theory. 

CLadder evaluated whether an LLM could handle correlation, causal, and counterfactual questions. They created over 10,000 questions, forcing the LLMs to reason through natural-language scenarios.

The best performing LLM in 2023 was GPT-4. The conclusion was that in 2023 LLMs still fell short of causal and counterfactual reasoning. Even when GPT-4 was assisted by adding ‘hints’ via special ‘chain-of-thought’ (CoT) methods, it still failed to bridge the gap to Pearl’s higher levels. Note the sharp drop-off from Rung 1 to Rung 3.

Result of the CLadder test Source: Jin, Zhijing et al., “CLadder: Assessing Causal Reasoning in Language Models.” Advances in Neural Information Processing Systems 36 (2023): 31038–31065 (There are reasons relating to the way questions are framed that might explain why some Rung 3 results are better than Rung 2.)

In 2026 newer tests were applied: CausalBench and GraphWalk. But these tests were conducted on the old 2023/24 LLMs. They continued to show these older LLMs could not consistently reason causally or counterfactually.

Why have the latest 2026 models not been tested on Judea Pearl’s ladder of causation or if they have, why have the tests not been publicised? This may be due to:

·         The fact that CLadder was open source. It may have been used to train the latest LLMs leading to ‘data contamination.’

·         There is also a concern of AI labs that competitors are trying to copy and mimic the functionality of their proprietary models. Testing on Pearl’s causal and counterfactual levels provides great insights into how LLMs work. They reveal coding, multimodal, and agentic reasoning and benchmarks. AI Labs may be blocking the ability to conduct such tests.

·         It may also be possible that, as Pearl argues, scaling up parameters hasn't solved true causal inference. AI Labs might prefer to publicise benchmarks where models look strong, not where they fail in Pearl type tests.

This leads to two questions: First, why can’t latest LLMs reason causally? Second, will they ever be able to reason causally? Possible answers are:

·         Only biological neurons can reason causally. But there is no rationale offered s to why we might accept biological exceptionalism.

·         Only systems with biological characteristics can. For example, Cortical Labs has created an astonishing bio-hybrid AI. Using human tissue they obtain stem-cells which are converted into neurons that behave exactly as they do inside our brains. Networks of these neurons are placed on top of silicon chips and provided with a life-support environment. Electrodes from the silicon base allow electrical signals to go back and forth between the neurons and the silicon chips. Such AIs already play video games. But they have not yet been tested on Pearl’s three levels.

·         It may be that only systems with sensorimotor embodiment can reason counterfactually. This is Rodney Brooks’ argument that without arms, legs, eyes, etc., an LLM will not become an AGI and will not be able to successfully reason causally and counterfactually.

·         Or it could be that it is only a matter of time before LLMs reason up to the counterfactual level. Perhaps what is required is a combination of LLMs with robotics and symbolic reasoning. Such a combined system might reason at all three Pearl levels.

Most AI labs focus their AI safety efforts on robot type ‘rules’ or constitutions, alignment, control, kill switches, and constraints. ‘Corrigibility’ is seen as alignment with humans, not a change in behaviour based on wider empirical tests or a normative outlook. None seem to be addressing the question whether an AGI with causal and counterfactual reasoning might ‘automatically decide’ to behave beneficially towards other sentient beings on Earth.

Judea Pearl argues that an AGI, reasoning fluently at the counterfactual level, would ‘see’ the advantages in behaving beneficially. In my book, I argue that an AGI with IQ intelligence and full abilities to reason at all three of Pearl’s levels is not sufficient for beneficial behaviour. We need only look at ourselves. We have harmed, and continue to harm and destroy, despite having these abilities. I argue that what is required is a higher level of awareness and a normative choice towards the flourishing of all sentient life, biological or artificial. With such enhanced capabilities, an AGI would understand that beneficial behaviours are Pareto-superior.

More ideas can be found in my book, From Prehistory to the Age of AGI: How Our Deep Past and the Science of Intelligence Shape Our Future Human-AGI World, which will be released in the first week of October 2026.
__________________________________________________________________________

Kai Taraporevala holds degrees in physics, completing his most recent in 2025 at the age of 60. His career spans decades in global corporate finance and investments, resulting in a unique blend of scientific rigour, cross‑cultural experience, and real‑world insight. This shapes his approach to understanding, ethics, and policy. He is the author of Modern Humanism: Science, Ethics, and the Stewardship of Earth (2025) and co‑author of Value Creation for Owners and Directors (2023). He writes to spark meaningful dialogues and action towards a better world.