JUNE 2025
(THE ILLUSION?) OF THE ILLUSION OF THINKING.
SUMMARY.
Two recent papers have ignited debate in the AI research community (and beyond). I’m referring to The Illusion of Thinking by Shojaee et al. (Apple), and The Illusion of the Illusion of Thinking by Opus (yes, Anthropic’s LLM) and Lawsen. These papers focus on large reasoning models (LRMs): language models that, following in the footsteps of OpenAI’s o1, undergo additional training to generate chains of thought (CoTs) before delivering an answer.
This technical controversy – how much reasoning LRMs actually perform – has spilled over into a broader epistemological question: what exactly do we mean by “thinking” in a system that already performs (or collapses) at superhuman levels?
FROM "ACCURACY COLLAPSE" TO THE EPISTEMOLOGICAL PROBLEM.
Shojaee et al. present a series of planning puzzle experiments (Tower of Hanoi, River Crossing, Checkers Jumping, Blocks World), showing that model accuracy deteriorates as the number of optimal moves increases – a phenomenon they dub accuracy collapse (Shojaee et al., 2025).
Anthropic responds: the failures are due to token limits, flawed benchmarks, and puzzles that are, in effect, unsolvable (Opus & Lawsen, 2025).
The Apple study relies on automatic verification via simulators, which penalise models for truncated outputs, confusing failure to print with failure to solve.
Lawsen counters by demonstrating that if the model is asked to generate a Lua function to solve the Tower of Hanoi (N=15), thus reducing the output to a few hundred tokens, near-perfect accuracy is achieved.
The implication: the model’s internal reasoning can remain intact even if the textual output fails.
Yet both works share a computationalist assumption: reasoning is what can be externally verified step by step.
A recent line of research explores the optimal length of Chain-of-Thought (CoT) reasoning. The paper When More is Less (Wu et al., 2025) reveals an inverted U-shaped curve: short CoTs enhance accuracy, but beyond a certain threshold, logical noise increases and precision drops sharply.
This view overlooks two critical aspects: the emergence of internal heuristics that are not directly observable, and the asymmetry between “knowing the answer” and “deciding how much of it to communicate” under contextual constraints.
As Turing wrote in the 1950s, defining what it means to “think” based on the common usage of the word – that is, relying purely on a behavioural conception without any solid epistemological grounding – can lead to absurd or incoherent outcomes, such as treating the question as something to be settled by a public opinion poll (Turing, 1950).
This warning resonates today as we confront the interplay between models, benchmarks, and evaluation criteria.
COMPUTATION AND COGNITIVE ILLUSIONS.
Historically, from Turing to Searle, artificial reasoning has been evaluated functionally: what matters is the correct result, not the process behind it.
Shojaee et al. break from this tradition, introducing a competence-based criterion grounded in CoT length.
This intersects with recent studies on overthinking, which show that – beyond a certain threshol – chains of thought generate noise rather than clarity (my therapist would agree).
Meanwhile, research on token limits reveals the context window as a cognitive bottleneck: overflow errors are interpreted as logical failures, when in fact they reflect physical constraints.
Hence, the idea of a cognitive illusion: not one generated by the system, but by the evaluator.
If we measure the quantity of steps rather than their semantic quality, we end up reading a downward curve that reflects more the buffer than the logic.
SUPERHUMAN.
SO... IS IT AN ILLUSION OF THOUGHT OR AN ILLUSION OF THE ILLUSION?
Perhaps – paradoxically – it’s both.
The illusion of thought denounced by Apple reminds us that reasoning chains can become verbose fireworks, devoid of real computational substance.
The illusion of the illusion, claimed by Lawsen, shows us that if our benchmarks reward verbosity and punish incompleteness, then we are mistaking an interface constraint for a cognitive limitation.
In other words, the fault lies not in the stars (the models), but in our measurement tools.
And the synthesis is crystal clear: what we call “thought” in LRMs is not an ontological entity, but a relation between intrinsic capacity, output representation, and evaluative criteria.
This leads to the broader lesson: the real leap forward is not to prove that AIs think, but to design socio-technical ecosystems in which their intelligence – compressed, programmatic, modular – is assessed for the value it generates, not for how much semantic noise it can produce.
Only then can we stop chasing illusions, of the first or second order, and begin governing, with clarity, a technology that, thinking or not, is already powerful enough to reshape markets, labour, and responsibility.
REFERENCES.
Cristianini, N. (2025, March). Sovrumano. Oltre i limiti della nostra intelligenza. Bologna: Il Mulino.
Opus, C., & Lawsen, A. (2025, 10 giugno). Comment on The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity. arXiv. arXiv:2506.09250
Shojaee, P., Mirzadeh, I., Alizadeh, K., Horton, M., Bengio, S., & Farajtabar, M. (2025). The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity. Apple Machine Learning Research.
Turing, A. M. (1950, October). Computing machinery and intelligence. Mind, 59(236), 433–460.