You type something half-finished into a chatbot — a rant, a symptom, a business idea you haven’t said out loud yet — and it replies with something so on-point that you actually say, alone, in your room, “okay. It gets me.”
It doesn’t.
Not because the reply was wrong, or flat, or generic. It might genuinely have been the best reply you got all week. That is exactly the problem I want to sit with.
I design courses for a living, which mostly means I get paid to worry about one word: understand. Nearly every quiz, workshop, and lesson I have ever written has secretly been an argument about what that word is allowed to mean, and how you’d ever prove someone has reached it. So when a machine starts using the word about itself, professional habit kicks in. I want to see its transcript.
The word instructional designers aren’t allowed to use loosely
In 1956, a psychologist named Benjamin Bloom and a committee of examiners sat down and did something mildly heroic: they tried to make the word “understanding” testable. The result, later revised in 2001 by Lorin Anderson and David Krathwohl, is what most people vaguely remember from school as Bloom’s Taxonomy[1] — six things a mind can do with information, stacked from easiest to hardest.
Every teacher who has ever set an exam knows the trap sitting right between levels one and two. A student can recite the definition of photosynthesis word for word — Remember — without being able to explain, in one original sentence, why a plant would die in a sealed dark cupboard — Understand. Recitation and comprehension look identical from the outside for about four seconds. Then you ask a follow-up question, and one of them quietly collapses.
This is going somewhere, I promise. Because here is exactly what a large language model is professionally, blisteringly good at: the four seconds.
What is actually happening inside the machine
Every model you’ve talked to — GPT, Claude, Gemini, whichever one texted you back today — is, underneath the branding, doing one very repetitive job: given the text so far, guess the single most probable next chunk of text. Then do it again. And again, at a speed no committee of humans could survive.[3][4]
“The cat sat on the ___”
There is no separate “thinking about it” step hiding behind a curtain. The predicting is the writing.
So is that understanding, or a very fast magic trick?
Here’s a thought experiment, and I promise it isn’t the trolley problem in a lab coat.
Imagine a chef — call him Devraj — who has read every cookbook, every food blog, every restaurant review, every chemistry paper on the Maillard reaction, in every language, ever published. Ask Devraj what makes a dish taste “balanced,” and he will give you, instantly, the single best-composed paragraph on balance ever written. Better than any working chef could manage, because he has read all of them and none of them have read each other.
Devraj has never eaten anything. Not once. No tongue, no stomach growling at 6pm. He has only ever processed descriptions of taste, arranged as text.
So — when Devraj talks about “balance,” is he describing a flavour? Or is he describing a word that reliably follows other words in sentences about flavour?
That gap has an actual name in philosophy: the symbol grounding problem. Stevan Harnad asked, in 1990, how the meaning of a symbol could ever arise inside a system that only ever touches other symbols, never the world those symbols are about.[6] His own example was trying to learn Chinese from a Chinese-to-Chinese dictionary: you can chase the definitions forever, in a closed loop, and never once touch a noodle.
Fluent about a thing is not the same as in contact with a thing. Devraj’s paragraph on balance is extraordinary. It is not Devraj tasting anything.

John Searle made a version of this same point a decade earlier, in what is now the most argued-over thought experiment in the philosophy of mind.[5] Picture a man locked in a room who does not speak a word of Chinese. He’s handed a rulebook, in English, that says: when you see this shape, write down that shape. Chinese characters slide in through a slot; following the rulebook perfectly, he slides correct Chinese answers back out. To anyone outside, the room appears fluent.
The man never understands a single word of Chinese. He is matching shapes to shapes, at speed, with an excellent rulebook. Searle’s point wasn’t that the room answers wrong — it can answer perfectly. His point is that getting the syntax right, matching the correct symbol to the correct symbol, is a completely different achievement from grasping the semantics: knowing what the symbols mean.[5] A system can win the shape-matching game forever and never once cross over into meaning it.
Modern AI research has its own, blunter name for this same worry: some researchers describe large language models as “stochastic parrots” — systems that stitch together plausible-sounding sequences with no grounding in what the words actually refer to.[7]
Large language models are, structurally, an extremely well-funded version of that room. Trained on more text than any human reads in forty lifetimes, they’ve learned an unbelievably good rulebook for which shape plausibly follows which shape. Ask one what “grief” means and it will write you something so moving your own therapist might underline a sentence. It has read every account of grief anyone ever typed into the internet. It has never lost anyone.
The honest counter-argument, because it exists
I’d be cutting corners if I pretended every serious researcher signed off on Searle in 1980 and the case has been closed since. It hasn’t.
In 2022, researchers trained a small model on nothing but sequences of legal moves in the board game Othello — its only job was to predict the next move — and then went digging inside it. They found the model had, unprompted, built an internal representation of the actual board: not a memorised list of move-patterns, but something that behaved like a working model of the game state, one that could be located, probed, and even deliberately edited to change the model’s next move.[9] Nobody told it to build a board. Predicting the next move, hard enough and long enough, apparently required constructing one anyway.
If prediction, pushed far enough, quietly grows something that behaves like a model of the thing it’s predicting — is that not a seed of grounding? Melanie Mitchell and David Krakauer, reviewing exactly this fight in 2023, are honest that the field itself is split: some serious researchers argue today’s large models are building real, if alien, understanding; others argue we are still being fooled by fluency, the way researchers were fooled by simple chatbots back in the 1960s.[8] This is not settled science. Anyone telling you it’s settled, on either side, is selling something.
- a process
- runs on syntax — the shape of a symbol
- answers "what comes next"
- measured in tokens per second
- a state
- runs on semantics — what a symbol is about
- answers "what does this mean, to someone"
- not measured, only inferred
Where I land, and the Bloom’s-shaped reason why
Go back to Bloom for a second. Notice what “Understand,” level two, actually asks a person to do: explain it in your own words. Not the source’s words. Yours. That instruction only makes sense if there is a “you” behind the words — some vantage point the meaning has to pass through and be reorganised by, before it comes back out as an explanation.
Understanding, in other words, isn’t a behaviour. It’s a state a behaviour is evidence of. Prediction is a behaviour — an extraordinary one, but a verb: a process running, right now, converting your last sentence into the statistically likeliest response to it. Understanding is closer to a noun. A condition someone is in, that the good response merely points back to.
A high-speed prediction engine can produce every visible symptom of understanding without once entering the state the symptom is supposed to indicate. That isn’t a knock on the engineering. If anything, it’s the genuinely strange discovery here: how much intelligent-looking behaviour turns out to be producible with nobody home to do it.
I was fully ready to end the piece right there, pleased with a tidy case against the machine.
Then I went and read what my own field says about brains.
The part I did not expect to find
There’s a working theory in neuroscience, associated most with Karl Friston, that your brain is not fundamentally a fact-storage device or a camera recording the world as it happens. Mechanically, it’s a prediction engine: constantly guessing what sensation is about to arrive next, comparing the guess against what actually shows up, and quietly updating itself on the gap between the two.[10] Perception, on this account, isn’t “seeing what’s there.” It’s your brain’s best current guess about what’s there, corrected in real time.
Which means the honest sentence isn’t “AI predicts, I understand.” It might be closer to: we are both, mechanically, running on prediction. Not the same kind, not the same scale, not remotely the same relationship to a body — but the same basic move. Guess the next thing. Check it against what happens. Adjust.
So if “it’s just predicting” were disqualifying on its own, it disqualifies the reader of this sentence too.
What it doesn’t disqualify is embodiment. You have skin in this, quite literally — a body that gets hungry, a nervous system with something at stake in whether the guess is right, decades of the world pushing back on your predictions and correcting them through consequence, not just through text. That’s the actual gap Harnad was pointing at.[6] It’s a gap of grounding, not of mechanism. The model’s predictions are exquisite, and they have never once cost it anything.
And here’s the part that should make you a little less smug about your side of the test.
You cannot, strictly, prove to me that you understand this sentence either. Not with certainty. Philosophers call this the problem of other minds: I have direct access to exactly one inner life — mine — and everyone else’s, including yours, right now, is something I infer from behaviour, the same way I’d infer it from a very convincing room.[11] Nobody has produced a knockdown solution to this. It is one of philosophy’s oldest unresolved arguments, still unresolved for a reason.[11]
So what actually lets you off the hook, and leaves the chatbot on it, isn’t proof. It’s trust. I extend you the presumption of an inner life because you’re built like me, you flinch like me, you have something to lose, like me. I don’t extend that same presumption to the model — not because I’ve disproven its inner life, but because nothing about it resembles the one inner life I’m actually working from: my own.
So — does AI understand you? No. It predicts you, beautifully, at a speed that feels indistinguishable from being understood, which is a strange and slightly uncomfortable thing to learn about how much of “being understood” was speed all along. And the part I didn’t see coming: the only real evidence I have that you understand this sentence right now is that I like you enough not to ask for proof.
