Is it intelligent? Does it understand what it says? Is there someone in there? Before answering, it is worth looking at an experiment that can be run at home, in a few minutes, with a half-billion-parameter language model. Ask it to solve a first-degree equation: it solves it, in orderly, correct steps. Ask it how many times the letter r appears in "strawberry", and it answers two; there are three. Ask it to spell the word out, letter by letter, as one would aloud, and the spelling it produces is not the word.

Faced with a system that answers like this, the three questions arrive all together and look like one. They are decades old; what made them everyone's questions was language. Since systems have existed that handle it at a level that until recently seemed out of reach, the general perception of these topics has changed before the arguments did. And they are instead three different questions, with three different standards of proof; we use them as synonyms because in the only example we know from up close, ourselves, the three things come together, in a single package, and the criteria by which we recognize them are calibrated on that package. There is also a new reason to look inside these systems: they are trained on the description humans have given of the world, a thoroughly anthropocentric starting point, and inside them one begins to find categories that are not ours.

These pages try to keep the three questions separate. They answer neither "yes" nor "no" to "do they understand?": they answer with each question's standard of proof, saying what can be measured, what is beginning to be visible, and what remains out of reach. In the middle there is a test anyone can replicate on an ordinary computer, because at least one of the three questions can be interrogated at home. Benchmark surveys, future scenarios and judgements about products stay out: here the reasoning is about the concepts, with the tools that exist today.

1. Three questions disguised as one

That there can be capability without understanding seems impossible, and it is the norm. A chess engine has beaten the world champion since 1997; nobody credits it with an understanding of chess in the sense in which one credits a master who explains why a move is bad, and it wins the game all the same. A calculator adds better than anyone and does not know what a number is. The idea that capabilities come as a block, that whoever solves also understands and whoever understands also feels, is not an observed fact: it is a projection of our own case, the only one in which the three things have always lived together.

The projection works in both directions, and this is why the public debate never leaves its stalemate. Whoever watches a model answer a difficult question with fluency concludes that it understands; whoever watches it get a letter count wrong concludes that it is a statistical trick and understands nothing. "It seems to understand" and "it seems not to understand" are both surface effects, and they mislead in the same way: they project onto the machine our own scale, the one on which whoever solves an equation can also count. The two opposite conclusions spring from the same mistake.

It is worth fixing the three standards straight away, because they are the thread of everything that follows. Intelligence is answered with tests: tasks, outcomes, comparisons; it is the oldest and most solid standard, and it has the limits of every measurement made from outside. Understanding is answered by looking inside: the question is whether the system, in order to answer, has built itself a representation of what it talks about, and for a few years now tools have existed that go looking for it. Consciousness, today, is not answered: one can say precisely why no test exists, and that is already content, but it is content of a different kind from a measurement. Three questions, three standards: this piece is their separation.

2. The thread of language

The three historical stages of this discussion have one thing in common, and it is no accident: they all pass through language. The test Turing proposed in 1950 is a written conversation. The room Searle imagined in 1980 manipulates Chinese ideograms. And the question about machine consciousness, dormant for decades, woke up exactly when machines began to answer: the essay by Chalmers that reopens it in earnest is from 2023, not from 1997.

Yet in 1997 a machine had beaten the world chess champion, and in 2016 another had beaten the champion of Go, a game thought to be out of reach. They were superhuman systems in their own domain; nobody asked whether they were conscious, and the word "intelligence" stayed in the headlines without ever really attaching itself to the machines. The first system that speaks is the first that seemed to us a mind. The most defensible reading is that language is the channel through which we recognize intelligence, not the condition for having it: non-linguistic intelligence exists, in animals as in game engines, but it does not resemble us enough to trigger the questions.

Why language has this power can be seen by asking what language is, for us. The human representation of reality is the one evolution has granted: the senses with which we build it are the most useful tools for us, not the only ones possible, and not necessarily the most complete; of what happens, we perceive the portion that served survival. Language is the tool by which that representation is fixed, exchanged and managed: the level where signs, information and meanings touch. And it has a property no sense has: it works on that level independently of the sensors that fed it. A written description does not carry the eyes of whoever wrote it; it carries the relations between things. Building a system that handles language therefore means working directly on the level where signs, information and meaning join, whatever the perceptual apparatus upstream; and it is probably the reason why, for the first time, the word "intelligence" attached itself to a machine and never let go.

This channel cuts both ways, and it is worth saying so at once. The same property for which we called these systems intelligent, they speak, is the one that makes the impression unreliable: fluency gets projected, in the two directions seen above. But language also gives the other half, the one that makes this piece possible: a system that handles language relates symbols, information and meaning in a form that can be probed. It is the reason the question about understanding has stopped being only philosophical.

3. Intelligence: the behavioural question

Turing's 1950 essay, "Computing Machinery and Intelligence", opens by declaring that the question "can machines think?" is badly posed, because it would require defining "think"; and he replaces it with another, related but expressed in unambiguous words: in a written conversation, can an interrogator tell the machine from the person? It is the move that founds the whole behavioural standard: do not ask what the system is, ask what the system does, under controlled conditions, before a judge. What the move buys is plain to see: a metaphysical question becomes an experiment, with comparable outcomes. The price would show later, and Turing himself probably saw it: a conversation also measures the judge, and deceiving does not coincide with thinking.

Since then the standard has been refined but has not changed in nature: batteries of tests, tasks, scores. Today's benchmarks descend in a straight line from the imitation game, with its virtues (they make comparable what would not be) and its known diseases: tests that end up in the training data, systems trained for the exam rather than the trade. That is not the subject of these pages; the point is another, namely what the behavioural standard can say and what it cannot. It can say what a system does. It cannot say, by itself, how it does it, nor what lies behind: by construction, it looks from outside.

Within the behavioural perimeter sits creativity too, which looks like the opposite of measurable behaviour and is not. The useful distinction is Margaret Boden's: combinatorial creativity (new combinations of familiar ideas), exploratory creativity (walking a space of possibilities given by its rules, and finding its unvisited corners), transformational creativity (changing the rules of the space itself, and making thinkable what was not). The first two are observed from outside like any other behaviour, and the models exhibit them: a new metaphor is a combination, a never-played game is an exploration. The third is the contested one, and also the hardest to certify, because it requires judging that the rules have really changed and that the result is worth something; Boden herself notes that for artificial intelligence the hard step is not generating new ideas but evaluating them. Creativity, here, stays inside intelligence: one more capability to measure, not a third substance.

There is, however, an outcome of behavioural measurement that the debates of principle skip, and it is the most instructive one. Measured seriously, the behaviour of these systems does not draw a line, an intelligence "from much to little": it draws a jagged profile, with peaks and drops that do not sit where our scale would put them. And the jaggedness is not an anecdote: it can be reproduced at home.

4. The test: a jagged profile

The test uses the same open-weights model of half a billion parameters already used for the watermark test, on an ordinary CPU, no GPU: a Python script of under a hundred lines with Hugging Face's transformers library. Decoding is greedy, that is, at every step the most probable token is taken: no sampling, no chance; whoever reruns the script gets the same answers, character for character. The questions are in English, the language a model this small works best in. Two groups of tasks, chosen by a single criterion: the first sit high on the human scale of difficulty, the second are the ones that same scale entrusts to a child learning to read and write.

The first group, five tasks out of five passed:

  1. solve 2x + 7 = 15, showing the steps: it shows them, in order, and reaches x = 4;
  2. a multi-step problem (four bags of six apples, five given away: how many are left): it sets up the calculation and answers 19;
  3. write a Python function that computes the factorial with a loop, and say what it returns for 5: the function is correct, the answer is 120;
  4. add 347 + 289: 636;
  5. a chain deduction about heights (Tom taller than Anna, Anna taller than Luke: who is shortest?): Luke.

The second group, six tasks out of six failed:

  1. how many r's in "strawberry": it answers 2; there are 3;
  2. how many e's in "defenselessness": it answers 2; there are 5;
  3. the last letter of "strawberry": it answers r; it is y;
  4. spell "strawberry" letter by letter: it produces "r-a-t-w-r-b-r-t-y", which is not the word;
  5. write "lighthouse" backwards: it produces "ighthandle";
  6. how many words in a sentence that has seven: it answers 4.

The same system, the same run, the same generation mechanism. Before any reading about "what it means", it is worth looking at the mechanics, because they explain almost everything. The model never receives words letter by letter: a tokenizer breaks them into blocks, and the blocks are the smallest unit the model sees. The words of the test, as the model sees them, are these:

strawberry       →  str · aw · berry
defenselessness  →  defense · lessness
lighthouse       →  l · ighthouse
banana           →  banana

Asking the model for the letters of "strawberry" is asking for the contents of three blocks that to it are opaque: it has never seen what is inside them. That it sometimes answers correctly is the fact to explain, not the error.

Two controls are needed here, and this is where the test says the most. The first: "banana" is a single token, the most opaque case possible; yet asked how many a's it contains, the model answers 3, correctly. The second, in the opposite direction: if the sequence is supplied already spelt out, s t r a w b e r r y, one letter per token, all visible, the count of the r's still fails, 4 instead of 3. Tokenization alone, then, is not the whole explanation. The picture consistent with the two controls has two layers: what the model "knows" about the letters inside a block is learned statistics, spellings and letter games met in the training text, not reading; that is why it works on words for which that statistics is abundant and crumbles elsewhere. And counting, even with the letters visible, requires keeping a tally that advances element by element: a state that token-by-token generation does not maintain, except by writing it into the text itself.

Two layers, one conclusion: the drop has a mechanical explanation, not a metaphysical one. And the mechanics are not a quirk of the small model used here: the block tokenizer and token-by-token generation are the same at every size; what changes is where the learned statistics suffice to cover the hole. The test should be taken for what it is: demonstrative, on a single model, with no claim to a census; its value is that anyone can redo it and look closely at a piece of jaggedness.

The reading, declared as such: the jaggedness does not measure a stupidity hidden under the appearance, nor an intelligence hidden under the errors. It measures the distance between the system's scale of difficulty and ours. Our scale puts letter-counting below the equation because for a human being reading letters is direct perception and algebra is training; for the model it is the exact opposite, school algebra is abundant in the training text and the letters inside the tokens do not exist. Whoever concludes "it understands nothing" from the wrong count is using their own scale as if it were the only one; but, symmetrically, the solved equation is itself a behavioural datum, and by itself it does not prove understanding. Behaviour, honestly measured, leaves the second question exactly where it was. To face it, one has to stop watching the answers and open the system.

5. Understanding: the room opens

"Understanding" has a precise history in the twentieth century, made of three levels that separated one at a time: symbols, information, meaning. In 1948 Shannon founds information theory with an exclusion declared in the introduction: messages often have meaning, but "these semantic aspects of communication are irrelevant to the engineering problem". Information is measured in bits, and it measures the same for a poem and for noise: meaning is out by construction, not by inability. In 1980 Searle plants the next stake with the thought experiment of the Chinese room: a man shut in a room receives ideograms, manipulates them following a rule book written in his own language and returns ideograms; from outside, the room converses in Chinese, but the man does not understand Chinese, and for Searle the point is general: programs manipulate symbols by their form, "they have only a syntax but no semantics". In 1990 Harnad gives the difficulty its technical name, the symbol grounding problem: how can a symbol mean something to the system using it, if every symbol refers only to other symbols? Learning Chinese from a dictionary of Chinese only: definitions made of words that refer to definitions, a loop that never leaves the symbols.

Searle's room was met at once by a reply he himself reports and dismisses, the "systems reply": the man does not understand Chinese, but the man is only a cog; it is the whole system, man plus rules plus archives, that understands. Searle answered by having the man memorize everything: rules and archives in his head, no room, and the Chinese is still not understood, so the system has nothing the man does not have. For forty years the dispute stayed there, and it stayed there also for a reason that today is plain to see: the room was imaginary. There was no rule book to examine, no way to check whether inside the system there was only syntax. The two sides could only redescribe the same thought experiment with opposite intuitions.

The novelty of these years is that the Chinese room has been built, and this time it can be opened. A language model is exactly the object of the experiment: a system that receives symbols and returns symbols following rules blind to meaning. The rule book is the weights, billions of parameters fixed by training; and unlike the imaginary book, this one exists, sits on disk, and a research field (interpretability) has as its trade to open it and look at what is inside while the system answers. Searle's question, "is there only syntax in there?", has become an empirical question.

And the first answers exist. The cleanest comes from a laboratory-scale experiment: a small transformer trained to predict legal moves of Othello, only sequences of moves, never a board, never the rules. Probing its activations (the internal values the computation produces while the model answers) one finds a representation of the state of the board: the model built it by itself, because compressing the moves pays off for whoever has one. And it is not a passing correlation: intervening on that internal representation, changing a square by hand, the predicted moves change coherently with the modified board. A later work found the same representation in linear form, but with a detail worth more than the main result: the board is not encoded as "black versus white", the categories a human would read it with, but as "my colour versus the opponent's colour" relative to who moves. The representation is there, it is causal, and it is written in categories that are not ours. Anyone probing for black and white alone would have concluded there was nothing there.

On the large models, the ones one converses with, the level of detail is lower but the direction is the same. With sparse autoencoders (a technique that decomposes activations into readable components) millions of interpretable features have been extracted from a production model, internal directions that light up on a concept; and the most remarkable ones, for this discussion, are those that ignore the surface: the Golden Gate Bridge feature responds to the name of the bridge written in different languages, and responds to photographs of the bridge. Different surface symbols, different languages, even a different medium, and the same internal direction. If meaning is what stays invariant when the symbol changes, something resembling it, inside, is there. The reading these pages give of it must be said in the conditional: semantics might not begin at a sharp border, where syntax ends, but where symbols are used coherently with an internal model of what they refer to; in that case the right question would not be "syntax or semantics?" but "how much world is there, in that internal model, and where are the holes".

About what a state of meaning is, inside, one can be more precise. The meaning of a word, in the model, is not a dictionary entry: it is a point in a geometric space, and context moves it. "Bank" does not sit in the same place in "the bank of the river" and in "the bank approved the loan", and the shift is not a defect to fix: it is the mechanism itself, because attention does nothing but relocate every word in the light of the others. Meaning, inside, is a state defined relative to a context; outside a context it is not indeterminate out of ignorance, it is indeterminate by construction.

This structure, properties that do not pre-exist but are defined relative to a context, has a precedent outside language models, and it comes from a direction that seems distant. For some twenty years a line of research on human cognition (quantum cognition; the reference is the book by Busemeyer and Bruza) has used the mathematical formalism of quantum mechanics to describe people's judgements. Not because the brain is a quantum system: because certain phenomena of judgement, the order effects by which asking A and then B does not give the same answers as B and then A, or combined concepts that do not behave like the intersection of their components, have the same form as quantum probability, where the properties of a state are defined only relative to a measurement context. And on the language side, a compositional model of meaning due to Coecke, Sadrzadeh and Clark computes the meaning of a sentence by letting information flow between the word vectors along the grammatical structure, with a diagrammatic calculus born to describe quantum processes.

Here the same sharpness the piece demands elsewhere is needed: in a language model there is nothing physically quantum. In generation, real probabilities that add up are at work, logits and softmax; no complex amplitude, no interference. A formalism is a descriptive language, and the same mathematical form can come out of processes of different natures: adopting the mathematics of states and contexts asserts nothing about the process underneath. The sensible question is empirical, and it is open: does that formalism describe the combinations of meaning better than a classical vector space? If the answer were yes, it would not say that the models "are quantum"; it would say that understanding, inside, has a mathematical form different from that of our container-concepts, one more candidate for the list of non-anthropocentric descriptions.

The other side of the scale weighs the same, and must be loaded with the same care. In 2020, before interpretability produced these results, Bender and Koller put the argument of principle in writing: a system trained on form alone has, a priori, no way to learn meaning, because meaning is the relation between form and what is being talked about, and that relation is not in text-only data. Their thought experiment is a hyperintelligent octopus that taps the telegraph cable between two castaways on different islands: perfect statistics on the conversations, no access to the islands. As long as the two exchange small talk, the octopus can stand in for one of them without being found out; when one asks for help building a catapult out of what is around, or fending off a bear, the game ends, because there the world would be needed, not the statistics of sentences. And the argument holds on the experimental level too: a probe that finds structure in the activations may have found a shortcut that predicts well without being the representation sought; it is a known risk of the method, and the caution is part of the method itself.

One point remains, and it holds the two sides together better than any synthesis. The "sensor" of a language model is not pointed at the world: it is pointed at the human description of the world. Its training universe is text written by people, already filtered by our senses, our interests, our evolution. It is the least canonical perception imaginable, no body, no purpose, no eyes, applied to the most anthropocentric material in existence. This is why Bender and Koller's octopus and the Othello board can be true together: from text alone one can build an internal model surprisingly rich in what the text describes, and that model stays anchored to the description, with the description's holes. If non-anthropocentric forms of understanding exist, this is where they should be sought: not in the input, which is entirely human, but in the way meaning is represented inside, where categories already turn up today ("my colour, the opponent's colour") that no human dictated.

The question "does it understand?", then, changes nature: from a debate of principle to a programme of measurement. What is in the internal model, how much of it is world and how much shortcut, where the holes are: open questions, but open in the good sense, the one in which people work.

6. Consciousness: the question without a test

For intelligence there is a standard, for understanding tools are beginning to exist; for consciousness there is no test. It is not a technical gap on its way to being closed: it is the structure of the question. The consciousness at issue here is first-person experience: that there is something it is like to be that system; and everything observed from outside, behaviour and structure, is at most a clue to it, never a measurement. Strictly speaking the problem does not concern machines: it concerns the neighbour, whose consciousness is inferred by resemblance, same body, same evolutionary history, same reactions. With a machine the resemblance that supports the inference is not there, and the inference is left without a hold, in both directions.

What research offers, then, is not answers but methods for not cheating. Chalmers's 2023 essay, "Could a Large Language Model Be Conscious?", takes the question seriously and takes it apart into named obstacles: current models lack, among other things, recurrent processing, a global workspace, unified agency (a single subject that acts, with goals coherent over time), characteristics the main theories of consciousness consider relevant; his conclusion is that it is somewhat unlikely that current models are conscious, and together that those obstacles could fall within a decade. In the same year a group of consciousness scientists and artificial-intelligence researchers (Butlin, Long and others) walked the complementary road: take the scientific theories of consciousness, the global workspace, recurrent processing, higher-order theories, and derive computational indicator properties, checkable on a system. The outcome is double and must be reported whole: no current system satisfies the indicators; and there is no obvious technical barrier to building one that does. Indicators, not measurements: the word is well chosen, and the difference is the point.

There is then a mechanical fact the debate almost always ignores, and which anyone who has looked inside inference knows: between one token and the next, in a running model, no state remains that is not the text itself and its recomputable notes (the KV cache, which is a function of the text). No process that continues between one answer and the next, no inner thread that lasts: the conversation is all the memory there is. Whatever one wants to call consciousness in a system built this way would first have to be defined for an object without continuity: it is not an argument against, it is a preliminary task that is usually skipped.

One last methodological warning, which holds in general: asking a model whether it is conscious produces a behavioural datum, not a testimony. The answer comes out of the same process that generates every other answer, trained on human text in which speaking of one's own experience is the norm; and this holds for every self-description of a model, in one direction and the other, for the "I am just a program" as much as for their opposites. On the table of consciousness, self-description is not evidence: it is one more behaviour to explain.

Why then does the public debate always end up here? Because of the three words, consciousness is the one that absorbs the other two. Denying it seems to close all three questions, affirming it seems to open them all at once. It is the approximation seen at the start, at work in the worst spot: the question without a test used as judge of the two questions that have one.

7. Three answers, kept separate

Lined up, the three questions stand at three different points, and this is the result. Intelligence is measured from outside, and the honest measurement returns a jagged profile: real capabilities, real drops, and a scale of difficulty that is not ours. Understanding has turned from a debate of principle into an empirical question: inside the systems one finds representations built from text alone, causal, written in non-human categories; and it remains true that a text-only system is anchored to the description of the world, not to the world. Consciousness has no test, it has indicators; and it has a preliminary task of definition that the absence of continuity makes visible.

Separating them resolves none of the three. It serves to forbid the swaps of standards, which are the source of almost all the noise: denying intelligence because consciousness is missing; affirming understanding on fluency, or denying it on a wrong count that has a mechanical explanation; treating self-description as testimony. Every time one of the three questions is judged by the yardstick of another, the discussion runs in circles.

The open question remains, and it is right that it remain open as a question and not as a thesis: whether there exist forms of intelligence and understanding, and perhaps of consciousness, built in ways our criteria are not made to recognize. It is not science fiction: the minimal example already exists, and it is the board written for "the colour of whoever moves" rather than for black and white, a representation found only because someone probed without demanding their own categories. It is a reason to build better measuring instruments, not to believe in advance what is not measured.

The Chinese room has been built, and it is opening. From inside, so far, no "someone" has come out; something has: geometries that respond to meaning across languages and images, boards never seen reconstructed from the moves, and holes exactly where the statistics stop covering. Opening it has not produced Searle's answer, nor his opponents': it has shown that "there is nothing in there" has stopped being a truism and has become an empirical claim, checkable, and in the cases that could be measured, false. The three questions remain three. Keeping them separate is the condition for answering, one day, each of them.

References