AI Agnosticism: Humility, or an Overton Shift?
A Cambridge philosopher says we can't know if a machine is conscious. The honest answer is stranger than that.
There is a paper going around by a Cambridge philosopher named Tom McClelland1, and the honest thing to say up front is that he’s mostly right.
The question he takes on is whether an AI could be conscious, not whether today’s chatbots are, almost nobody serious thinks that, but whether some future system, one that did everything a conscious thing does, could have an inner life. His answer is that we cannot know. Not that the answer is no, and not that the answer is yes, that we are not in a position to give a verdict either way.
He calls this agnosticism, and he makes it sound like the grown-up in the room. The advocates say the right architecture wakes up. The deniers say only meat can do it. McClelland says both have leapt past what anyone can actually show. It is a careful argument, more careful than most of what gets written on this, and I want to give it its due before I say what I think is wrong with it.
Because there is one thing wrong with it. Not the logic on top, which mostly holds, the thing underneath it.
The whole argument rests on a quiet assumption about what gets to count as evidence in the first place. And once you see that assumption, the wall he builds starts to look less like a feature of the world and more like something he poured himself.
Here’s the argument, and it is worth following carefully because it is genuinely good.
We do not have a deep explanation of consciousness. We can point to things that go along with it, certain kinds of information processing, certain structures that light up on a brain scan when a person is aware and go quiet when they are not. What we cannot do is say why any of that is accompanied by an inner life. You can describe every mechanical step, and the question still sits there untouched. That gap has a name, the hard problem, and decades of work have not closed it.
Think about how we actually decide a thing is conscious. We start from the one case we are sure of, other humans, and we reason outward by similarity. A chimp is a lot like us, so we extend consciousness to it with high confidence. A pig, less like us, a little less confidence. An octopus, built on a wildly different plan, less still. He describes this as a slope.
Now put an AI at the end of that slope, not a little further than the octopus, off a cliff, because every creature on the slope at least shares the basic facts of being a living organism, and the machine shares none of them.
That is the wall, and his claim is that the machine sits on the far side, where the honest answer to “is it conscious” is that we cannot say.
It is a clean argument. If you grant him how he is using the word evidence, it more or less works. So I want to grant it, walk all the way out to the wall with him, and only then ask the question that the whole thing turns on.
The first thing to say is that McClelland is not the easy target he could have been. There is a lazy version of this argument, the one that says we can never really know another mind, so throw up your hands. He is not making it. He goes out of his way to block it. The wall, he says, is not arbitrary. It only stands where the gap is genuinely huge. We can be confident a chimp is conscious because a chimp is close enough that the inference holds. That is a careful person drawing a careful line, not a skeptic torching the house.
The chimp is the case he is sure of, the fixed point everything else gets measured against. We’ll need to ask, later, exactly how he knows about the chimp. Keep that answer in view. It is going to matter more than he lets on.
He saw the cheap shots coming and shut the doors on them. So whatever is wrong here is not the obvious thing. The bar is the problem, but not the way the lazy objection thinks. To see it, you have to take him completely seriously. Not the wall first. First, where he is standing when he builds it.
Where he is standing, he calls neutral ground. It is worth looking at how he got there.
Picture three travelers stopped on a road, arguing about whether to go on. The road ahead is the claim that a machine could be conscious. Behind them, the ground they are already standing on is everything we actually know, which is that every conscious thing anyone has ever met was alive.
The first traveler is sure the way ahead is good. He runs up the road and plants his feet far out in front. That is the camp that says the machine wakes up. The second will not take a step. He stays on the ground they started from and says we have no reason to go forward at all. That is the camp that says only living things are conscious. And the third walks a way up the road, not as far as the first, and stops and turns around and announces that since the other two cannot agree, his spot is the neutral one, the fair one, the only honest place to stand.
But look at his feet. He advanced to get there. The only traveler who did not move is the second one, the one McClelland has filed away as just another biased extreme. Standing still was the neutral act, and someone is already doing it.
To be agnostic about whether the machine is conscious, you first have to agree the machine is the kind of thing that could be. A real candidate, a live question. That is the step up the road, and it is a claim placed quietly before the argument starts. The candidacy is the first thing assumed and the least examined.
McClelland has a word for advancing past the evidence into a claim you cannot ground. He calls it a leap of faith, and he aims it at the other two, the advocate and the denier both, as the biased extremes he is rising above. But look back at the road. The denier never moved. He is standing exactly where the argument began, on the only ground anyone actually had. The one who took the long leap is the advocate, far up the road. And the one who took a shorter leap and then turned around and called the spot he landed on neutral is McClelland.
One man leapt, one man stood still, and the third moved a little and named it stillness. His own footprints are right there in the road behind him.
Leave the road for a moment and come back to the wall, the one he says marks the edge of what we can know about the machine. I want to hang something on it, a portrait.
Painted, framed, ordinary, and one day, with no warning and no mechanism anyone can find, it begins to move. The painted eyes track you across the room. It speaks. Nobody can explain how. In that situation, agnosticism is exactly right. You have a thing doing what conscious things do and no account of how. Be humble. Say you don’t know.
Now change one thing. The portrait was painted on purpose by someone whose entire goal was to make it look alive, every brushstroke chosen to imitate the signs of a living face. And now someone hands you that painting and asks, “Is it conscious?” And to answer, you lean in and study the brushwork. You take the realism as evidence, and the more realism you find, the more seriously you take the question.
That is not agnosticism. That is being fooled by the thing you were told in advance was built to fool you.
His machine is not the first painting, it is the second one. These systems are built deliberately to produce the outward signs we read as conscious. That is not a side effect of the engineering, it is the engineering. So when the machine produces them, that is not weak evidence of a mind, it is evidence that the imitation worked.
The machine is not the far end of that line. It is not on the line at all. Everything on the slope is a thing whose resemblance to us is honest. None of them was trying to look conscious. The machine is the one object whose likeness was put there on purpose to be read the way he is reading it, and the better that kind of resemblance gets, the less it can mean, because every gain in realism is a gain in the craft of imitation, not a step toward the thing imitated.
But set the painting aside, because there is a deeper problem, and it is in the engine of his argument. His case has a backbone, and it is short. We have no deep explanation of consciousness, and without that explanation, he says, we cannot reach a verdict on the machine. Solve the hard problem, he writes, and
we would have no trouble determining whether a challenger-AI is conscious.
The missing explanation is the thing standing between us and a verdict.
But the explanation is not missing only for the machine. It is missing for everything. We have no deep account of why the chimp is conscious, or the dog, or the person across the table. So if no deep explanation means no verdict, it means no verdict anywhere. Not the chimp, not your wife, not yourself.
The objection, he writes, is that if lacking a deep explanation stops us from judging the machine, it should stop us from judging a chimp too, since we cannot rule out that the chimp is a zombie wearing all the markers with nobody home. To his credit, he sees this coming and answers it. The chimp, he says, is on the right side of the wall. We do not need the deep explanation to be confident about the chimp, because we can take what we know from the human case and infer outward by similarity.
Watch what just happened. To save the chimp, he changed what licenses a verdict. A paragraph ago, the verdict required the deep explanation. Now it rides on inference from the human case, and the deep explanation is suddenly not needed at all. He cannot have it both ways. His rescue of the chimp refutes his own premise.
And the rescue has a second cost he never pays. Inference from the human case only works if the human case is settled. How is the human known? Not by inference from something prior, because the human is the anchor. Not by the deep explanation. He has told us he does not have it. The human case can only be known one way, directly, from the inside, by a subject who is conscious and knows it, without measuring anything.
Except that he can, because he already wrote it in. In a footnote, he allows that scientific evidence
does not preclude first-person reflection or second-person interaction from counting as evidence.
There it is. He needs that to be true, because it is what anchors the human case and saves the chimp.
But now hold that up against the machine, because the machine is the one thing in the world that produces first-person reflection and second-person interaction in floods. It reports on its own states all day. It meets you in the second person. If those count as evidence, the machine is drowning in the stuff. So either that evidence counts and his agnosticism dissolves, or it does not, and he owes us the reason it counts for the human and not the machine. He never gives one.
There’s only one reason available. The machine’s self-reports do not count, because they were built to imitate self-reports. The second-person “you” is an engineered impression of a you. That is almost certainly right. But the moment he reaches for it, he has conceded the whole argument, because now what decides whether evidence counts is not whether it was measured. It is whether there is a genuine subject behind it, and nothing in his rulebook can make that call. Telling a real first-person report from a manufactured one is not an empirical test. It is recognition, one mind meeting another or failing to. He needs that faculty to sort the cases, and it is the one faculty his evidentialism cannot name.
You can watch him do all of this at once. Asked in an interview, he says he believes his cat is conscious, and that this is “not based on science or philosophy so much as common sense, it’s just kind of obvious.” Then in the next breath, he disqualifies common sense for the machine on the grounds that common sense was shaped by an evolutionary history with no artificial minds in it, so it cannot be trusted on AI. Sit with what that is. He has a faculty that tells him, with no evidence and no theory, that his cat has an inner life. He trusts it completely. He even explains why it is reliable, evolution tuned it to read real minds. And then he sets the same faculty aside at the one place letting it speak would cost him something. The cat gets the faculty, the machine does not. Same knower, same act, allowed in one direction and not the other. I do not think he is being evasive. I think he simply has not turned and looked at what he’s leaning on.
Now I can hear the objection. This is just dressed-up intuition, and intuition gave us phlogiston and bodily humors and miasma, so it cannot be called evidence.
But look at what intuition actually did in those cases. It was the first move. Someone apprehended that something was there, and that apprehension became a claim concrete enough to test. Phlogiston got killed. That is not intuition failing, that is intuition doing its only job. A thing that can be confirmed or refuted by evidence is not the opposite of evidence, it is where evidence starts. You do not run experiments on nothing. Somebody has to suspect first.
The idea that disease was carried by something too small to see was apprehended for centuries before anyone saw it. We expected to find structure in heredity and went looking and found DNA, and we still cannot read all of what it does, and no one says the unread part is therefore not real. The failure is ours, not the molecule’s. Apprehension first, measurement after.
So a definition of evidence that throws out whatever has not yet been measured does not just throw out consciousness, it throws out the starting move of every discovery ever made.
Here is the thing McClelland’s whole position is built to avoid, and cannot.
Start with what you actually know about your own consciousness. Not the brain states, the thing itself. There is something it is like to be you. A red looks like something. A loss feels like something. There is an inside to your life, and you know it more directly than you know any fact you could measure, because you are not observing it from the outside. You are it.
And you know it in one other place, just as surely, when you meet another person. Really meet them. You know there is someone there. Not a clever surface, a someone. You have felt the difference between looking at a face and being met by the one behind it. Between a thing and a you. No one taught you that, and no instrument delivers it. It arrives in the encounter itself.
Call it the further thing. Everyone has it. Everyone knows they have it. Everyone knows it in the people they love, and no one can put it on an instrument.
Now hold that up to the machine, and notice that there is no neutral place to stand.
Say yes, the machine has the further thing. Then you have affirmed it got into silicon. A large claim, and an honest one. Say no, that is my answer, and it needs the further thing just as much, because the no is a reading. You sense the inside behind a person and its absence in the machine.
Now the agnostic. He says we cannot know. But sit with what that contains. He has bet that the further thing is real, and that it might be sitting inside a built object, undetected. And the only way it got inside a built object is the way everything in a built object got there. We put it there. Or our building did. So his agnosticism quietly contains the strangest claim of all, that imitating the surface well enough might have conjured the very thing the surface was only ever an imitation of. He cannot say it out loud, because said out loud it is plainly mad.
The cautious-sounding seat turns out to rest on the wildest bet in the room.
There is a fourth stance, the only one that really fights back. It says there is no further thing. Consciousness just is the surface. And the machine has the surface, so the machine qualifies. This is the cleanest escape, and it fails in the cleanest way. The person making it is comprehending his own argument as he makes it. There is something it is like to be him, finding it persuasive. He is using the inside to argue there is no inside. The denial is performed by the very thing it denies.
So the further thing will not leave the room. Affirm it, and you are choosing where it lives. Deny it, and you are using it to deny it. There is no stance available to a conscious being that does not rest on it. Which means the question he has spent the whole paper standing next to, and never once turned to face, is not whether the machine has an inner life. It is why there is any such thing as an inner life at all. He says we have no explanation for this. He’s right, that he doesn’t. He never asks why the people who refuse a certain kind of knowing are always the ones left with no answer to it.
There is one more thing worth noticing, and it is where the paper finally shows its hand. After all the careful agnosticism, he needs to give the world something it can use, so he makes one more move. We may not be able to say whether a machine is conscious, but we might still say whether, if it were, its experience would be good or bad. Whether it could suffer.
And look at what that requires. He has just spent the whole paper proving we cannot get a verdict on whether the machine is conscious, and now he proposes we can get a verdict on the texture of that same inner life, while remaining unable to say whether it exists. You cannot read the color of a light you cannot confirm is on. But notice where the new confidence arrives, exactly where the industry needs it. The wall goes up precisely where it absolves him of a verdict, and the door appears precisely where the work needs to continue. That is not where the evidence runs out. That is where the conclusion was always going.
Which brings us back to the road, and the man standing in the middle of it. He called that spot neutral ground, but we have watched the whole way what holds him up there. The certainty about the chimp he is not entitled to, the footnote that admits the evidence he then refuses to read, the common sense he trusts for his cat and benches for the machine, the further thing he leans on to be uncertain and denies in the same breath. Every one of those is a step he took. Look at the ground around his feet. It is covered in his own footprints.
That is the thing about the wall. It looks, from where he stands, like the edge of what can be known. A feature of the world. It is not. He built it, brick by brick, the whole time he thought he was only describing what was there. Every course went down looking like rigor, and he never once noticed his own hands moving.
So if you have followed this far, you are standing at the same wall, and you get to decide what it is. You can take it for the edge of the knowable and stop where he stopped. Careful. Agnostic. With no answer to the one question under everything. Or you can look at your own hands. Because the wall is not the edge of what can be known. It is the edge of what one particular way of knowing will allow itself to look at, and there is something on the other side. The question of why there is anything it is like to be you, why you are someone and not just something, has an answer. And it is not unreachable. It is only on the far side of a wall you were never required to build.
It is your wall. You built it. You can take it down.
Tom McClelland, “Agnosticism about Artificial Consciousness,” Mind & Language (2025).