In 1991, at a computer graphics conference, Loren Carpenter handed several thousand strangers a game of Pong and no instructions. Each person held a cardboard wand, red on one side, green on the other. A camera read the colours; the balance of the room moved the paddle. The left of the auditorium drove the left paddle, the right drove the right. Nobody was told how it worked. Within seconds the crowd was playing, and playing well. The paddles tracked the ball. Rallies held. A room full of people who had never met produced competent play out of nothing but the average of their individual guesses.
It's usually told as a parable about collective intelligence, and it is one. But the more interesting version is what happens when you change a single variable. Let the crowd see itself. Let people react not to the ball but to the motion of the paddle they are collectively producing — correcting for the correction, chasing the thing they are creating — and the wisdom evaporates. The paddle judders. It oscillates. One person swinging wildly is enough to start a wobble that the others amplify as they try to compensate. The same several thousand people, in the same seats, become a herd.
That is the uncomfortable part, and it's worth saying plainly. The intelligence was never in the crowd. It was in their independence. The crowd was never wise; its independence was. Remove that one condition and the people don't get less clever — they stay exactly as clever as before — but the mechanism that turned their guesses into competence quietly stops working.
Keep that room in mind. We are building a very large paddle.
The wisdom of a crowd rests on one fragile condition. Not size. Not even variety, though variety helps. It rests on independence — each person's answer being their own, reached without looking at anyone else's.
Think of how the game worked. Every wand in that auditorium was one person's private guess about where the paddle should go. Nobody coordinated; the screen simply added the guesses up. When a thousand people guess on their own, their mistakes point in every direction and cancel out, and what's left is surprisingly close to right. But when those same thousand copy each other, the mistakes stop cancelling and start stacking. Independent guesses correct each other. Copied ones compound. This is not a metaphor; it is arithmetic.
Now put a language model in the room. Not on the agenda — in the room, in the sense that every person around the table has already, privately, asked it what they think before they arrive. The strategy lead checked her framing against it. The analyst had it draft the first version. The sceptic used it to sharpen his objection. By the time the meeting opens, the diversity of independent answers the room is meant to aggregate has already collapsed, because the answers are no longer independent. They have a common parent. Five people are speaking; one source is talking.
This is what happens with a bigger paddle. In Carpenter's game no single wand mattered; the competence came from averaging thousands of small, independent guesses. A model in an organisation is not another wand. It is an input so heavily weighted that the rest stop counting. The room still produces something that looks like a group's judgement — a consensus, a recommendation, a deck — but the paddle is really tracking a single source and calling it a crowd.
And it does not take much to cause this. In 2011 a group of researchers ran a simple experiment. They asked people factual questions that had a number for an answer — how far it is from London to Rome in miles, that sort of thing — and, as expected, the average of everyone's independent guesses came out close to the truth. Then they ran it again, but this time let people see what the others had guessed before locking in a final answer. That was all it took. The group bunched together — everyone's answers drew closer to each other — but they got no nearer the truth. They agreed more and knew no better.
If simply glimpsing the group's average can do that, then a fluent, confident, always-available machine that every person consults in private is not a mild nudge. It is the strongest social influence ever introduced into an organisation. And it does not feel like influence at all. It feels like help.
The obvious worry is that the machine will be wrong and the team will follow it off a cliff. But that is rarely what happens. More often the machine is right — frequently sharper than the person asking it. The danger sits one step upstream, in the question. A good answer depends on a good question, and someone who has stopped thinking for themselves asks shallow ones. The model answers exactly what it was asked, beautifully, and the team goes over the cliff not because the machine failed but because nobody framed the problem well enough to see the cliff was there. Rubbish in, rubbish out — now delivered with total fluency.
The same 2011 experiment found one more effect, and it is the most dangerous. After people saw each other's answers and bunched together, they didn't just agree more — they felt more confident. The agreement itself felt like proof. When your answer drifts toward everyone else's, that shared position feels like confirmation you were right, even though nothing about the huddle made the answer any more correct. People came away more confident and no more accurate — the worst possible pairing.
More recent work on people using AI to make decisions finds the same signature. Give someone a fluent explanation from a model and their confidence climbs while their accuracy stays flat — and worse, their ability to catch the model's mistakes drops. In one study, most people typed a single question, read the answer, and stopped. Why look further? It was so well written.
This is the mechanism that should worry a leader, because it is invisible from the chair you are sitting in. A team that has quietly converged on a single source looks, from the head of the table, exactly like a team that has done its homework and reached hard-won agreement. The room is calm. Everyone nods. The recommendation is coherent and well argued. Rising confidence looks like rising quality. It is not. It is the sound of independence leaving the building, and from the inside it is indistinguishable from a job well done.
There is a stranger cost still, and it falls not on the team but on everyone reading the team from outside. Normally, when a spread of independent guesses fans out, the truth sits somewhere in the middle of them — that's why the average works. But when a group herds, they pile up together in one spot, and that spot is not necessarily where the truth is. The truth gets stranded off to one side while everyone crowds around an agreed answer that feels right precisely because everyone else is standing on it. So anyone outside who trusts "what the group settled on" is now further from the truth than if the group had never compared notes at all. The agreement did not just fail to help. It made the group a worse guide for everyone downstream.
Watch what that does inside an organisation. The studies of AI-assisted work already show the shape of it: writers given the same model produced individually better work whose outputs grew measurably more alike; consultants using AI generated higher-quality solutions across a narrower and narrower range. Each person improves. The collective flattens. And the people who depend on that collective's judgement — the board reading the recommendation, the regulator reading the filing, the market reading the guidance — are reading a signal that has quietly lost its independence, dressed as a confident consensus.
So the organisation is at its most persuasive precisely when it is least readable — because the thing that consensus used to encode, many minds arriving separately at the same place, is exactly the thing that is no longer happening.
Which raises the question the whole apparatus is built to avoid. If your team has never looked more aligned, and alignment was supposed to be the goal — what is that agreement actually measuring? A room of people who thought for themselves and arrived, separately, at the same place? Or a team quietly guided to the same answer by the same machine, who forgot they ever had a judgement of their own?