Wisdom of Crowds: Why Averaging Guesses Works, From Galton's Ox to the Jelly-Bean Jar

In 1906, at the West of England Fat Stock and Poultry Exhibition in Plymouth, visitors paid sixpence for a ticket on which to guess the weight of an ox once it had been slaughtered and dressed. Francis Galton, then 84 and a lifelong collector of measurements, got hold of the tickets afterwards. The middle guess was almost exactly right.

That small result has grown into one of the best-known ideas in social science: that a crowd's combined guess is often better than nearly any single guess in it. It is also a practical trick. If you play What's Your Guess? with friends, or even alone, the same arithmetic can carry a run further than any one person's instinct.

Galton collected 800 tickets and set aside 13 that were spoiled or illegible, leaving 787 guesses. The ox's dressed weight was 1,198 pounds. The middlemost guess, the median, was 1,207 pounds: nine pounds too heavy, an error of under 1%.

He published the result in Nature in March 1907 under the title Vox Populi, "the voice of the people". His interest was not really cattle. He wanted to know how much to trust the judgement of an electorate, and he concluded that the result was "more creditable to the trustworthiness of a democratic judgment than might have been expected." In a letter to the journal a few weeks later, he gave the mean of the guesses as well: 1,197 pounds, a single pound out. When the statistician Kenneth Wallis went back over Galton's figures in 2014 and corrected a few slips, the mean matched the true weight exactly.

Galton was careful to say why the contest suited the test. Some of the guessers were butchers and farmers with an expert eye for cattle; others, he supposed, went on what they could pick up and on their own fancies. The sixpenny fee kept out practical jokers, the hope of a prize made everyone try, and nobody was swayed by speeches: each guess went down on its own ticket.

Nearly a century later the journalist James Surowiecki opened The Wisdom of Crowds (2004) with Galton's ox, and the phrase became shorthand for the whole idea. The book's examples are still the clearest.

One is the television quiz Who Wants to Be a Millionaire?, where a stuck contestant could phone a friend, usually someone chosen for knowing things, or ask the studio audience to vote. Surowiecki reports that the friends gave the right answer almost 65% of the time. The audience picked the right answer 91% of the time.

Another is the jelly-bean jar, a staple of fêtes and classrooms. Surowiecki describes the finance professor Jack Treynor running it with his students: the jar held 850 beans, the group's average guess was 871, and only one of the 56 students made a better guess than the group.

Surowiecki argued that a crowd is wise only under four conditions. Its members need diverse opinions, so they make different mistakes. They need to be independent, so they are not just echoing each other. The crowd needs to be decentralised, drawing on what different people know locally. And there must be a way to aggregate the guesses into one answer, such as an average or a vote.

Every guess is the truth plus an error. Some errors are too high and some too low, and when the guesses are independent, those errors pull against each other in the average. The more guesses, the more completely the random part cancels.

The economist Scott Page put this as an exact identity in his book The Difference (2007), measuring error as the square of the miss. The crowd's error equals the average individual's error minus the diversity of the guesses, how spread out they are around their own average. A crowd can never do worse than its average member, and the more its members disagree, the more it beats them.

A tiny example shows it. Suppose the answer is 1,000 and three people guess 700, 900 and 1,500. Their individual misses are 300, 100 and 500, an average of 300. Their average guess is 1,033, a miss of only 33. Nobody in the group was that close.

Guesses at an ox's weight cluster tightly, because everyone at a livestock show has a rough idea what an ox weighs. Questions about big numbers behave differently. If a jar holds 2,000 sweets, one person might guess 400 and another 20,000. The high guess is ten times too big; the low one is five times too small. But in an ordinary average the 20,000 counts for far more, and a single wild guess of a million would wreck the whole thing.

For any question where guesses spread across powers of ten, combine them on a log scale. The simplest way is the median, the middle guess, which is what Galton used and which ignores how extreme the extremes are. The more exact way is the geometric mean: multiply the guesses together and take the root. Both fit the way What's Your Guess? scores, as a ratio, where a guess three times too big is as wrong as one three times too small.

The conditions matter, and each one can fail.

• Shared mistakes do not cancel. Averaging removes random error, not error everyone shares. If a whole room half-remembers the same wrong statistic, the crowd will repeat it confidently.
• Talking first destroys independence. In a 2011 study in PNAS, Jan Lorenz and colleagues asked 144 students in Zurich factual questions about Switzerland over five rounds and, in some groups, showed them the others' answers between rounds. Their guesses drew together without getting more accurate as a group, and their confidence rose. The crowd became less wise and more sure of itself.
• Loud voices set anchors. Whoever speaks first gives everyone else a number to adjust from, and people rarely adjust far enough. A group that discusses before guessing often ends up with one person's guess and many signatures.
• No knowledge, no wisdom. A crowd's average is only as good as the scraps of knowledge spread through it. On a question where nobody has any relevant feel at all, the average of noise is still noise, and on some kinds of question people lean the same way, such as underestimating very large counts.

You do not need other people to get some of the benefit. In a 2008 paper in Psychological Science, Edward Vul and Harold Pashler asked people factual questions and then, unexpectedly, asked them to guess again. The average of a person's two guesses was more accurate than either guess alone. The gain was smaller than adding a second person's guess, and larger when the second guess came three weeks later rather than straight away, presumably because by then the first was forgotten.

Stefan Herzog and Ralph Hertwig found a way to get more of that benefit at once, which they called dialectical bootstrapping (Psychological Science, 2009). Make a first estimate. Then assume it is wrong, think about why it might be, and make a second estimate based on that different view. The average of the two beats the first estimate more reliably than a plain second guess does.

In the game, that means a second route. Estimate a question once from population, say, and once from a physical size or a price. If the two land near each other, take their middle on a log scale; the answer box understands powers, so (400 × 9,000)^0.5 gives the geometric mean directly. If they land far apart, you have learned that one route has a broken piece, which is worth more than either number.

The setup that makes a crowd wise is easy to arrange at a table:

1. Read the question aloud, unit and all.
2. Everyone writes a guess privately. No talking, no faces.
3. Reveal all the guesses at once.
4. Take the median and type it in.
5. Only then discuss: whose reasoning was closest, and why?

Played this way, you can watch the effect round by round: the median will often sit closer to the answer than most of the guesses that made it, and the argument afterwards is half the fun. For more ways to play as a group, including scoring games where the crowd competes against itself, see our estimation questions for family game night. And for why a single mind's guesses tend to be too sure of themselves in the first place, read about overconfidence in estimates.

Francis Galton, Vox Populi, Nature 75 (1907). James Surowiecki, The Wisdom of Crowds (Doubleday, 2004). Scott E. Page, The Difference (Princeton University Press, 2007). Jan Lorenz, Heiko Rauhut, Frank Schweitzer and Dirk Helbing, How social influence can undermine the wisdom of crowd effect, PNAS (2011). Edward Vul and Harold Pashler, Measuring the crowd within, Psychological Science (2008). Stefan Herzog and Ralph Hertwig, The wisdom of many in one mind, Psychological Science (2009). The wisdom of the crowd article on Wikipedia gathers much of the later research.