The Dig · Declassified

CIA analyst proved ‘likely’ means nothing without a number

Sherman Kent’s 1950s experiment showed that vague probability words scatter across a range so wide they become useless for intelligence judgement.

In the early 1950s, Sherman Kent ran a small, ugly experiment inside the Office of National Estimates. He asked a room of experienced analysts what percentage they attached to the phrase ‘serious possibility’. The answers ranged from around 20 percent to 80 percent. Same words; wildly different meanings in different heads.

The finding held up because it was blunt and repeatable. Kent tested other stock phrases too: ‘probable’, ‘likely’, ‘we believe’. Each one produced a spread of numbers wide enough to sink a policy decision built on it. If an analyst writes ‘probable’ meaning 60 percent and a president reads it as 90 percent, the paper has failed before it left the building.

This is why the finding earns Diamond status; it survived hostile scrutiny for decades and got written into agency tradecraft standards, the ‘Words of Estimative Probability’ scale that assigns numeric ranges to terms like ‘almost certain’ and ‘remote’. Kent’s argument was not stylistic. It was operational. Words lie.

The awkward part is that the fix rarely sticks. Analysts still reach for hedge words because ranges feel like exposure, and nobody wants to be the one who put 35 percent on a coup that happened. The programme proved the problem in the 1950s. Six decades of declassified files show agencies still arguing about how to say what they mean.

Source: The Dig / declassified CIA files source