Deep down, educated people know they put too much faith in the statistics they read: 300,000 individuals die each year in the United States from illness related to obesity? "Wow!" we say, failing to think through the special interest of groups, such as the American Obesity Association, involved in disseminating that number; failing to analyze why a newspaper article from The Associated Press citing that number might need to be doubted; failing to question how such a specific number can be determined without qualification.
Joel Best, a sociology and criminal- justice professor at the University of Delaware, reminds us that we need to treat statistics skeptically. Best is not the first to sound the warning, but his book, "Damned Lies and Statistics," is a clearly written primer for the statistically impaired. It is as important to discussions of public policy as any book circulating today.
Best is persuasive early in the book that citizens do not have to feel helpless about numbers. Most individuals handle small numbers well. As he says in a homely but effective example, "Everyone understands that it makes a real difference whether there'll be three people or 30 coming by tomorrow night for dinner. A difference (30 is ten times greater than three) that seems obvious with smaller, more familiar numbers gets blurred when we deal with bigger numbers."
As an example of a bigger number, Best uses the purported count of homeless in the United States. Is it 300,000 or 3 million? Nobody really knows. Yet many citizens are willing to accept uncritically the smaller estimate or the estimate 10 times larger, believing that either way it's a large number indicating a possibly intractable problem.
Such uncritical acceptance baffles and angers Best: "If society is going to feed the homeless, having an accurate count is just as important as it is for an individual planning to host three - or 30 - dinner guests."
By way of example, he introduces us to a caring soul working at a shelter for runaways. That person decides to collect statistics for one month on the shelter's clients. But do the individuals passing through that month represent all runaways? Of course not. Many runaways never seek shelter at such places. So, do the clients that month at least represent runaways who turn to shelters? Probably not. If it's winter, the clients might be different from those who appear in the summer, to mention just one of many variables.
To think wisely about such numbers, Best suggest thinking through these questions:
* Who created this statistic?
* Why was this statistic created?
* How was this statistic created?
The answers to those questions usually show that statistics are not created in a vacuum, that each creator of a statistic has a political-social-economic agenda, no matter how well-intentioned.
Best explains the four basic sources of flawed statistics (bad guesses, deceptive definitions, confusing questions, biased samples). Then he examines mutant statistics and discusses the illogic of statistical comparisons that attempt to equate differing time periods, places, groups, or social problems.
In his introduction, he recalls a statistic from a doctoral dissertation which claimed that every year since 1950, the number of American children gunned down had doubled. Huh? Simple multiplication would have shown that such doubling each year was illogical. By 1987, for instance, the number of gunned-down children (137 billion) would have surpassed all of the world's population throughout history.
Best peppers the text with such compelling examples. Anti-stalking groups somehow arrived at an estimate of 200,000 persons being regularly harassed throughout the country. Within months, that estimate had hardened into fact, with no meaningful attribution, no explanation of methodology. Journalists became part of the problem, as they often are in such instances, by failing to scrutinize the claims.
Is it possible to actually prove anything with statistics? For a brief passage, Best sounds something like Bill Clinton, claiming it depends on what "prove" means. Some questions are answerable with access to the best statistics possible. Other questions are not. That sounds so obvious. But the obvious has yet to sink in.
Steve Weinberg is a journalist in Columbia, Mo.
(c) Copyright 2001. The Christian Science Monitor