In the Statistics Business, Ideology Colors the Truth
AMERICAN society is in the midst of a new social era, distinct in many important regards from those of earlier times. One important feature of this setting - often called postindustrial - involves the increased prominence of ideas and of the media that disseminate them.
In this new setting, statistics - used to include a wide array of quantitative measures of social experience and performance - have assumed an inportance never before envisioned. The political debate is often between competing sets of statistics. Not surprisingly, there is an enormous temptation to manipulate data for various political ends. To the degree that Cynthia Crossen's new book, ``Tainted Truth: The Manipulation of Fact in America,'' reminds us of this problem's magnitude, she deserves our attention.
Unfortunately, the author sees that there is a problem, but often misunderstands its source and character. For example, Crossen writes that money is behind the ``explosion of corrupted information.'' Various groups, typically those of the business sectors, want certain conclusions and buy them. She writes, for example, of some university professors who sell their souls to self-interested sponsors. There are undoubtedly such corrupted academics, but in 30 years in academic social-science research, including 17 as director of a large university-centered research facility, I can say confidently that this kind of selling isn't widespread.
The corruption of a lot of academic social science accrues not from crass material interests, but from the fact that those of us who do the research are interested participants. We care about the findings because we have chosen sides in the great debates of the society. It's ideology, not money, which is corrupting.
Crossen covers a number of very different areas where social statistics are prominently employed - from food and health statistics, to those involving the environment, to public opinion.
On the whole, her coverage is breezy and superficial. The misuse of social data - finding out where it went wrong - is in fact a very complicated business. The nature of the misuses vary from sector to sector, and they are sometimes quite subtle.
Crossen devotes considerable attention to public opinion. She writes that survey research is a ``soft science'' that yields findings not ``replicable.'' If she means that the findings of a particular survey lack the measurement concreteness and specificity of, say, the reading of the temperature of a pool of sea water with a finely calibrated thermometer, she is certainly right. But a lot of science lacks that precision.
In this regard, survey research is no different than, for instance, theoretical physics. And its findings are, in fact, replicable. An enormous body of sound replication adds greatly to our confidence in the resultant information.
The chief problem those of us in opinion research face is the same one, noted above, that researchers in other social fields confront: ideology. This is demonstrated in Crossen's coverage of the impact of polls on the Clarence Thomas hearings.
Crossen argues that polls mistakenly showed public support for Thomas's confirmation to the Supreme Court which led the Senate to do something it should not and might otherwise not have done.
``Clarence Thomas thanked the American people for their support. If the polls had revealed a doubting public, he might not have become a justice of the US Supreme Court.... Polls - small, hurried and crude - may have tipped the balance on one of the most important political judgments of our time,'' she writes.
Here, however unintentionally, Crossen is herself tainting the truth. From the time George Bush nominated Thomas in the summer of 1991, on through the Senate vote to confirm on Oct. 15, a total of 29 public polls were taken, according to an extensive survey of the data made by the Roper Center for Public Opinion Research. Everyone of them found large majorities or pluralities in favor of confirmation.
The percentage declaring themselves against confirmation was smaller at the beginning, because the number of undecideds was greater, but the relationship between support and opposition was essentially constant throughout.
In addition, in every poll conducted since the Senate completed action, a majority was found to have favored Thomas's confirmation.
If the data are so clear, why is it that Crossen selects this as an example of the misuse of survey research? The answer to this involves less a criticism of Crossen's book than a sobering reminder to all of us engaged in work with social statistics. We all care about the results of political developments like the Thomas nomination. We choose sides. And, if we are not very careful, this choosing shapes the way we view the data we collect.
* Everett Carll Ladd is professor of political science at the University of Connecticut and president of the Roper Center for Public Opinion Research.