Polling: a look inside the machinery of public opinion surveys
Polling: Behind the scenes at Gallup, interviewers and editors try to find out how Americans will vote on election day. With the media's dependence on public opinion statistics, news consumers must educate themselves about which surveys provide valuable data and why.
(Page 4 of 7)
Other questions are leading. Such as this example Krosnick suggests: "Some people believe that Barack Obama was born outside the United States. How much do you agree or disagree?"Skip to next paragraph
Subscribe Today to the Monitor
The question sends a signal that the interviewer – or survey sponsor – might have an agenda. A more neutral way to ask would be to also include a mention that others believe Obama was born in the US.
Yet another way to skew results, says Krosnick, is to target respondents who favor a candidate or issue. "If you prefer to slant a poll in a particular direction, then you can make decisions about who to go after and how to go after them," he says. "If a candidate does better in higher socioeconomic status [areas], use more respondents in [those areas]."
Science of probability
Once Newport and his colleagues formulate poll questions, the survey is sent to Tara McGhee, a survey design editor in Omaha. Ms. McGhee takes every path through the survey to make sure it's programmed correctly; in other words, depending on a respondent's reply – if he or she offers support for Romney or Obama, for example – that person is branched through the survey questions differently. Follow-up questions are prompted by their answers. She also checks spelling and grammar.
"We are expected to be perfect," says McGhee, who has been with Gallup for 10 years. "We are the last set of eyes on it. My personal mission is quality."
Stephanie Morrow, in the firm's nearby Lincoln, Neb., call center, is in charge of getting the teams of individuals in five offices – including Houston, where the Spanish-language outreach is done – ready to make their calls. About 100 interviewers worked the June political poll. As the survey got started – it ran for four nights, from Thursday, June 7, to Sunday, June 10 – Ms. Morrow and her team did random quality checks, listening to recordings of the calls.
As it has done for years, Gallup outsources a critical part of the polling process – sampling – to a firm called Survey Sampling International, which has an office in Connecticut.
This is an element of the undertaking that most respondents and citizens are blind to – and it's vital. Sampling is the process of finding a random group of Americans – 1,004 in the case of the June Gallup poll – whose responses represent the same views that would be obtained if every American adult were interviewed. (Sample sizes do vary among reputable outfits and can be smaller, though experts believe the number used by Gallup, and others, is a good one.)
Survey Sampling International gives Gallup a set of listed and unlisted phone numbers. And those numbers are entered into Gallup's sampling and interviewing software and then called automatically by the system. Gallup inter-viewers take over from there.
The process involved in choosing those numbers is called random-digit dialing. The scientific premise – the equal probability of selection principle – is that if all possible homes, in other words every number in America, are available to be randomly called, the survey will produce a collection of answers that can be generalized to reflect the entire population.
Note, too, that businesses must be weeded out of any call list.
"The power of random sample is fairly miraculous," Newport says.