Polling: a look inside the machinery of public opinion surveys
Polling: Behind the scenes at Gallup, interviewers and editors try to find out how Americans will vote on election day. With the media's dependence on public opinion statistics, news consumers must educate themselves about which surveys provide valuable data and why.
(Page 6 of 7)
One element of the polling process that's outside the express control of Gallup and its interviewers is the truthfulness of respondents. Pollsters say honesty is less of a problem in political surveys than with those exploring personal habits like alcohol use, sexual activity, church attendance, or compliance with the Internal Revenue Service.Skip to next paragraph
Subscribe Today to the Monitor
"It's more of a problem on questions about behaviors that are strongly socially desirable or undesirable," Keeter, of Pew, says.
Truthfulness could be a factor in answers to that Gallup question about supporting an otherwise qualified individual who happens to be Mormon. There's a danger that the respondent wouldn't be forthcoming because saying no to a Mormon candidate, because of his faith, might reveal an unacceptable prejudice.
But Keeter says that so far this election cycle, any reluctance among voters to communicate that view hasn't panned out in polling: "We certainly get plenty of people who are willing to express that feeling."
Meanwhile, a respondent's predisposition to oppose a candidate based on that person's race or religion usually lines up with a host of other reasons the individual wouldn't otherwise support the candidate in question.
How 1,000 can mimic millions
When the June poll calling was completed, the process got more complicated – hardly a simple tally.
Anna Chan, a Princeton-based Gallup methodologist, took over. She "weighted" the responses mathematically to bring the percentages in the sample closer to known data about the demographic distribution of characteristics – age, gender, region, education, race, etc. – in the general population.
In other words, each respondent is assigned a weight within the sampled group to match his or her representation in the general population.
If Gallup is short respondents in certain demographic categories, those answers will get re-calculated essentially so that they are reflected appropriately in the final results.
"One of the realities of survey research is you're not going to get a perfect representation of the population based on who you're able to call and get a hold of and interview," Jones said. "We take our sample and match it. If we're short on 18-to-29-year-olds, which we routinely are, we'll give them a higher weight in the survey." Likewise, older Americans in the sample would have to be weighted down in influence to compensate mathematically.
The weighted results of the June poll were sent to a data analyst in Princeton – Dave Banas – who produced cross tabulations, a tool used by researchers to analyze the raw data across a range of demographics. This provides Gallup with a more specific sense of what trends or news they've gleaned from the poll. What issues are paramount, for example, to younger voters and which candidate they prefer. Or whom Hispanics are supporting in the White House contest.
From there, Newport and his team determine what's valuable, and several related stories are written and posted to Gallup's website over the course of weeks.
One of those stories explained the results of that question about voting for a well-qualified candidate who happens to be a Mormon. It turns out not much has changed since the father of this year's Republican nominee made his run for the White House; the June poll showed 18 percent of adults would not back a Mormon.
So what should voters read into that number? It's interesting, Newport notes, that numbers for other demographics have moved dramatically over time. In 2007, for example, 43 percent of adults wouldn't back an otherwise qualified gay or lesbian for the nation's top job. That number was down to 30 percent in this poll.