Polling: a look inside the machinery of public opinion surveys
Polling: Behind the scenes at Gallup, interviewers and editors try to find out how Americans will vote on election day. With the media's dependence on public opinion statistics, news consumers must educate themselves about which surveys provide valuable data and why.
(Page 3 of 7)
The public can seek out a reputable poll's methodology – easiest to do when browsing results online – to determine if a poll was automated.Skip to next paragraph
Subscribe Today to the Monitor
Andy Smith, director of the University of New Hampshire Survey Center, says with a laugh that he puts his 11-year-old on the phone to answer robopolling questions. Obviously, Mr. Smith's son is not of voting age.
"I'd just keep away from them," Smith says of the surveys, suggesting they do a poor job of communicating the public's position on issues.
Gallup uses humans to craft the polls and conduct them. It's the people at Gallup who inject the enterprise at turns with critical subjectivity.
Gallup's Mr. Newport, based in Washington, and his colleagues – managing editor Jeff Jones in Princeton, N.J., and senior editor Lydia Saad in Connecticut – were responsible for crafting the June survey.
During a 90-minute conference call – conducted with the efficiency, shorthand, and wonkish tone of longtime colleagues – they reviewed a core set of questions asked monthly since 2001. Unlike others in the field who might seek quick headlines with data, Gallup's team is looking to explore what Americans are feeling now and to establish a long-term trend of opinion (which itself can produce headlines, too).
The three wended their way through pages of possible questions, including those about Mr. Obama's approval ratings, perception of the job market, support for third-party candidates, and top reasons the respondent is supporting a particular presidential candidate.
With five months until voters go to the polls, they thought June marked a good time to assess the role of religion in the presidential contest – and Newport noted that he had gotten calls from journalists, academics, and regular citizens asking about the issue.
Mr. Romney is, as election watchers know, a Mormon. Newport, Mr. Jones, and Ms. Saad thought it wise to update the trend on Romney by asking voters if they would support a generally well-qualified candidate who happened to be Mormon. Much as Obama's race could have been a roadblock for him last cycle, there's interest in determining if Romney's religion might be a sleeper factor this year.
"I think religion is going to be at least subliminally an issue in this election," Newport said on the conference call.
While the public has shown growing acceptance over time for black, female, Roman Catholic, and gay candidates, the numbers for Mormons have remained "flat historically," Jones said. When Romney's father, George Romney, ran for the Republican nomination for president, an April 1967 Gallup poll indicated that 17 percent of Americans would not support a Mormon's candidacy.
While question substance is one goal of the call – and a follow-up conversation a day later – another is to make sure the questions asked will prompt a phone conversation of 18 minutes or less. Gallup research indicates that respondents tend to tune out or jump off a call at that point.
Mr. Krosnick of Stanford notes that this preliminary part of the poll process is key because poor question construction can create bias and ultimately influence the data. Surveys that pose agree-or-disagree, true-or-false, or yes-or-no questions typically push people in the affirmative direction, he says.