Do public opinion surveys work anymore?

A new Pew study shows that less and less people are willing to respond to public opinion surveys. This increases the cost of reliable surveys and raises the question of whether the minority of respondents are representative of the population as a whole.

|
PEW Research Center
This Pew Research Center chart shows how the difficulty of finding public opinion survey respondents has increased since 1997. Today, less than one in ten households contacted will give an interview for a public opinion survey.

Public opinion surveys provide a wealth of information about beliefs in America and around the world. For example, they document how much public approval for same-sex marriage has been increasing, how Facebook has infiltrated many of our daily lives, and how humanitarian aid affects how citizens of other nations view America.

But pollsters face a significant challenge. As the Pew Research Center for the People & the Press notes in a new study, survey response rates continue to plummet:

Fifteen years ago, more than one in three of households responded to surveys. Today, that rate is less than one in ten.

That increases the cost of reliable surveys — to get a reasonable sample, you need to try to contact more households. Even more important, declining participation raises the question of whether the minority of respondents are representative of the population as a whole. The Pew Research Center study took a close look at that question:

The general decline in response rates is evident across nearly all types of surveys, in the United States and abroad. At the same time, greater effort and expense are required to achieve even the diminished response rates of today. These challenges have led many to question whether surveys are still providing accurate and unbiased information. Although response rates have decreased in landline surveys, the inclusion of cell phones – necessitated by the rapid rise of households with cell phones but no landline – has further contributed to the overall decline in response rates for telephone surveys.

A new study by the Pew Research Center for the People & the Press finds that, despite declining response rates, telephone surveys that include landlines and cell phones and are weighted to match the demographic composition of the population continue to provide accurate data on most political, social and economic measures. This comports with the consistent record of accuracy achieved by major polls when it comes to estimating election outcomes, among other things.

This is not to say that declining response rates are without consequence. One significant area of potential non-response bias identified in the study is that survey participants tend to be significantly more engaged in civic activity than those who do not participate, confirming what previous research has shown. People who volunteer are more likely to agree to take part in surveys than those who do not do these things. This has serious implications for a survey’s ability to accurately gauge behaviors related to volunteerism and civic activity. For example, telephone surveys may overestimate such behaviors as church attendance, contacting elected officials, or attending campaign events.

However, the study finds that the tendency to volunteer is not strongly related to political preferences, including partisanship, ideology and views on a variety of issues. Republicans and conservatives are somewhat more likely than Democrats and liberals to say they volunteer, but this difference is not large enough to cause them to be substantially over-represented in telephone surveys.

In short, opinion surveys likely overstate civic activity, but otherwise appear to track other observable political, social, and economic variables.

You've read  of  free articles. Subscribe to continue.
Real news can be honest, hopeful, credible, constructive.
What is the Monitor difference? Tackling the tough headlines – with humanity. Listening to sources – with respect. Seeing the story that others are missing by reporting what so often gets overlooked: the values that connect us. That’s Monitor reporting – news that changes how you see the world.

Dear Reader,

About a year ago, I happened upon this statement about the Monitor in the Harvard Business Review – under the charming heading of “do things that don’t interest you”:

“Many things that end up” being meaningful, writes social scientist Joseph Grenny, “have come from conference workshops, articles, or online videos that began as a chore and ended with an insight. My work in Kenya, for example, was heavily influenced by a Christian Science Monitor article I had forced myself to read 10 years earlier. Sometimes, we call things ‘boring’ simply because they lie outside the box we are currently in.”

If you were to come up with a punchline to a joke about the Monitor, that would probably be it. We’re seen as being global, fair, insightful, and perhaps a bit too earnest. We’re the bran muffin of journalism.

But you know what? We change lives. And I’m going to argue that we change lives precisely because we force open that too-small box that most human beings think they live in.

The Monitor is a peculiar little publication that’s hard for the world to figure out. We’re run by a church, but we’re not only for church members and we’re not about converting people. We’re known as being fair even as the world becomes as polarized as at any time since the newspaper’s founding in 1908.

We have a mission beyond circulation, we want to bridge divides. We’re about kicking down the door of thought everywhere and saying, “You are bigger and more capable than you realize. And we can prove it.”

If you’re looking for bran muffin journalism, you can subscribe to the Monitor for $15. You’ll get the Monitor Weekly magazine, the Monitor Daily email, and unlimited access to CSMonitor.com.

QR Code to Do public opinion surveys work anymore?
Read this article in
https://www.csmonitor.com/Business/Donald-Marron/2012/0521/Do-public-opinion-surveys-work-anymore
QR Code to Subscription page
Start your subscription today
https://www.csmonitor.com/subscribe