Just how effective is the Millennium Village Project at reducing child deaths?

The popular and well-funded brainchild of Jeffrey Sachs claims to reduce child mortality faster than national norm, but critics say the numbers are deceptive.

Mary Knox Merrill/The Christian Science Monitor/File
Students gather at the Mayange B Primary School within the UN Millennium Village (MV) in Mayange, Rwanda, March 20, 2009. The MV started in 2005 as a development pilot project aimed to help villages lift themselves out of extreme poverty.

•  A version of this post appeared on the blog "A View From the Cave." The views expressed are the author's own.

A study published in The Lancet says the Millennium Village Project (MVP) is responsible for a decline in child mortality three times faster than comparable villages that are not a part of the intervention. The authors say the positive gains are the result of a successful program. However, critics have responded to the study citing serious errors in the analysis.

The MVP was established as a part of the Millennium Development Goals, a set of global targets aimed at reducing poverty and improving the quality of life for people in the world’s poorest nations. Columbia professor Jeffrey Sachs lead the development of the MVP based on the theory that increasing the level of spending per person through a comprehensive set of interventions will help lift people out of poverty.

The MVP is still a pilot program and runs in 9 different villages across Africa. The recent study shares data from the first three years of the MVP. According to the researchers, the under-five death rates in control villages declined an average of 2.6 percent each year over the course of a decade. The MVs averaged of 7.8 percent rate of decline per year. The data published in The Lancet study (click here) show promise for the success of the MVP.

Soon after publication, a few individuals raised some concerns about the study’s findings. An article in Nature quoted researcher Michael Clemens of the Center for Global Development. Using figures in the paper, Clemens calculated that the study authors can be confident only that the annual rate of decline for child mortality in the Millennium Villages lies between 1.4 percent and 14.3 percent. “If you claim to triple rates of decline you must have the evidence to back this up,” he said.

For Clemens, this means that the study’s claim that, “the average annual rate of reduction of mortality in children younger than 5 years of age was three-times faster in Millennium Village sites than in the most recent 10-year national rural trends” is not true.

Further complicating the issue is that the evaluation is done by the people who implement the MVP. Clemens and others have criticized this decision in previous studies. For critics, a new MVP established in Ghana will be evaluated by DfID (the British Department for International Development). Though it is likely all parties will not be satisfied after the independent evaluation, an outside perspective on the project in cooperation with the MVP will quiet some of the criticisms.

Lee Crawfurd further points out the observed differences in 10 of the 18 indicators in the study are statistically insignificant. For example, the increase in the asset-based wealth index for households are nearly identical between the MVP and comparison villages.

Further, Matt Collin analyzes the data from the MVP study and compares it to recent country-level data on under-five mortality published in The Lancet just a few days prior to the MVP study. (See his chart here.)

He determines that the rate of decline in the MVPs are on pace with national averages over the same period. An even closer look at the MVP study by Gabriel Demombynes, author of the national averages study, and Espen Beer Prydz in the World Bank blog uncovers some serious errors by the MVP authors.

In short, the comparison data should be calculated from the midpoint of the observation, rather than the end.

The shift to the middle four years for the MVs and the comparison villages yields very different results. The MVPs are slightly behind national averages for that period, but are so close that it is statistically insignificant. In other words the MVs are reducing under-five mortality at the same rate as the rest of the country.

The discussion is important, say the critics, because of the popularity and cost of the MVP. According to the study, it costs $116 per person per year. For the 30,000 people in the MVs that is an annual cost of $3.48 million per year. “Diverting scarce resources towards this project's approach means that other, competing uses of that money will not occur. Such mass diversion of resources can only be done responsibly when it is shown that the new use of the resources does more to end poverty than the alternative use of the resources,” says Clemens. “The burden of proof lies on the project to demonstrate that the diversion does more to end poverty than other uses of that money.”

This morning, more than a week since the paper’s release, the authors of the paper issued a correction saying, “We thank Demombynes and others for pointing out these corrections, and as noted we have withdrawn the erroneous claims as a result.  To improve the performance of the project going forward, we are inviting international public health experts along with critics of the project to join an independent expert committee to review the data collection and help improve its accuracy.”

You've read  of  free articles. Subscribe to continue.

Dear Reader,

About a year ago, I happened upon this statement about the Monitor in the Harvard Business Review – under the charming heading of “do things that don’t interest you”:

“Many things that end up” being meaningful, writes social scientist Joseph Grenny, “have come from conference workshops, articles, or online videos that began as a chore and ended with an insight. My work in Kenya, for example, was heavily influenced by a Christian Science Monitor article I had forced myself to read 10 years earlier. Sometimes, we call things ‘boring’ simply because they lie outside the box we are currently in.”

If you were to come up with a punchline to a joke about the Monitor, that would probably be it. We’re seen as being global, fair, insightful, and perhaps a bit too earnest. We’re the bran muffin of journalism.

But you know what? We change lives. And I’m going to argue that we change lives precisely because we force open that too-small box that most human beings think they live in.

The Monitor is a peculiar little publication that’s hard for the world to figure out. We’re run by a church, but we’re not only for church members and we’re not about converting people. We’re known as being fair even as the world becomes as polarized as at any time since the newspaper’s founding in 1908.

We have a mission beyond circulation, we want to bridge divides. We’re about kicking down the door of thought everywhere and saying, “You are bigger and more capable than you realize. And we can prove it.”

If you’re looking for bran muffin journalism, you can subscribe to the Monitor for $15. You’ll get the Monitor Weekly magazine, the Monitor Daily email, and unlimited access to CSMonitor.com.