The idea of Big Data for Development (or “BD4D” — you saw it here first, I’m coining it update: um, nevermind.) seems to be gaining momentum. The practice of mining large datasets has been around in private business for a while, as large corporations use sales records or other data to better understand customer behavior. I’ve voiced my skepticism about how quickly these practices will transfer over to contexts of low connectivity, computer literacy, automation and organizational investment.

However, even if those barriers and data quality issues are overcome, it sounds like others have serious concerns about the principles and political implications of BD4D. Here are a few links I’ve gathered while brushing up on the subject:

  1. Big data and development: “The second half of the chess board” — Wolfgang Fengler sounds a hopeful note for the potential of BD4D.
  2. Lies, Damned Lies and Big Data — David Hales issues a warning that the rush to use Big Data may be “data rich” but “theory poor” — with scientific as well as ethical and political implications.
  3. Big Data for Development: From Information to Knowledge Societies? — Patrick Meier reviews a recent academic paper and its conceptual framework, but worries about the de-politicizing nature of Big Data Analysis.
  4. Beware the Big Errors of ‘Big Data’ — Not about development specifically, but Nassim Taleb warns about the dangers of cherry-picking: “Big data may mean more information, but it also means more false information.”
  5. Big Data for Development: Challenges & Opportunities — A paper from UN Global Pulse. I’ll confess that I haven’t read it in detail yet. From a quick skim, it’s probably a useful resource. However, it looks like it stops short of discussing the political implications. (h/t Monitoring and Evaluation News)
  1. Thanks for the links! I had seen David Hales’ piece a few days ago, and its nice to see some other articles to add to the reading list!


  2. Thanks for the list! You might also be interested in the analysis and challenge below

    Big data and development organizations: What happens when you move from theory to practice?

    Can big data help development organisations improve their operations?


  3. […] companies and organizations will catch up eventually. A lot of claims have been made for how Big Data will revolutionize analysis. However, it doesn’t seem like there are accepted methodologies […]


  4. This is interesting from a public health background with a little experience nosing around in international health circles. Torturing the data to get the preferred response is often something health researchers hire statisticians to do. I recently read Confessions of an Economic Hitman, and learned that the author John Perkins personally commissioned the invention of Markov modeling as a way of shoehorning available data into a model that can be relatively easily rigged to produce a preconceived result. Markov models are now the gold standard for estimating complex health outcomes in the form of summary statistics (DALYs), partly because there are point-and-click programs for using natural history projections to take rates of preventable disease and estimate long-term rates of premature death or disability that can be attributed to them.

    This is getting silly. Summary statistics on the global burden of disease really shouldn’t be used except for decorative purposes in introductory paragraphs to research papers, to give a token nod to the fact that the scope of one’s health topic of choice is global. But because the available data is scattered and poor in quality, Markov models are needed to smooth the numbers and standardize outcomes so that the research community can look like they’re all on the same page. If readers weren’t so sure looking at summary statistics made them more knowledgeable about the issues, none of this would be necessary. The data certainly doesn’t have the strength to justify drawing conclusions from the numbers. I hesitate to credit anyone claiming other sorts of development numbers of the international variety can do better. It makes me worry about the Open Government Partnership tying up local governments with unproductive data management paperwork when the performance indicators are sure to be politicized and nominally tied to aid awards that are traditionally more about patronage politics than anything else. I hope it doesn’t turn into the sort of mess public schools in the U.S. are dealing with since the Bush-era testing reforms that were supposed to be an innovation to improve accountability.

    I love the idea of transparency and evidence-based decision making, but I’m worried about how easily this sort of thing could go sideways, and hamstring local administrators with busywork from central government bureaucracies that actually get something out of the paper-pushing.


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: