Skip to content

Category: Methods

Published May 7, 2012

Implementation Bias in Randomized Controlled Trials

From a new paper (link opens a .pdf file) by Oxford’s Tessa Bold and her coauthors:

The recent wave of randomized trials in development economics has provoked criticisms regarding external validity and the neglect of political economy. We investigate these concerns in a randomized trial designed to assess the prospects for scaling-up a contract teacher intervention in Kenya, previously shown to raise test scores for primary students in Western Kenya and various locations in India. The intervention was implemented in parallel in all eight Kenyan provinces by a nongovernmental organization (NGO) and the Kenyan government. Institutional differences had large e ffects on contract teacher performance. We find a signifi cant, positive effect of 0.19 standard deviations on math and English scores in schools randomly assigned to NGO implementation, and zero effect in schools receiving contract teachers from the Ministry of Education. We discuss political economy factors underlying this disparity, and suggest the need for future work on scaling up proven interventions to work within public sector institutions.

Bold et al.’s finding points to an important problem with the findings of many randomized controlled trials (RCTs): No matter how careful one is in ensuring that subjects are randomly assigned to the treatment and control groups, almost all RCTs rely on only one implementing partner.

Continue readingImplementation Bias in Randomized Controlled Trials
Published April 24, 2012

Identifying Causal Relationships vs. Ruling Out All Other Possible Causes

Portrait of Artistotle (Source: Wikimedia Commons.)

I was in Washington last month to discuss my work on food prices, in which I look at whether food prices cause social unrest, at an event whose goal was to discuss the link between climate change and conflict.

As many readers of this blog know, disentangling causal relationships from mere correlations is the goal of modern science, social or otherwise, and though it is easy to test whether two variables x and y are correlated, it is much more difficult to determine whether x causes y.

So while it is easy to test whether increases in the level of food prices are correlated with episodes of social unrest, it is much more difficult to determine whether food prices cause social unrest.

In my work, I try to do so by conditioning food prices on natural disasters. To make a long story short, if you believe that natural disasters only affect social unrest through food prices, this ensures that if there is a relationship between food prices and social unrest, that relationship is cleaned out of whatever variation which is not purely due to the relationship flowing from food prices to social unrest. In other words, this ensures that the estimated relationship between the two variables is causal. This technique is known as instrumental variables estimation.

Identifying Causal Relationships vs. Ruling Out All Other Causes

As with almost any other discussion of a social-scientific issue nowadays, the issue of causality came up during one of the discussions we had at that event in Washington. It was at that point that someone implied that it did not make sense to talk of causality by bringing up the following analogy:

Continue readingIdentifying Causal Relationships vs. Ruling Out All Other Possible Causes
Published March 19, 2012

Slides of My Keynote Lecture at Last Weekend’s “Economics and Management of Risk in Agriculture and Natural Resources” Conference

I was trained as an agricultural and applied economist, so I have spent a lot of time doing research on risk as it relates to agriculture and development (see here and here for published articles).

Because of this, I have been involved with the annual Economics and Management of Risk in Agriculture and Natural Resources conference for the past few years.

I first presented at that conference in 2009, and since I had then volunteered to organize the conference, I was in charge of the conference program in 2010 and of logistics in 2011.

This year, I was asked to give the keynote lecture, in which I chose to discuss what the “credibility revolution” that took place in economics over the past ten years or so — which has lead to economists to adopting stricter standards of evidence and of statistical identification — means for agricultural and applied economics as a field.

In case you have an interest in this topic, I am making the slides of my keynote lecture are available. I think the content of those slides is especially relevant for current graduate students of agricultural and applied economics.

The Economics and Management of Risk in Agriculture and Natural Resources conference is usually held somewhere on the Gulf Coast. This year, it was held in Pensacola, FL. I took the picture on top of this post while walking along the beach early Saturday morning.