Skip to content

Marc F. Bellemare Posts

Published May 7, 2012

Implementation Bias in Randomized Controlled Trials

From a new paper (link opens a .pdf file) by Oxford’s Tessa Bold and her coauthors:

The recent wave of randomized trials in development economics has provoked criticisms regarding external validity and the neglect of political economy. We investigate these concerns in a randomized trial designed to assess the prospects for scaling-up a contract teacher intervention in Kenya, previously shown to raise test scores for primary students in Western Kenya and various locations in India. The intervention was implemented in parallel in all eight Kenyan provinces by a nongovernmental organization (NGO) and the Kenyan government. Institutional differences had large e ffects on contract teacher performance. We find a signifi cant, positive effect of 0.19 standard deviations on math and English scores in schools randomly assigned to NGO implementation, and zero effect in schools receiving contract teachers from the Ministry of Education. We discuss political economy factors underlying this disparity, and suggest the need for future work on scaling up proven interventions to work within public sector institutions.

Bold et al.’s finding points to an important problem with the findings of many randomized controlled trials (RCTs): No matter how careful one is in ensuring that subjects are randomly assigned to the treatment and control groups, almost all RCTs rely on only one implementing partner.

Continue readingImplementation Bias in Randomized Controlled Trials
Published April 24, 2012

Identifying Causal Relationships vs. Ruling Out All Other Possible Causes

Portrait of Artistotle (Source: Wikimedia Commons.)

I was in Washington last month to discuss my work on food prices, in which I look at whether food prices cause social unrest, at an event whose goal was to discuss the link between climate change and conflict.

As many readers of this blog know, disentangling causal relationships from mere correlations is the goal of modern science, social or otherwise, and though it is easy to test whether two variables x and y are correlated, it is much more difficult to determine whether x causes y.

So while it is easy to test whether increases in the level of food prices are correlated with episodes of social unrest, it is much more difficult to determine whether food prices cause social unrest.

In my work, I try to do so by conditioning food prices on natural disasters. To make a long story short, if you believe that natural disasters only affect social unrest through food prices, this ensures that if there is a relationship between food prices and social unrest, that relationship is cleaned out of whatever variation which is not purely due to the relationship flowing from food prices to social unrest. In other words, this ensures that the estimated relationship between the two variables is causal. This technique is known as instrumental variables estimation.

Identifying Causal Relationships vs. Ruling Out All Other Causes

As with almost any other discussion of a social-scientific issue nowadays, the issue of causality came up during one of the discussions we had at that event in Washington. It was at that point that someone implied that it did not make sense to talk of causality by bringing up the following analogy:

Continue readingIdentifying Causal Relationships vs. Ruling Out All Other Possible Causes
Published April 18, 2012

Land and Politics

From a new article by Jean-Marie Baland and Jim Robinson in the American Journal of Political Science:

In this article, we argue that when patron-client relations are grounded in economic relationships, such as between landlord and worker, we should expect clientelism to influence not just how public policy, the state, and the political system work, but also how the economy works. We develop a simple model of the economic consequences of electoral clientelism when voting behavior can be observed. Landlords/patrons provide economic rents to workers, and in exchange workers vote for parties favored by landlords. As votes are used by the landlords to accumulate political rents, vote control increases the demand for labor and for land. The model implies that the introduction of the Australian ballot, which destroys this form of clientelism, should lead to a fall in the price of land in those areas where patron-client relationships are strongest. We test the predictions of the model by examining in detail the evolution of land prices in Chile around May 31, 1958, for which we collected original data. A characteristic of rural Chile at this time were patron-client relations based on the inquilinaje system, by which a worker, the inquilino, entered into a long-term, often hereditary, employment relationship with a landlord and lived on his landlord’s estate. We show that the introduction of the Australian ballot in 1958 led to a fall of about 26% in land prices in the areas where these patron-client relationships were predominant.

This article is a followup of sorts to Baland and Robinson’s 2010 American Economic Review article, in which they show that the introduction of the secret ballot in 1958 in Chile led to greater support for labor-friendly political parties.