Mariah Ehmke and Jason Shogren have a paper in the in the latest issue of Applied Economic Perspectives and Policy which serves a companion piece and a rejoinder to the Barrett and Carter paper I blogged about earlier this week.
Here is the abstract:
“Recent work with Randomized Controlled Trials (RCTs) in development economics has contributed to economists’ use of the experimental mindset to inform policy choices. Development scholars, however, question the authority of RCT evidence, and worry that the RCT trend will turn their profession away from theory and econometrics. We examine this challenge, as well as RCTs’ role within the broader experimental area, with a thorough review of relevant literature. We find that generic RCT fears are overstated. Experimental methods should be evaluated as a tool to test theory, search for patterns, and to pre-test new institutions. From this mindset, we see unexplored pathways that may benefit from the experimental mindset, further economic theory, and reduce poverty.”
I am not sure that development economists question the authority of RCT evidence, which is usually the most convincing. What is being questioned is the fact that the research agenda often seems to be suffering from a streetlight effect.
In other words, some fear that the only research questions we choose to answer will be the ones we know we can answer, i.e., those that are “randomizable.”
My sense is that the identification of causal relationships — which comes from good research design and does not necessarily require an RCT or any fancy statistical techniques — is indeed crucial when trying to answer a well-worn research question (e.g., “What is the impact of participation in agricultural value chains on smallholder welfare?”) There will always be a market, however, for highly original research questions relying on less-than-ideal identification.