Skip to content

Category: Social Sciences

Taubes on the Weakness of Observational Studies, and a Methodological Rant

One caveat is observational studies, where you identify a large cohort of people – say 80,000 people like in the Nurse’s Health Study – and you ask them what they eat. You give them diet and food frequency questionnaires that are almost impossible to fill out and you follow them for 20 years. If you look and see who is healthier, you’ll find out that people who were mostly vegetarians tend to live longer and have less cancer and diabetes than people who get most of their fat and protein from animal products. The assumption by the researchers is that this is causal – that the only difference between mostly vegetarians and mostly meat-eaters is how many vegetables and how much meat they eat.

I’ve argued that this assumption is naïve almost beyond belief. In this case, vegetarians or mostly vegetarian people are more health conscious. That’s why they’ve chosen to eat like this. They’re better educated than the mostly meat-eaters, they’re in a higher socioeconomic bracket, they have better doctors, they have better medical advice, they engage in other health conscious activities like walking, they smoke less. There’s a whole slew of things that goes with vegetarianism and leaning towards a vegetarian diet. You can’t use these observational studies to imply cause and effect. To me, it’s one of the most extreme examples of bad science in the nutrition field.

That’s Gary Taubes in a FiveBooks interview over at The Browser. Taubes is better known for his book Good Calories, Bad Calories, in which he argues that a diet rich in carbohydrates is what makes us fat and, eventually, sick, and in which he argues in favor of an alternative diet rich in fats.

I really don’t know what kind of diet is best for weight loss, but I do want to stress Taubes’ point about the weakness of observational studies, even longitudinal ones. It is not uncommon for social science researchers to say “Well, we’ve been following these people over time, so we can use fixed effects to control for unobserved heterogeneity.” That is, they control for what remains constant for each unit of observation over time, which is made possible because they have more than one observation for each unit of observation. I have certainly been guilty of that.

Linear Regression and Causality for Neophytes

(This is an update on a post I had initially written at the start of the academic year. I figured it would come in handy, as many of us are busy writing our syllabi for the spring semester.)

If you teach in a policy school, a political science department, or in an economics department that grants Bachelor of Arts instead of Bachelor of Science degrees, chances are some of your students are not quite conversant in the quantitative methods used in the social sciences.

Many of the students who sign up for my fall seminar on the Microeconomics of International Development Policy or for my spring seminar on Law, Economics and Organization, for example, are incredibly bright, but they are not familiar with regression analysis, and so they don’t know how to read a regression table.

This makes it difficult to assign empirical papers in World Development for in-class discussion, let alone empirical papers in the Journal of Development Economics.

While I do not have the time to teach basic econometrics to students in those seminars, I have prepared two handouts for them to read in preparation for reading papers containing empirical results, which I thought I should make available to anyone who would rather not spend precious class time teaching the basics of quantitative methods. I have used both these handouts in my development seminar last fall, and my students said that they had learned quite a bit from reading them.

The first handout is a primer on linear regression, which shows analytically and graphically (and hopefully painlessly) what a regression does, and why it is such a useful tool in the social sciences. Perhaps more importantly, this handout also explains how to read a regression table.

The second handout primer on the identification of causal relationships in the social sciences, which discusses the distinction between correlation and causation and explains two ways in which social scientists go about making causal statements (i.e., randomized controlled trials and instrumental variables), with a few examples. I suggest supplementing this handout with a reading of Jim Manzi’s “What Social Science Does–and Doesn’t–Know” in City Journal as well as with Esther Duflo’s TED talk.

Of course, neither handout is a substitute for a course in econometrics or on research design, respectively, but these handouts are intended primarily for undergraduates or Masters students with little to no quantitative background.

(Update: This post by Tom Pepinsky also offers a very good introduction to the identification of causal relationships. HT to Chris Blattman for this great find.)

What Grinds My Gears: “Organic Can Feed the World”

In a post over at the Atlantic, Barry Estabrook begins as follows:

Given that current production systems leave nearly one billion people undernourished, the onus should be on the agribusiness industry to prove its model, not the other way around.

Let’s ask ourselves whether organic agriculture can feed the world, shall we? “The way I see it, Barry, this should be a very dynamite show!”

Well Barry, it turns out the agribusiness industry has already proven its model: It has survived the market test for several decades.

If organic is so much better, why is it that the most democratic of all institutions — the market — is not allowing it to win out? Could it be that it’s because organic is more expensive?

(Update: Johanna, a reader, made an excellent point about agricultural subsidies in the comments, which has made me change my mind about the viability of “conventional” agriculture relative to organic if we were to get rid of agricultural subsidies.)

And another thing: the one billion people that go undernourished? Their plight is the result of lack of storage and transportation infrastructures, which both add significant transaction costs to the market price of food and leave many people out of the market altogether, and not because of a lack of food to go around.

Even if we could magically motivate donors to fund storage and transportation infrastructure (because let’s face it Barry, is there anything sexier for donors than to invest in refrigeration technology or roads?) is more expensive food really the answer to chronic undernourishment?