A recent letter to the editor of the New York Times, in response to an article titled “How Single Motherhood Hurts Kids“:
As the product of a single-mother family, I take issue with the broad generalizations in your article. To those single mothers who work diligently to provide good homes for their children, the headline itself is an affront. The assumption is that single motherhood hurts children and this is how that happens.
In fact, this article is really about single, low-income, multipartner mothers, many of whom are teenagers. That is only a segment of the broad range of mothers who are single.
Setting aside the claim that single motherhood hurts kids, which I am in no way qualified to assess, I wanted to make an important point for consumers of social science results with this post. My point consists of two related sub-points, viz.
- The vast majority of findings in the quantitative social sciences are true on average.
- By definition, there is a whole distribution around that average, which means that the effect might be negative for some, zero for others, and positive for still others.
In other words, if I tell you that having a college degree increases your annual income by $30,000 on average, it is likely that the along the distribution of the average treatment effect of going to college on annual income, there are some people for whom the effect is negative — they went to college, but they ended up with a worse job than they would’ve had otherwise gotten had they accumulated four years of experience instead — and there are some people for whom the effect is more on the order of $100,000 per year.
Does the author of the letter to the editor quoted above make the mistake of assuming that average effects are uniformly distributed? Not really, because he recognizes that if you condition on certain things (high-income, adult single-mothers who have few partners), the result is likely to disappear.
And it’s not just the general, non-quantitative lay readers of the New York Times who make the mistake of assuming a uniform distribution of treatment effects. Every once in a while, I will get a comment in a referee report that goes something like “You find that an increase in food prices increases the number of food riots, but there are cases where food gets so expensive that people eat too little and, as a consequence, they are too weak to riot” (this is a fictitious example, by the way). To which your response should be: “Well, yeah.”
The whole point of running regressions, in most cases, is to know what happens on average. To be sure, there is a certain movement toward characterizing heterogeneous treatment effects in applied microeconomics, but doing so is difficult, because causally identifying average effects is difficult enough as is, and causally identifying heterogeneous effects is even more difficult.