Last updated on November 15, 2021
An excerpt from a post over at the interestingly named Kids Prefer Cheese:
Just don’t do [IV] …
Here are the problems.First of all, no matter what you may have read or been taught, identification is always and everywhere an ASSUMPTION. You cannot prove your IV is valid. …I pretty much refuse to let my grad students go on the market with an IV in the job market paper. No way, no how. Even the 80 year old deadwoods in the back of the seminar room at your job talk know how to argue about the validity of your instruments. It’s one of the easiest ways to lose control of your seminar.
As we say in Minnesota: That’s different. Two things:
- What about those cases where you have an IV that is randomly assigned? For example, what about those cases where you offer people randomly varying cash incentives to take up some kind of treatment in order to estimate the LATE? Here, there is no assumption (or is that ASSUMPTION?) made, beyond the assumption that the significance of your IV in your first-stage regression (i.e., its relevance) is not the result of chance.
- Honestly, if we are going to go there–that is, question weak IV tests/tests of relevance because a rejection of the null might be due to chance, and indict IV for that–then can I point out that it is also an assumption that your failure to reject the null in test of parallel trends in a diff-in-diffs context is not due to chance? (That is, when the parallel trends assumption is actually testable.)
And then there’s the following (the emphasis is mine):
We’ve had really good luck placing students who used Diff in diff (in diff), propensity score matching, synthetic control, and even regression discontinuity. All of these approaches have their own problems, but they are like little grains of sand compared to the boulder-sized issues in IV.
Huh? I have seen propensity score matching applied wrongly way more often than I have seen poor IVs or IVs whose exclusion restrictions were not really explored and discussed. I have lost count of the number of papers I have read where the authors think that matching on observables also implies matching on unobservables, and that one can just use PSM to buy causal identification on the cheap.
Don’t get me wrong: If you are going to use an observational IV, you do need to think very carefully about how and why it meets the exclusion restriction. And if it does meet it, you need to pray that it will be a relevant IV. But there are clear cases where IV works, and that is especially the case in a setting where you randomly assign the IV, or in quasi experimental settings where people are assigned to some treatment at random (e.g., Angrist’s famous Vietnam draft lottery setting).
Identifying causal effects is hard. Willingly limiting yourself to a subset of methods and declaring one method off-limits is like a football coach saying he doesn’t want his quarterback to ever try to pass the ball.