{"id":11364,"date":"2015-10-05T05:00:45","date_gmt":"2015-10-05T09:00:45","guid":{"rendered":"http:\/\/marcfbellemare.com\/wordpress\/?p=11364"},"modified":"2021-11-15T15:27:50","modified_gmt":"2021-11-15T21:27:50","slug":"metrics-monday-friends-do-let-friends-do-iv","status":"publish","type":"post","link":"https:\/\/marcfbellemare.com\/wordpress\/11364","title":{"rendered":"‘Metrics Monday: Friends *Do* Let Friends Do IV"},"content":{"rendered":"

An excerpt from a post<\/a> over at the interestingly named Kids Prefer Cheese:<\/p>\n

Just don’t do [IV] …<\/p>\n

Here are the problems.<\/div>\n
<\/div>\n
First of all, no matter what you may have read or been taught, identification is always and everywhere an ASSUMPTION. You cannot prove your IV is valid. …<\/div>\n

I pretty much refuse to let my grad students go on the market with an IV in the job market paper. No way, no how. Even the 80 year old deadwoods in the back of the seminar room at your job talk know how to argue about the validity of your instruments. It’s one of the easiest ways to lose control of your seminar.<\/p><\/blockquote>\n

As we say in Minnesota: That’s different.\u00a0Two things:<\/p>\n

    \n
  1. What about those cases where you have an IV that is randomly assigned? For example, what about those cases where you offer people randomly varying cash incentives to take up some kind of treatment in order to estimate the LATE? Here, there is no assumption (or is that ASSUMPTION?)\u00a0made, beyond\u00a0the\u00a0assumption\u00a0that the significance of your IV in your first-stage regression\u00a0(i.e., its relevance) is not the result of chance.<\/li>\n
  2. Honestly, if we are going to go there–that is, question weak IV tests\/tests of relevance because a rejection of the null might be due to chance, and indict IV for\u00a0that–then can I point out that\u00a0it is also<\/em> an assumption\u00a0that your failure to reject the null in test of\u00a0parallel trends in a diff-in-diffs context is not due to chance? (That is, when the parallel trends assumption is actually<\/em> testable.)<\/li>\n<\/ol>\n

    And then there’s the following\u00a0(the emphasis is mine):<\/p>\n

    We’ve had really good luck placing students who used Diff in diff (in diff), propensity score matching<\/strong>, synthetic control, and even regression discontinuity. All of these approaches have their own problems, but they are like little grains of sand compared to the boulder-sized issues in IV.<\/p><\/blockquote>\n

    Huh?\u00a0I have seen propensity score matching applied\u00a0wrongly way more often than\u00a0I have seen poor IVs or IVs whose exclusion restrictions were\u00a0not really explored and discussed. I have lost count of the number of papers I have read where the authors think that matching on observables also\u00a0implies matching on unobservables, and that one\u00a0can just use PSM\u00a0to buy\u00a0causal identification on the cheap<\/a>.<\/p>\n

    Don’t get me wrong: If you are going to use an observational IV, you do<\/em> need to think very carefully about\u00a0how and why it meets the exclusion restriction. And if\u00a0it does meet it, you need to pray that it will be a relevant IV. But there are clear cases where IV works, and that is especially the case in a setting where you randomly assign\u00a0the IV, or in quasi experimental settings where people are assigned to some treatment at random (e.g., Angrist’s famous Vietnam draft lottery setting).<\/p>\n

    Identifying\u00a0causal effects is hard. Willingly limiting yourself to a subset of methods and declaring\u00a0one method off-limits is\u00a0like a football\u00a0coach saying he doesn’t want his quarterback to ever try to pass the ball.<\/p>\n","protected":false},"excerpt":{"rendered":"

    An excerpt from a post over at the interestingly named Kids Prefer Cheese: Just don’t do [IV] … Here are the problems. First of all,<\/p>\n

    Continue reading‘Metrics Monday: Friends *Do* Let Friends Do IV<\/span><\/a><\/div>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-11364","post","type-post","status-publish","format-standard","hentry","category-uncategorized","entry"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p1gPg8-2Xi","_links":{"self":[{"href":"https:\/\/marcfbellemare.com\/wordpress\/wp-json\/wp\/v2\/posts\/11364","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/marcfbellemare.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/marcfbellemare.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/marcfbellemare.com\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/marcfbellemare.com\/wordpress\/wp-json\/wp\/v2\/comments?post=11364"}],"version-history":[{"count":17,"href":"https:\/\/marcfbellemare.com\/wordpress\/wp-json\/wp\/v2\/posts\/11364\/revisions"}],"predecessor-version":[{"id":13920,"href":"https:\/\/marcfbellemare.com\/wordpress\/wp-json\/wp\/v2\/posts\/11364\/revisions\/13920"}],"wp:attachment":[{"href":"https:\/\/marcfbellemare.com\/wordpress\/wp-json\/wp\/v2\/media?parent=11364"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/marcfbellemare.com\/wordpress\/wp-json\/wp\/v2\/categories?post=11364"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/marcfbellemare.com\/wordpress\/wp-json\/wp\/v2\/tags?post=11364"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}