I had to address the topic of significance–broadly defined–at some point in this series of posts. With that said, this post is not about the seeming arbitrariness of the conventional levels of significance, i.e., of the 10, 5, and 1 percent levels of significance. First, because like Ellickson (1994), I believe that social norms emerge and evolve to minimize transactions costs, and the conventional levels of significance do just that by minimizing the amount of explanation producers of empirical findings have to provide for their findings and by minimizing the amount of thinking the consumers of empirical findings have to do in order to figure out just how (statistically) significant those findings are. In other words, the conventional levels of significance provide something for everyone to hang their hat on, and they are easy to explain to the general public.
And second, because as Noah Smith noted last summer, “[d]issing p-values in 2015 is a little like dissing macroeconomics in 2011–something that gives you a free pass to sound smart in certain circles … But like all hipster fads, I expect this one to fade.”
Rather, this post is about statistical vs. economic significance. Every so often, you run into a paper in which the authors have a good story, a good identification strategy, and robust, statistically significant findings, but in which there is little to no discussion of the findings’ economic significance.