Last updated on November 9, 2014
Dave Giles, whose Econometrics Beat is one of my favorite economics blogs, is not fond of what he calls the “cookbook” approach to econometrics:
I’ll lay it on the table – I am definitely not a fan of “Cookbook Econometrics.”
Here’s what I’m referring to.
It’s pointless, and frankly dangerous, to simply tell students what to do, without telling them why. And I know that this applies to more than just econometrics. When I’m explaining to students why we go through the proofs of important results, I usually make the following points.
First, depending on the nature and level of the course, I may or may not expect them to be able to reproduce the proof in a test or exam – usually, that’s the least of my concerns. Second, the real benefit in being led through the proof of a standard result in econometrics is that enables you to see exactly where (and how) any underlying assumptions are actually used.
When framed that way, it is certainly difficult to argue that students should be taught the cookbook approach to econometrics. But after teaching a half-semester “cookbook econometrics” course, I think it is considerably more nuanced than Dave explains it in the post quoted above (and in this other post of his).
In Defense of the Cookbook Approach
While I agree that students should be taught both the “what” and the “why” of econometrics, my “cookbook” econometrics class is for students who have already taken two semesters of PhD-level econometrics. As such, it builds on a year-long sequence of graduate econometrics training aimed at teaching our students the “why.” In other words, my class is taught in view of applying the theory taught in the first year.
The problem is that there is often a wide gap between the theory and practice of econometrics. This is especially so given that the practice of econometrics is often the result of social norms, and because not following these norms often leads to referees viewing your work with suspicion.
Here’s a simple example. We are all taught that we can test for exogeneity (or alternatively, that we can test which of fixed or random effects to use) using a Hausman test. The Hausman test tests the null hypothesis that your variable of interest is exogenous against the alternative hypothesis that it is endogenous (alternatively, it tests the null hypothesis that random effects is the right approach against the alternative hypothesis that fixed effects is the right approach).
A Simple Example
It is typically the case, however, that our instructors have little time to go into the details of what people do in practice. This is typically because there is a whole lot of material to cover in those first-year graduate econometrics classes. So imagine a student who has not taken a “cookbook” econometrics class writing a paper where they deal with panel data. Their variable of interest happens not to vary within units of observations.
For example, imagine they have longitudinal data on a sample of individuals, and they are interested in the effect of gender on wage; gender does not typically vary over the course of one’s lifetime, so with individual fixed effects, the effect of gender is swallowed up by the individual fixed effect.
Luckily, our student has learned how to perform a Hausman test. So they perform a Hausman test and, joy of joys, the test says that random effects should be used. Our student merrily estimates her wage equation with random effects, writes up her paper, and submits it to a journal. Four six to twelve months later, she receives a rejection letter, and two nasty referee reports. One of them, in a momentarily lapse of nastiness, turns this into a teachable moment: “No one uses random effects with observational data anymore. Unless you randomly assigned your variable of interest, stick to fixed effects.”
A Complement–Not a Substitute
If only the student in the example above had taken a cookbook econometrics class! This is especially important given the social norms that are involved in the practice of econometrics: what to include, what to exclude, what to show, what to emphasize, when to include an appendix and what to include in it, etc.
Because our students tend to read only the best papers in their field courses (e.g., development or labor economics), it is often difficult for them to tell what differentiates good papers from average or bad papers (this is best learned through refereeing). Moreover, because students tend to read only the best published papers in their field courses, they don’t get to see the version that was submitted by the authors and which managed to convince reviewers. What they see instead is the “perfect” version, the one which was published after the editor said: “You don’t really need to include all those robustness checks or those alternative specifications, I’m convinced that your result holds.” Well, yeah, convinced because those robustness checks and alternative specifications were included to begin with.
In short, what I am saying is that cookbook econometrics should be a complement to rigorous, basic first-year PhD econometrics rather than a substitute for it. Even the best textbooks (say, those that will occasionally tell you what practitioners do; Wooldridge’s text is great for that) only offer a snapshot of the social norms people follow when doing applied work at a point in time. Those norms evolved all the time, and our graduate students have enough to worry about when they read articles and working papers that it might be difficult for them to learn those norms when they focus on other things. Thus the need for a cookbook econometrics primer.