Marc F. Bellemare Posts

I will be co-teaching a course titled Advanced Econometrics: Recent Methods and Issues at the University of Copenhagen from June 22 to 26, 2020 with my colleague Arne Henningsen. Though the link lists Arne as the instructor, I will be teaching the lecture part of the course, and Arne will be teaching the lab part of the course.

If you are interested in taking the course, enrollment is open to students outside of the University of Copenhagen, and as of writing, registration is about $145, which is a bargain (though if you do register, you are obviously responsible for your travel and accommodation costs, but Copenhagen is lovely in June).

The course is not identical to the course Arne and I co-taught in May 2018 at Copenhagen, though it will certainly share some similarities. For starters, because I am teaching the entirety of Morgan and Winship (2015) to our first-year PhD students this semester, there will be a lot of emphasis on the nuts and bolts (i.e., potential outcomes model and directed acyclic graphs) of causal inference. Second, we will also update the material by covering recent papers which have been published since we last taught the course. Third, I will be holding office hours every day, and I will be happy to discuss your research project in detail with you.

Here is the course description:

Since 2015, I have served as co-editor of leading field journals in agricultural economics almost continuously, first at Food Policy (2015-2019), and now at the American Journal of Agricultural Economics (officially since January 1, but I started handling manuscripts at the end of July), where I am slated to serve until the end of 2023.

Both those experiences have been hugely rewarding.

Both those experiences, however, also come with their fair share of frustrations.

Perhaps the worst aspect of editing a journal is dealing with would-be reviewers who are at once always-takers and never-givers (ATNGs)–that is, those academics who just love to submit to a journal but never have time to review for it.

The behavior of ATNGs is all the more annoying when I see bright young academics ask #EconTwitter how many reviews is too many for an assistant professor, because they have done one per week since the start of the calendar year and they feel like they aren’t really in a position to say “No” to journal editors–who may eventually get asked to write external review letters for those same young academics.

The behavior of ATNGs imposes a huge externality on those who (feel like they) cannot say no. Speaking with colleagues who also serve as editors at the AJAE and elsewhere, this behavior is much more common than most people suspect. And lest you think that there is some kind of editor fixed effect at play, the same ATNGs keep coming up when comparing notes with other editors in my area of research.

Recently, I discussed this with one of my mentors, who has extensive experience editing journals. His solution has been to directly tell ATNGs that for the peer-review system to work, people have to review in addition to submit, and that he will not be sending out any of their papers for review until they start reviewing for him. After all, for the system to work, for every paper of yours that gets sent out for review at a journal, you should be prepared to do two or three reviews for that journal.

As such, and because I believe in an equitable division of labor when it comes to peer review, I am seriously thinking of adopting the same policy for those manuscript I handle at the AJAE.

One concern that arose when discussing that policy with others was “Doesn’t it unfairly penalize the coauthors of ATNGs?” There is no doubt it does, but the hope here is that those same coauthors will put pressure on their ATNG coauthors so that those ATNGs become better citizens of the profession.*

All of my time away from home is spent in an academic department or in volunteer organizations, so I am painfully aware of the fact that in the absence of incentives, it’s always the same people who show up to do the work. But as an economist, I also know that when they are available, incentives work: Going back to my mentor, he told me that after he’d initially adopted the aforementioned policy toward the two ATNGs he had to deploy it with, one of them stopped submitting to the journal he edited altogether, and the other started reviewing ipso facto. From a social-welfare perspective, either outcome is better than having to deal with ATNGs.

* Alternatively, “Won’t we be missing out on good manuscripts if we do that?” Again, no doubt, but there is a considerable supply of excellent manuscripts–there are way many more of them than we can ever hope to publish–and I am happy accepting those manuscripts that are sitting right at the margin.

A few weeks ago, I received the galley proofs for my forthcoming paper in the American Journal of Agricultural Economics (AJAE) on price risk. Because the AJAE is just now transitioning from one publisher (Oxford University Press) to another (Wiley), and because I am one of four co-editors of the journal, this was a good occasion to go over some of the journal’s house rules for how papers look like in the journal.

One of the things that struck me as weird in the initial set of galley proofs that I received was that, fit those tables where all three of the usual symbols of statistical significance (i.e., *, **, and *** to denote statistical significance at less than the 1, 5, and 10 percent levels) were not used, the journal’s production team had seen fit to only list those symbols that were actually used in the table.

So for example, if a table reported findings that were significant at the 1 and 5 percent level, but did not report findings that were significant at the 10 percent level, the symbols ** and *** were defined in the table’s notes, but not the symbol *. Similarly, if a table reported a finding that was significant at the 5 percent level, but did not report findings that were significant at the 1 or 10 percent levels, the symbol ** was defined in the table’s notes, but not the symbols *** and *.

Presumably, the journal’s production team did that to save space–however infinitesimally little of it–on each page where a table appeared.

This struck me as counter to good statistical reporting practice: When looking at a table, we are no less interested in the dogs that didn’t bark than we are interested in the dogs that did bark. With table notes that define symbols in the usual way (i.e., defining *, **, and *** for coefficients significant at the 10, 5, and 1 percent levels), a coefficient without any stars next to it is understood not to be significant at any of those levels.

With a table only defines * and **, a busy reader (or a reader who is not as well-verse in statistics as most of the readers of this blog; say, a policy maker) will have no idea whether any of the coefficients significant at the 5 percent level are significant at the 1 percent level. In practice, the difference between a coefficient that is significant at the 5 percent level or at the 1 percent level can translate into decisions in which a policy maker or manager is respectively “pretty sure” or “almost certain,” and we should strive to be as clear as possible in how we define the results we report.

We have the social norms we have for good reasons. No matter how some people want to get rid of any talk of statistical significance,* the social norm scholars have settled on when reporting statistical results is to talk of the three usual levels of statistical significance. Defining only those symbols that appear in a table to save a small amount of journal page space can be misleading regarding what the authors chose to report, and it should be opposed whenever possible.

* I encourage those readers to read Ellickson’s Order without Law or his 1989 JLEO article for a good explanation of why we have the social norms that we have–and why the majority of those norms are not going away.