With binary outcomes (i.e., when\u00a0Y<\/em>\u00a0can only be equal to one or zero, such as in questions answered by “Yes” or “No”), people often like to use the percentage of ones and zeroes correctly predicted, and report that as a measure of goodness-of-fit. Kennedy, in his classic econometric intuition-building treatise, argued that this was not a very good measure:<\/p>\n It is tempting to use the percentage of correct predictions as a measure of goodness of fit. This\u00a0temptation should be resisted: a na\u00efve predictor, for example that every y = 1, could do well on\u00a0this criterion. A better measure along these lines is the sum of the fraction of zeros correctly\u00a0predicted plus the fraction of ones correctly predicted, a number which should exceed unity if\u00a0the prediction method is of value. See McIntosh and Dorfman (1992)<\/a>.<\/p><\/blockquote>\n This could use a bit of explanation: Suppose we have\u00a0Y<\/em> = (0, 1, 1, 1, 1, 1, 1, 1), and we have a vector of predicted values of\u00a0Y\u00a0<\/em>be (0, 1, 1, 0, 1, 1, 1, 0). The usual percentage-of-correct-predictions measure would be 0.75, since 75% of observations are correctly predicted, or 6 out of 8. But one can do even better by guessing “all ones.” Indeed, if I were to guess all ones, I’d get 87.5% goodness of fit, or 7 out of 8.<\/p>\n What McIntosh and Dorfman (1992) suggested instead was to add up (i) the fraction of correctly predicted zeros (in my example, 100%) and (ii) the fraction of correctly predicted ones (in my example, 50%). In my example, then, the total McIntosh-Dorfman goodness-of-fit measure would be 1.5 which, by McIntosh and Dorfman criterion standards, would be deemed a good fit, since it exceeds 1.<\/p>\n Now, if your reaction to the above was this:<\/p>\n Consider the following example from a referee report I received on my 2012 World Development<\/em> article about the welfare impacts of participation in contract farming<\/a>.<\/p>\n In that referee report, the reviewer was faulting me for low pseudo R-square measures on a probit, and suggested that I report the percentage of correct predictions. Notwithstanding the fact that that pseudo R-square measures are pretty bad (see Estrella, 1998<\/a> on that point), I responded with the Kennedy quote above, and in the published version of my paper, in table 5, I actually report three measures: the pseudo R-square (0.081), the percentage of correct predictions (0.63), and the Dorfman-McIntosh measure (1.29). Note that although the percentage of correct predictions and the McIntosh-Dorfman measure are consistent with one another (if I assume I predicted 63% of both ones and zeros correctly, I get a McIntosh-Dorfman of 0.126), the pseudo R-square tells me that only 8 percent of the variance in the dependent variable is explained by my left-hand side variable, which does strike me as misleading in this case.<\/p>\n","protected":false},"excerpt":{"rendered":" In econometrics, goodness-of-fit measures tell us what percentage of the variation in a dependent variable is explained by the explanatory variables. If you’ve ever taken<\/p>\n![\"PeterGriffin\"](\"http:\/\/marcfbellemare.com\/wordpress\/wp-content\/uploads\/2014\/02\/PeterGriffin-580x440.jpg\")
<\/a><\/p>\n