I was sitting in my office on Friday afternoon when one of our third-year PhD students dropped by with an applied econometric question: “When should I use weights?”
After telling her to go read Solon et al.’s 2015 piece in the JHR symposium on empirical methods, I decided to reread that paper for myself and blog about it this week. In the near future, in part II, I’m hoping to tackle Andrews and Oster’s new NBERÂ working paper on weighting for external validity.
Before I begin, some clarification: throughout this post, I’ll be discussing the use of sampling weights. If you are a Stata user, this refers to that statistical package’s -pweight-, i.e., “weights that denote the inverse of the probability that the observation is included because of the sampling design.” I have never had to rely on -aweight-, -fweight-, or -iweight-, so I wouldn’t know when to use them.
Suppose you oversample a specific group in order to get more precise estimates for that group. For instance, suppose you are interested in the opinion of LGBTQ students. If you randomly sample individuals from a given population of students, you may not have enough LGBTQ respondents in your sample, and so whatever descriptive statistics you come up with for that sub-group might be too noisy. Thus, you may wish to over-sample LGBTQ respondents in order to improve precision. What I mean by this is that you would randomly sample respondents from each group–LGBTQ and non-LGBTQ–until you have the right number. So if you target a sample size of n=100 and you’d like 50% respondents from each group, you split the population in two groups (assuming that’s easy to do; in the case of LGBTQ students, it might not be easy to do) and sample from each until each group has 50 observations.