Unobserved heterogeneity, Grouped Random Effects and the EAMP algorithm: Karun Adusumilli (UPenn)

Abstract: We propose Grouped Random Effects as a new approach for nonlinear panel data models with unobservables. This posits that observations can be separated into groups, each having its own prior on unobserved heterogeneity, and with the parameters of the prior unknown and differing across groups. Both random effects and grouped fixed effects are special cases of this. The model can be estimated by jointly maximizing over group assignments, common and prior parameters. We propose a novel and fast algorithm to carry out this maximization, termed EAMP, that augments the standard EM algorithm with two additional steps: Assignment (A) for group assignment and Propagation (P) for updating the prior. We further show that the steps in the EAMP algorithm are closely related to those for mean-field Variational-Bayes inference in Dirichlet mixture models. Advantages of the Grouped Random Effects approach include automatic first order bias correction, need for fewer number of groups, and greater flexibility in modeling unobserved dynamics. We illustrate our methods using two examples: the first studies heterogeneity in income dynamics using PSID data. The second studies heterogeneity in the effect of union status on wages. In the case of income dynamics, we find large heterogeneity in the magnitudes and persistence of income shocks. This heterogeneity can explain the observed non-normality and nonlinear persistence of income shocks.