An Evaluation of the quality of interviewer and virtual observations and their value for nonresponse bias reduction

Abstract

With the decline of survey response rates over the past decade, survey researchers need to gather useful auxiliary variables for all sampled units and reduce nonresponse bias through adaptive design or nonresponse weighting adjustments. One potential source of auxiliary information is interviewer observations of characteristics of sampled units. Compared with area-level characteristics, which researchers often have available for reducing bias due to survey nonresponse, characteristics at the dwelling unit level may provide more information about survey variables of interest and result in weight adjustments that could potentially reduce bias further. These observations, however, may vary greatly among observers, and may lack the quality needed for survey data producers. To investigate the quality and usefulness of such observations, this study assesses completeness, validity, interviewer variance, and predictive power for bias reduction in a national pilot study for both in-person interviewer observations and virtual observations. This paper sheds light on the dwelling unit characteristics that are harder to observe, differences among interviewer and virtual observations, the potential value added beyond area-level characteristics for nonresponse adjustments, and ways to improve the observations.