Case Prioritization in a Panel Survey Based on Predicting Hard to Survey Households by Machine Learning Algorithms: An Experimental Study

Abstract

Panel surveys provide particularly rich data for implementing adaptive or responsive survey designs. Paradata and survey data as well as interviewer observations from all previous waves can be utilized to predict fieldwork outcomes in an ongoing wave. This manuscript contributes to the literature on how to best make use of these data in an adaptive design framework applying machine learning algorithms. In a first step, different models were trained based on past panel waves. In a second step, we assess which model best predicts fieldwork outcomes of the following wave. Finally, we apply the superior model to predict response propensities and base case prioritizations of households at risk of attrition on these predictions. An experimental design allows us to evaluate the effect of these prioritizations on response rates and on nonresponse bias. Increasing prepaid respondent incentives from 10 to 20 euros substantially decreases attrition of low propensity cases in personal as well as telephone interviews and thereby helps reduce nonresponse bias in important target variables of the panel survey.