Against All Odds: On the Robustness of Probability Samples Against Decreases in Response Rates

Abstract

Responses rates in surveys with probability samples have decreased in the last decades, buthas this decrease caused a decline in sample quality? Our paper addresses this question withan analysis of methodological data describing 776 surveys from four cross-national surveyprojects: European Quality of Life Survey, European Social Survey, European Values Study,and International Social Survey Programme, between 1999 and 2020. Based on a theoreticalmodel of factors that shape unit nonresponse and unit nonresponse bias, we estimate causal ef-fects of historical time on both, nonresponse and nonresponse bias, as well as the contributionof nonresponse to nonresponse bias. Analyses show that the decline in response rates was notaccompanied by an increase in nonresponse bias, which is a reassuring result for all social surveyusers.