Against All Odds: On the Robustness of Probability Samples Against Decreases in Response Rates

Abstract

Responses rates in surveys with probability samples have decreased in the last decades, buthas this decrease caused a decline in sample quality? Our paper addresses this question withan analysis of methodological data describing 776 surveys from four cross-national surveyprojects: European Quality of Life Survey, European Social Survey, European Values Study,and International Social Survey Programme, between 1999 and 2020. Based on a theoreticalmodel of factors that shape unit nonresponse and unit nonresponse bias, we estimate causal ef-fects of historical time on both, nonresponse and nonresponse bias, as well as the contributionof nonresponse to nonresponse bias. Analyses show that the decline in response rates was notaccompanied by a drop in nonresponse bias, which is a reassuring result for all social surveyusers.