Evaluating the impact of response enhancement methods on the risk of nonresponse bias and survey costs

  • Caroline Roberts University of Lausanne
  • Caroline Vandenplas University of Lausanne
  • Michèle Ernst Stähli FORS, Swiss Centre of Expertise in the Social Sciences
Keywords: R-indicator, European Social Survey, response rates, coefficient of variation

Abstract

The pursuit of high response rates to minimise the threat of nonresponse bias continues to dominate decisions about resource allocation in survey research. Yet a growing body of research has begun to question this practice. In this study, we use previously unavailable data from a new sampling frame based on population registers to assess the value of different methods designed to increase response rates on the European Social Survey in Switzerland. Using sampling data provides information about both respondents and nonrespondents, making it possible to examine how changes in response rates resulting from the use of different fieldwork methods relate to changes in the composition and representativeness of the responding sample. We compute an R-indicator to assess representativity with respect to the sampling register variables, and find little improvement in the sample composition as response rates increase. We then examine the impact of response rate increases on the risk of nonresponse bias based on Maximal Absolute Bias (MAB), and coefficients of variation between subgroup response rates, alongside the associated costs of different types of fieldwork effort. The results show that increases in response rate help to reduce MAB, while only small but important improvements to sample representativity are gained by varying the type of effort. These findings lend further support to research that has called into question the value of extensive investment in procedures aimed at reaching response rate targets and the need for more tailored fieldwork strategies aimed both at reducing survey costs and minimising the risk of bias.

Author Biographies

Caroline Roberts, University of Lausanne
Caroline Roberts is Senior Researcher in Survey Methodology in the Institute of Social Sciences at the University of Lausanne, and at FORS, the Swiss Centre of Expertise in the Social Sciences.
Caroline Vandenplas, University of Lausanne
Caroline Vandenplas is Postdoctoral Research Fellow in Survey Methodology in the Institute of Social Sciences at the University of Lausanne, and at FORS, the Swiss Centre of Expertise in the Social Sciences.
Michèle Ernst Stähli, FORS, Swiss Centre of Expertise in the Social Sciences
Michèle Ernst Stähli is Senior Researcher and Head of International Surveys at FORS, the Swiss Centre of Expertise in the Social Sciences.
Published
2014-08-10
Section
Articles