The Relationship Between Response Probabilities and Data Quality in Grid Questions

  • Tobias Gummer GESIS - Leibniz-Institute for the Social Sciences
  • Ruben Bach University of Mannheim
  • Jessica Daikeler GESIS - Leibniz Institute for the Social Sciences
  • Stephanie Eckman RTI International
Keywords: response propensity, measurement error, data quality, panel survey, adaptive survey design

Abstract

Response probabilities are used in adaptive and responsive survey designs to guide data collection efforts, often with the goal of diversifying the sample composition. However, if response probabilities are also correlated with measurement error, this approach could introduce bias into survey data. This study analyzes the relationship between response probabilities and data quality in grid questions. Drawing on data from the probability-based GESIS panel, we found low propensity cases to more frequently produce item nonresponse and nondifferentiated answers than high propensity cases. However, this effect was observed only among long-time respondents, not among those who joined more recently. We caution that using adaptive or responsive techniques may increase measurement error while reducing the risk of nonresponse bias.
Published
2021-04-10
How to Cite
Gummer, T., Bach, R., Daikeler, J., & Eckman, S. (2021). The Relationship Between Response Probabilities and Data Quality in Grid Questions. Survey Research Methods, 15(1), 65-77. https://doi.org/10.18148/srm/2021.v15i1.7727
Section
Articles