Attention Check Items and Instructions in Online Surveys with Incentivized and Non-Incentivized Samples: Boon or Bane for Data Quality?

Authors

DOI:

https://doi.org/10.18148/srm/2020.v14i1.7374

Keywords:

careless responding, inattentive responding, satisficing, detection method, measurement quality, data quality, online survey

Abstract

In this paper, we examine rates of careless responding and reactions towards detection methods (i.e., attention check item and instruction) in an experimental setting based on two different samples. First, we use a quota sample (with monetary incentive), a central data source for internet-based surveys in sociological and political research. Second, we include a voluntary opt-in panel (without monetary incentive) well suited to conduct survey experiments (e.g., factorial surveys). Respondents’ reactions towards the detection items are analyzed by objective, non-reactive indicators (i.e., break-off, item nonresponse, and measurement quality), and two self-report scales. Our reaction analyses reveal that the detection methods that we applied are not only well suited to identify careless respondents but also exert a motivational rather than demotivating influence on respondents’ answer behavior and, hence, contribute to data quality.

Author Biographies

Hawal Shamon, Forschungszentrum Jülich

Post-doctoral researcher at the Institute of Energy and Climate Research Systems Analysis and Technology Evaluation (IEK-STE)

Carl Clemens Berning, Johannes Gutenberg Universität Mainz

Post-doctoral researcher at the Department of Political Science

Downloads

Published

2020-04-10

How to Cite

Shamon, H., & Berning, C. C. (2020). Attention Check Items and Instructions in Online Surveys with Incentivized and Non-Incentivized Samples: Boon or Bane for Data Quality?. Survey Research Methods, 14(1), 55–77. https://doi.org/10.18148/srm/2020.v14i1.7374

Issue

Section

Articles

Similar Articles

<< < 2 3 4 5 6 7 8 9 10 11 > >> 

You may also start an advanced similarity search for this article.