Revisiting "yes/no" versus "check all that apply": Results from a mixed modes experiment

  • Gerry Nicolaas Ipsos MORI
  • Pamela Campanelli The Survey Coach
  • Steven Hope University College London
  • Annette Jäckle University of Essex
  • Peter Lynn University of Essex
Keywords: survey methods, mode of data collection, questionnaire design, check all that apply, forced choice

Abstract

The work of Smyth, Dillman, Christian, and Stern (2006) and Smyth, Christian, and Dillman (2008) compares “yes/no” questions to “check all that apply” questions. They conclude that the “yes/no” format is preferable as it reflects deeper processing of survey questions. Smyth et al. (2008) found that the “yes/no” format performed similarly across telephone and web modes. In this paper we replicate their research and extend it by including a comparison with face-to-face in addition to telephone and web and by using probability samples of the general adult population. A cognitive interviewing follow-up was used to explore the quantitative findings. Our results suggest there are times when the “yes/no” format may not perform similarly across modes and that there may be factors which limit the quality of answers.

Author Biographies

Gerry Nicolaas, Ipsos MORI
Head of Data Collection Methodology, Research Methods Centre, Social Research Institute, Ipsos MORI
Pamela Campanelli, The Survey Coach
Independent Survey Methods Trainer and Consultant, The Survey Coach
Steven Hope, University College London
Senior Research Associate, Institute of Child Health, University College London
Annette Jäckle, University of Essex
Senior Research Fellow and Associate Director of Innovations for Understanding Society, Institute for Social and Economic Research, University of Essex
Peter Lynn, University of Essex
Professor of Survey Methodology, Institute for Social and Economic Research, University of Essex
Published
2015-12-04
How to Cite
Nicolaas, G., Campanelli, P., Hope, S., Jäckle, A., & Lynn, P. (2015). Revisiting "yes/no" versus "check all that apply": Results from a mixed modes experiment. Survey Research Methods, 9(3), 189-204. https://doi.org/10.18148/srm/2015.v9i3.6151
Section
Articles