The Influence of the Design of Web Survey Questionnaires on the Quality of Responses

Abstract

he first objective of this article is to propose a conceptual framework of the effects of on-line questionnaire design on the quality of collected responses. Secondly, we present the results of an experiment where different protocols have been tested and compared in a randomised design using the basis of several quality indexes. Starting from some previous categorizations, and from the main factors identified in the literature, we first propose an initial global framework of the questionnaire and question characteristics in a web survey, divided into five groups of factors. Our framework was built to follow the response process successive stages of the contact between the respondent and the questionnaire itself. Then, because it has been studied in the survey methodology literature in a very restricted way, the concept of `response quality' is discussed and extended with some more `qualitative' criteria that could be helpful for researchers and practitioners, in order to obtain a deeper assessment of the survey output. As an experiment, on the basis of the factors chosen as major characteristics of the questionnaire design, eight versions of a questionnaire related to young people's consumption patterns were created. The links to these on-line questionnaires were sent in November 2005 to a target of 10,000 young people. The article finally presents the results of our study and discusses the conclusions. Very interesting results come to light; especially regarding the influence of length, interaction and question wording dimensions on response quality. We discuss the effects of Web-questionnaire design characteristics on the quality of data.