Essay: Sunday shopping - The case of three surveys

Abstract

There is a growing discussion about the use of non-probability sampling in survey research. Probability sampling is the preferred method of sample selection, but practical problems like reduced data collection budgets, increasing nonresponse rates, and lack of adequate sampling frames force researchers to use different sampling methods. Particularly, online surveys based on self-selection of respondents have become very popular. Some say that use of such alternative sampling methods is not without risks as often proper inference from sample to population is not possible. Others say that non-probability sampling can produce satisfactory estimates provided effective correction techniques are applied. To obtain more insight in various sample selection methods, it would be nice to be able to compare them in practical situations. This paper describes a case in which three different surveys were carried out on the same topic, at the same time, and with the same questionnaire, but with different sample selection methods: an online panel based on probability sampling, an online survey based on self-selection, and a face-to-face survey in shopping centers. The results of these three polls differ substantially. This is a warning to be careful when choosing a sample selection method.