Can we assess representativeness of cross-national surveys using the education variable?

Abstract

Achieving a representative sample is an important goal for every survey. High response rates are often referred to as an indicator of representativeness in survey methodology research. However, a low response rate does not necessarily imply low representativeness, so that alternative ways of assessing representativeness are needed in times where low response rates are almost ubiquitous. This study asks whether education, a socio-demographic variable covered by virtually every survey of individuals, is a good variable for assessing the representativeness of a realised survey sample. We examine this issue in two steps: Firstly, the distributions of the harmonised education variable in six official and academic cross-national surveys by country-year combination are compared with the respective education distributions in a high-quality reference dataset. Doing so, we identify many substantial inconsistencies. Secondly, we try to identify the sources of these inconsistencies, looking at both measurement errors in the education variables and errors of representation. Since in most instances, inconsistent measurement procedures can probably explain the observed inconsistencies, we conclude that the education variable as currently measured in cross-national surveys is, without further processing, unsuitable for assessing sample representativeness, and constructing nonresponse weights. The paper closes with recommendations for achieving a more comparable measurement of the education variable.