Measurement quality of 67 common social sciences questions across countries and languages based on 28 Multitrait-Multimethod experiments implemented in the European Social Survey

Authors

DOI:

https://doi.org/10.18148/srm/2021.v15i3.7816

Keywords:

European Social Survey (ESS), measurement quality, reliability, validity, Multitrait-Multimethod experiments, MTMM, social sciences, data quality, questionnaire dessign

Abstract

Survey data is used in many social science studies. The measurement quality of these data is crucial as it determines the accuracy of the information on which these studies are based. Besides, since these studies are used to provide insights to key political and social actors, it also determines the accuracy of the information on which crucial decisions are based. In this paper, we estimated the measurement quality (proportion of the variance of the observed survey responses explained by the latent trait of interest) of 67 common social sciences questions that were part of Multitrait-Multimethod experiments in the seven first rounds of the European Social Survey. These questions were asked using response scales with different characteristics and in up to 41 country-language groups. Our results show that measurement errors are omnipresent: the average measurement quality across all questions is 0.65. Thus, overall, on average 35% of the variance in the observed survey answers can be attributed to measurement errors. Furthermore, the size of errors varies across questions as well as across country-language groups. The questions’ average measurement quality across all country-language groups ranges from 0.25 to 0.88, depending on the response scale and topic, and the country-language groups’ average measurement quality across questions ranges from 0.52 to 0.76. Thus, the impact of measurement errors on applied research can be different depending on the exact question formulation and response scale used as well as on the country and language of interest. Consequently, in each study, researchers should consider assessing the size of the measurement errors of their variables and how this affects their results.

Downloads

Additional Files

Published

2021-08-19

How to Cite

Poses, C., Revilla, M., Asensio, M., Schwarz, H., & Weber, W. (2021). Measurement quality of 67 common social sciences questions across countries and languages based on 28 Multitrait-Multimethod experiments implemented in the European Social Survey. Survey Research Methods, 15(3), 235–256. https://doi.org/10.18148/srm/2021.v15i3.7816

Issue

Section

Articles

Most read articles by the same author(s)

Similar Articles

<< < 24 25 26 27 28 29 30 31 32 33 > >> 

You may also start an advanced similarity search for this article.