Non-unique Records in International Survey Projects: The Need for Extending Data Quality Control

Authors

  • Kazimierz Maciek Slomczynski
  • Przemek Powalko Polish Academy of Sciences
  • Tadeusz Krauze Hofstra University

DOI:

https://doi.org/10.18148/srm/2017.v11i1.6557

Keywords:

Survey Data Quality, Duplicate Records, Rare Events, Non-Random Errors in Survey Data

Abstract

For a given survey data file we define a non-unique record, NUR, as a sequence of all values in a given case (record), which is identical to that of another case in the same dataset. We analyzed 1,721 national surveys in 22 international projects, covering 142 countries and 2.3 million respondents, and found a total of 5,893 NURs concentrated in 162 national surveys, in 17 projects and 80 countries. We show that the probability of the occurrence of any NUR in an average survey sample is exceedingly small, and although NURs constitute a minor fraction of all records, it is unlikely that they are solely the result of random chance. We describe how NURs are distributed across projects, countries, time, modes of data collection, and sampling methods. We demonstrate that NURs diminish data quality and potentially have undesirable effects on the results of statistical analyses. Identifying NURs allows researchers to examine the consequences of their existence in data files. We argue that such records should be flagged in all publically available data archives. We provide a complete list of NURs for all analyzed national surveys.

Author Biographies

Kazimierz Maciek Slomczynski

Professor Emeritus, Department of Sociology OSU, Professor PAN

Przemek Powalko, Polish Academy of Sciences

Computer specialist, Institute of Philosophy and Sociology, Polish Academy of Sciences

Tadeusz Krauze, Hofstra University

Professor, Department of Sociology

Downloads

Additional Files

Published

2017-04-10

How to Cite

Slomczynski, K. M., Powalko, P., & Krauze, T. (2017). Non-unique Records in International Survey Projects: The Need for Extending Data Quality Control. Survey Research Methods, 11(1), 1–16. https://doi.org/10.18148/srm/2017.v11i1.6557

Issue

Section

Articles

Similar Articles

1 2 3 4 5 6 7 8 9 10 > >> 

You may also start an advanced similarity search for this article.