The Impact of Mixed Modes on Multiple Types of Measurement Error

Abstract

Mixed mode designs are becoming standard in the collection of survey data. Despite this, there are still unknowns regarding how mode (e.g., Web) or mode design (e.g., sequential mixed mode) impacts measurement error. Previous research has been limited by the confounding of selection and measurement mode effects and the investigation of only one type of measurement error at a time. In this paper, we use three waves of the Understanding Society Innovation Panel to investigate whether single-mode versus sequential mixed-mode and Web versus face-to-face modes have different impacts on measurement error. We make use of a quasi-experimental design that randomly allocated respondents to either a unimode face-to-face interview or a sequential mixed-mode (Web and face-to-face) design. Through this design, we implement a new multitrait-multierror model that estimates social desirability, acquiescence, and method effects simultaneously. The results show no differences in measurement error between single modes and mode designs with respect to acquiescence and method effect but some differences are found for social desirability. We discuss the practical implications of these findings and their possible causes in conclusion.