Measurement Equivalence in Sequential Mixed-Mode Surveys

Abstract

Many surveys collect data using a mixture of modes administered in sequential order. Although the impacts of mixed-mode designs on measurement quality have been extensively studied, their impacts on the measurement quality of unobservable (or latent) constructs is still an understudied area of research. In particular, it is unclear whether latent constructs derived from multi-item scales are measured equivalently across different sequentially-administered modes – an assumption that is often made by analysts, but rarely tested in practice. In this study, we assess the measurement equivalence of several commonly-used multi-item scales collected in a sequential mixed-mode (Web-telephone-face-to-face) survey: the Age 25 wave of the Next Steps cohort study. After controlling for selection via an extensive data-driven weighting procedure, a multi-group confirmatory factor analysis was performed to assess measurement equivalence across the three modes. We show that cross-mode measurement equivalence is achieved for the majority of scales, with partial equivalence established for the remaining scales. Although measurement equivalence was achieved, some differences in the latent means were observed between the modes. We conclude with a discussion of these findings, their potential causes, and implications for survey practice.