Effects of Replacing Telephone with Web, Mail, and Mixed-Mode Data Collection in an Establishment Follow-Up Survey

Abstract

Due to rising data collection costs, there is growing pressure to move away fromtraditional interviewer-administered mode designs in favor of fully self-administeredmode designs in ongoing panel surveys, including large-scale establishment panels.However, the consequences of moving to a fully self-administered mode design onfollow-up and cumulative participation in ongoing establishment panel surveys arelargely unknown. To address this research gap, we report the results of a follow-up modedesign experiment conducted in the second wave of the 2020 IAB-Job Vacancy Survey,an ongoing panel study in Germany. The experiment builds on a previously reportedmode design experiment conducted in the first wave survey, where establishments wererandomized to four self-administered mode designs (concurrent mail-web mixed-mode,sequential web-to-mail mixed-mode, single-mode web, and single-mode mail). In thesecond wave (i.e., follow-up) survey, reported here, respondents from the first wavewere further randomly allocated to 1) a continuation of the same self-administeredmode design from the first wave, or 2) a single-mode telephone design. The resultsshow that the continuation of self-administration leads to higher response rates (bothfollow-up and cumulative) for the single-mode mail and concurrent mail-web mixedmodedesigns and comparable response rates for the single-mode web and sequentialweb-to-mail mixed-mode designs, compared to the telephone follow-up design. Usingextensive administrative data, we do not find evidence that forgoing telephone follow-upsadversely affects nonresponse bias or subgroup participation compared to continuingwith self-administration in the follow-up wave. Potential cost savings (of up to 67%)were evidenced when replacing the telephone mode with a self-administered followupmode design. In-depth qualitative interviews revealed that establishments prefer aconstant mode design across waves due to the familiarity and routine of the responseprocess.