Effects of Replacing Telephone with Web, Mail, and Mixed-Mode Data Collection in an Establishment Follow-Up Survey

Abstract

Due to rising data collection costs, there is growing pressure to move away from traditionalinterviewer-administered mode designs in favor of fully self-administered mode designs in on-going panel surveys, including large-scale establishment panels. However, the consequences ofmoving to a fully self-administered mode design on follow-up and cumulative participation inongoing establishment panel surveys are largely unknown. To address this research gap, wereport the results of a follow-up mode design experiment conducted in the second wave of the2020 IAB-Job Vacancy Survey, an ongoing panel study in Germany. The experiment buildson a previously reported mode design experiment conducted in the first wave survey, whereestablishments were randomized to four self-administered mode designs (concurrent mail-webmixed-mode, sequential web-to-mail mixed-mode, single-mode web, and single-mode mail). Inthe second wave (i.e., follow-up) survey, reported here, respondents from the first wave werefurther randomly allocated to 1) a continuation of the same self-administered mode design fromthe first wave, or 2) a single-mode telephone design. The results show that the continuation ofself-administration leads to higher response rates (both follow-up and cumulative) for the single-mode mail and concurrent mail-web mixed mode designs and comparable response rates for thesingle-mode web and sequential web-to-mail mixed-mode designs, compared to the telephonefollow-up design. Using extensive administrative data, we do not find evidence that forgoingtelephone follow-ups adversely affects nonresponse bias or subgroup participation comparedto continuing with self-administration in the follow-up wave. Potential cost savings (of up to67%) were evidenced when replacing the telephone mode with a self-administered follow-upmode design. In-depth qualitative interviews revealed that establishments prefer a constant modedesign across waves due to the familiarity and routine of the response process.