Does Including Internet Users in the Mail Mode Improve the Data Quality of a Probability-Based Mixed-Mode Panel?

Abstract

Previous research on probability-based online panels shows that providing the offline-population with an alternative survey mode reduces coverage bias. However, little is known about whether offering an offline mode to internet users who are unwilling to participate in panel surveys online pays off in lower nonresponse bias. This study uses data from the GESIS Panel, a probability-based web and mail panel, to investigate how including internet users in the mail mode affects nonresponse bias of population and model estimates. The results show that internet users included in the mail mode differ from non-internet users assigned to the mail mode and panelists responding in the web mode in demographic variables and internet usage after the recruitment. However, excluding internet users in the mail mode from the data set rarely introduces a higher bias in estimates of demographic characteristics compared to data from a reference sample. Further analyses examine how mean and model estimates of three studies published with GESIS Panel data would have been affected by excluding internet users from the sample who are unwilling to provide survey data online. Here the findings show potential nonresponse bias in several estimates of means from the model variables. However, the model estimates of all three studies are largely the same after removing those cases from the analyses. Therefore, authors' conclusions would likely have remained unchanged if internet users had not been included in the mail mode.