Effects of Changing the Incentive Strategy on Panel Performance: Experimental Evidence From a Probability-Based Online Panel of Refugees

Abstract

This study investigated how changing the mode of incentive administration between two panel waves, spaced six months apart, affected longitudinal survey response. A split-ballot incentive experiment was used to compare shifting from an unconditional pre-paid incentive mode in the first wave to a conditional post-paid mode in the second wave, versus consistently using a conditional post-paid mode across both waves. Social Exchange Theory, Self-Perception Theory, and Leverage-Salience Theory form the theoretical framework for grasping response behavior. The main performance indicators for evaluating both incentive strategies were wave-specific and total panel participation, panel consent, and cumulative response. Results also sheds light on data quality, demographic composition, and survey costs. The experiment was implemented in the context of the IAB-BiB/FReDA-BAMF-SOEP survey “Refugees from Ukraine in Germany,” a probability-based register sample. Multivariate analysis indicated that unconditional pre-paid incentives were only superior within a single-wave perspective, and a constant conditional post-paid incentive strategy substantially outperformed the incentive strategy that changed the mode from initial pre-paid to post-paid. This was mainly the case in terms of participation and survey costs, while data quality and bias concerning demographics were similar across the groups. Based on these results, we derive practical implications.