Invitation Messages for Business Surveys: A Multi-Armed Bandit Experiment

Abstract

We investigate the design of a survey invitation message targeted at businesses. By varying five key elements of the survey invitation, we implement a full-factorial experiment with adaptive randomization instead of static group composition. Specifically, as the experiment progresses we apply a Bayesian learning algorithm that assigns more observations to invitation messages with higher starting rates. Our results indicate that personalizing the message, emphasizing the authority of the sender, and pleading for help increase survey starting rates, while stressing strict privacy policies and changing the location of the survey URL have no response-enhancing effect. Our implementation of adaptive randomization is useful for other applications of survey design and methodology.