Boosting Survey Response Rates by Announcing Undefined Lottery Prizes in Invitation Email Subject Lines Evidence from a Global Randomized Controlled Trial

Abstract

We test whether stating the possibility of winning an unspecified prize (announcing undefined lottery prizes, AULP, henceforth) in the subject line of a survey invitation email increases contact and response rates. As adherence to Muslim cultural norms and Islamic religious values might negatively affect the response to lottery-style prize assignment, we further explore whether the magnitude of those impacts differs for respondents located in Organization of Islamic Cooperation (OIC) member states. We also test for potential impacts of providing translations in a national/official language of the state the potential respondents’ organization is located in (in addition to the English default version) as well as for non-response bias and compromised response quality. To these ends, we conduct a unique randomized controlled trial on a global sample of over 5,000 key staff members of microfinance institutions. Our analysis yields four main results. First, on average, contact and response rates are significantly higher for the AULP treatment group. Second, we find intuitive regional heterogeneity in the impact of our treatment, i.e. AULP, between OIC member states and non-member states. Third, though our findings reveal a positive impact of providing translations in a relevant national/official language, in line with the literature on international survey research, translations do not generally boost AULP treatment effects, on average. Last, quality checks provide no evidence of AULP leading to non-response bias or lower response quality, as defined by various standard metrics.