In recent years, social scientists have increasingly relied on survey experiments to estimate causal effects. As with any experimental design, survey experiments can fail, deviate from (preregistered) plans, or yield results that researchers did not anticipate. Researchers find themselves in situations with null, unexpected, inconsistent, or inconclusive results and must then decide whether results reflect on the theory being tested, the experimental design, or both. Usually, the insights gained from such failures are not widely shared, even though they could be very useful to improve the design of experiments, quality of research, and transparency.
The planned special issue for Survey Research Methods is dedicated to learning from survey experiments that failed or led to unexpected results. Sharing the design and results from failed survey experiments, carefully considering their possible flaws, and talking about unexpected findings is useful to the development of theory (e.g., identifying scope conditions) and methods, and contributes to transparent research practice.
Reasons for survey experiments to fail might include poor treatment assignment or ineffective manipulations, unreliable or invalid measurements, underdeveloped theory or hypotheses, and underpowered designs. The overall goal of the special issue is to share insights about the causes and consequences of failures in survey experiments, so that others can learn how to avoid these failures and general recommendations for survey experimental methodology can be derived.
We invite submissions of short papers, essays, comments (of up to 3.000 words), and full-length articles (of up to 8.000 words) for this special issue, including case studies of failed survey experiments, meta-analyses, systematic reviews, as well as theoretical, methodological, and statistical contributions that may or may not rely on empirical data, simulated data, or no data at all. Possible research questions we invite authors to address in their submissions include but are certainly not limited to the following:
To submit, go to the SRM website and upload the article as a PDF, just like a standard SRM article; be sure to mention the special issue in the field for Author comments. In addition to the general Author Guidelines a maximum of one table is allowed per article (there are, however, no additional restrictions on the number of figures and graphs for the special issue).
Reviewing policies follow the common standards of SRM: Each submitted paper will be assigned to one of the two editors of the special issue. This supervising editor will then select two expert reviewers. The reviews and the reading of the supervising editor will be used for the final decision.