When Happiness Induction Techniques Make Respondents Unhappy: Failing to Induce Happiness During the Russo-Ukrainian War

1 Introduction

The impact of discrete emotions such as anger, anxiety, or happiness is an important and emerging field of research in political science (Marcus, 2000; Webster et al., 2022; Yu et al., 2021). As emotions structure cognition and thereby affect evaluations and decision-making, they can help explain individual differences, such as in political attitudes and voting behavior, or moderate relationships between variables like personality traits and behavioral outcomes (e.g., Mayer & Nguyen, 2021; Webster & Albertson, 2022). Emotions conceptualized as states are highly dynamic, yet the degree of emotional stability differs systematically across individuals (e.g., by personality traits; Webster & Albertson, 2022). Accordingly, survey research has made emotion priming and the measurement of its consequences a common practice, with established best-practice standards (Albertson & Gadarian, 2016). While it may be more difficult to reliably elicit some feelings, such as surprise, previous research on happiness has revealed that four main strategies are effective (Siedlecka & Denson, 2019): Most studies have indicated higher levels of happiness in the treatment group, regardless of whether visual stimuli were shown, music was played, joyful autobiographical memories in which pleasure was felt were recalled, or situations in which happiness should be felt were recalled. The combination of several approaches also led to a rise in happiness among respondents (Zhang et al., 2014).

However, almost all studies that induced happiness were conducted within stable political contexts and do not include extreme cases (for a review, see Siedlecka & Denson, 2019). In this regard, extreme cases include contexts where the induced emotion, in our case, happiness, is particularly low due to contextual factors, such as war or very unfavorable economic conditions.

In this paper, we aim to put three methods for experimentally inducing happiness to the test and analyze whether they are also able to induce happiness in an extremely unfavorable context: Ukraine during the Russo-Ukrainian war. We test whether the emotional recall of happy situations in the past, displaying happy images, or triggering happy imagined situations by using questions on joyful aspects of life are able to increase the feeling of happiness compared to a control group. However, our results show that these three techniques actually have the opposite effect on Ukrainians and reduce happiness in respondents. Our findings indicate that established affect-induction techniques validated in stable Western contexts may not transfer to more extreme settings, in which primes might lose their valence or backfire, e.g. when recalling happiness activates associations tied to loss, such as memories of earlier times or of people and places that are no longer present.

Building on these insights, our study speaks to a broader methodological issue in survey experimentation: the limited transferability of established affect-induction techniques across contexts. By analyzing a failed happiness manipulation, we identify scope conditions under which widely used experimental treatments may lose their intended meaning or even reverse in effect. This underscores that the validity of emotional primes is not inherent to the stimulus itself but contingent on respondents’ lived environments, cognitive associations, and situational constraints. More generally, our findings highlight the need to systematically interrogate the contextual dependencies of experimental manipulations, rather than assuming their portability across populations and settings, thereby contributing to ongoing debates on external validity and the generalizability of survey experiments.

2 Data and Methods

The data for this study are part of the Resettlement of Ukrainians Panel Study (ReUP), a larger online panel project on the feasibility of using social media ads to recruit panel participants in countries experiencing armed conflict (Mayer et al., 2024; Walter et al., 2022; see Supplementary Information (SI), part A for a detailed discussion of the project). The first wave was conducted in June 2022, a few months after the beginning of the Russo-Ukrainian war on February 24, with the intent to collect contact information for a panel study (N = 19,001; panel consent = 12,150). As we provided no incentive and the second wave was only launched in November, we experienced considerable panel attrition, mostly between the first and second waves, but also between the second and third waves.¹ For this paper, we draw on data from the second (24 November 2022 to 31 January 2023, N = 1980), third (11 July to 6 August 2023, N = 1542), and fourth wave of the panel survey, which was split into two time periods (8–24 February, N = 1059, and 11 April to 6 May 2024, N = 580). We refer to these panel waves as Studies 1–4 in this paper, with the two time periods of wave 4 treated as separate studies (see SI, part D for participant flow diagrams for all three waves).

For all waves but the fourth, we sent out one invitation and three reminders, and respondents were never incentivized. For the fourth wave, we sent out one invitation for Study 3, and 2 reminders for Study 4.

Initially, our aim for Study 1 was to replicate a study on the null effect of happiness on affective polarization (Yu et al., 2021) in the context of war-torn Ukraine. The original survey experiment was preregistered (see SI, part C3) and the study was approved by the ethics board of the University of Bamberg. However, after discovering in Study 1 that the emotional recall technique did not work as expected, we decided to focus on the methodological aspect of the project, i.e. the effectiveness of happiness induction techniques.

2.1 Happiness Induction Techniques

In Study 1, we used the emotional recall technique, asking respondents in the treatment group to recall a happy situation in their life, a prompt that has been successfully used many times in the past (e.g., Webster et al., 2022; Yu et al., 2021, see SI, part B for all wordings, part C for all preregistrations, and part D for participant flow charts). The control group was asked what they had for breakfast that morning, again a prompt that has been used frequently due to its emotional neutrality (e.g., Webster et al., 2022; Yu et al., 2021).

For Study 2, we used visual stimulus material. The control group received a photo of a paper stack (“Paper 3”) with neutral valence and low arousal, and the treatment group received a photo of a Caribbean beach (“Beach 1”) with high valence and neutral arousal, taken from the OASIS database (Kurdi et al., 2017). Each image was validated by over 100 different trials in the original study. However, as the validation was mainly done with respondents from the United States, it might be possible that such a picture can arouse slightly different affective ratings in other regions. Respondents were asked to describe what they saw to reinforce the images.

In Study 3, we replicated the emotional recall question from Study 1 to check whether the paradoxical results from Study 1 could still be observed after the war had been ongoing for several months. After half of the field time of wave 4, we decided to add a third induction technique, and modify the control prompt to inquire about an average day at age 16. This was done to ensure that the paradoxical findings were not due to the original control prompt’s focus on breakfast, which could have been associated with unpleasant experiences resulting from the war, such as power outages. However, we must add a cautionary note that recalling an event from a fixed age might be more difficult for older participants than for younger ones. In addition, negative experiences from this sensitive phase of life might affect the results as well. We refer to this as Study 4.

Participants were randomly assigned to conditions using LimeSurvey’s built-in random number generator which uses simple random assignment. Study 1 used unequal allocation (control 0.80, treatment 0.20), as we initially feared that the happiness treatment might lead to higher dropout due to increased cognitive effort whereas Studies 2–4 used equal allocation (0.50/0.50). We acknowledge that participants might remember their answers and the questions from the previous wave. However, the time gap between waves was always around six months or more, and the questionnaires contained many additional items, making it unlikely that participants would recall specific questions. Hence, we do not believe this poses a problem for our study.

2.2 Questionnaire and Manipulation Check

Of course, this bipolar question has the disadvantage that our happiness prompts might elicit both happy and sad feelings simultaneously, which would not be adequately captured by a bipolar format. However, as our original intention was to replicate the findings of Yu et al. (2021), we chose the bipolar version they also used.

In addition, a happiness-inducing technique should result in higher levels of happiness regardless of whether a unipolar scale (e.g., rating happiness from not happy to very happy) or a bipolar scale is used. Likewise, respondents in both the control and treatment groups should be equally affected by the response format, meaning that any differences in responses should stem from differences in the treatment stimuli rather than from the type of rating scale used. For example, Höhne et al. (2021) showed that end-labeled unipolar and bipolar scales (as in our case) are the most effective in meeting the equidistance criterion in contrast to completely labelled bipolar scales. Finally, we kept the rating scale constant throughout all studies in order to assure comparability.

2.3 Panel Attrition and Non-Response

Dropout was slightly but significantly higher in the treatment group in Studies 1 (1% vs. 3%, p < 0.01), 3 (2% vs. 4%, p < 0.01), and 4 (0% vs. 3%, p < 0.01), but not in Study 2 (1% vs. 2%). Item non-response differed only in Study 4 (25% control vs. 19% treatment, p < 0.05) (see also SI, Table A3). These effects are small, and results were substantively identical for compliers (ITT, Table 1) and the full sample (CACE, SI, Table E1), suggesting robustness. Logistic regressions of dropout and item non-response on treatment, covariates (age, gender, education, region), and treatment-by-covariate interactions yielded no significant interactions (Wald tests). Study 4 was the only exception where low variance prevented estimation. While attrition rates differ modestly between treatment and control conditions in some waves, we find no evidence that attrition is systematically related to observed covariates.²

3 Results

Table 1 displays the results of mean differences from two-tailed t‑tests for the different survey experiments. In Study 1, respondents in the treatment group reported significantly lower happiness than those in the control group (mean_diff = −0.30, p < 0.01). This result was replicated in Study 3 (mean_diff = −0.42, p < 0.01). Recalling a happy situation actually reduced self-reported happiness. Since we used a bipolar manipulation check, we cannot determine whether this effect stems from increased sadness. We explore this question further in the qualitative analysis below. To rule out the breakfast prompt as the cause of the effect, we used a different control prompt in Study 4, asking about an ordinary day at age 16. Those respondents showed similar levels on the happy-sad scale as participants in the breakfast control conditions of Studies 1 and 3. Regardless of the control prompt, average happiness levels remain similar across groups. This suggests that the happiness recall itself may trigger the seemingly paradoxical effect.

In Study 2, we used a visual stimulus which participants described. Although those in the treatment group reported slightly higher happiness (mean_diff = 0.14), the difference was not statistically significant. Study 4 used six happiness-related prompts (e.g., favorite song or place). Again, treatment group respondents reported slightly higher happiness than controls (mean_diff = 0.18), but the effect did not reach conventional levels of significance.

Because the findings for the emotional recall question contradicted our expectations and prior research, we examined the open-ended responses to better understand why the treatment might have reduced happiness. Fig. 1 presents five of the more representative and illustrative example responses from the treatment group. Open-ended responses were first machine-translated to English via DeepL and then checked for validity by a native Ukrainian and Russian speaker. All analyses were performed using quanteda (Benoit et al., 2018), tidytext (Silge & Robinson, 2016), and wordcloud (Fellows, 2011) in R (R Core Team, 2023). These responses suggest that recalling a positive memory often triggered associations with wartime events, which may have evoked sadness.

Fig. 1 Example responses from open-ended answers in treatment condition (Study 1). Responses were selected based on their representativeness and being illustrative of respective experiences.

This notion is supported by findings from sentiment analyses of the open-ended responses in all studies (see SI, part E).³ Responses in the treatment group were categorized more often as related to fear (e.g., mentioning “war”), while responses in the control group were more often associated with joy (e.g., mentioning “food”). Collocation word clouds (Fig. 2) show a strong link between “happy” and “war”, and weaker associations with “Russian”, “alive”, or “healthy”. These findings are consistent across all studies.

Fig. 2 Network of collocations for “happy” in treatment condition (Study 1). Network graph of open-ended responses to the emotional recall question. Shown are the collocations between the top 20 tokens (based on highest phi values) and the keyword “happy” (window = 5, i.e. the next five closest words) in the treatment group. Thicker lines indicate a higher frequency of bigram occurrences. The highest frequency exists for the combination of “happy” and “war”.

4 Conclusion

Understanding how emotions shape political behavior depends, among other things, on our ability to reliably manipulate affect in survey experiments. While established techniques exist for inducing discrete emotions like anger or happiness, our findings raise serious concerns about their validity in crisis contexts (see also Amano et al., 2023).

Our experimental manipulation failed because it uncritically transferred affect-induction methods validated in stable Western societies to a setting marked by war, displacement, and psychological stress. The emotional recall technique, often considered a gold standard, backfired and significantly reduced reported happiness. Visual and imaginal prompts also failed to produce the intended effect. These outcomes raise broader questions about how and when positive stimuli lose their intended valence under adverse conditions.

Future research should examine which features of “happy memories” transform them into painful reminders, e.g., their temporal distance, links to lost people or places, or references to now-impossible activities. Our open-ended data suggest such associations but call for more systematic inquiry. The varying outcomes across techniques highlight the role of cognitive elaboration. Actively recalling and describing a memory may trigger stronger detrimental effects than passive exposure to imagery. The intensity of the prime also matters: high-arousal primes may clash with crisis experiences, while subtler ones may be more appropriate.

Our findings also suggest that affective responses may be shaped by contrast or dissonance, as respondents juxtaposed past joy with present suffering. We did not measure these mechanisms, which was a design limitation. Future studies should assess whether it is the expectation to feel happy that produces dissonance when such an emotion feels out of reach.

These failures highlight the need for context-sensitive priming. Our collocation analyses showed that even the word “happy” was often linked to war. This underscores the risk that standard positive primes can evoke trauma. Cognitive pretesting, through think-aloud protocols or interviews (Blair & Conrad, 2011; Koskey, 2016; Perneger et al., 2015), is essential. Our study lacked this step but clearly would have benefited from it. Co-developing stimuli with affected communities may also improve cultural and emotional fit. Given these challenges, it may be more productive to explore alternative emotional targets, such as resilience, coping efficacy, solidarity, or moments of calm and agency. These emotional states are often more accessible in times of adversity and less likely to evoke painful comparisons to past hardships. Additionally, the timing of the emotional priming matters. Comparing the effects of primes that focus on past experiences with those oriented toward present perceptions or future hope—when contextually relevant—could help identify more effective and less distressing priming strategies.

Ultimately, our findings reveal the danger of assuming that affect-induction techniques are universally transferable. In emotionally volatile contexts, the research setting itself reshapes how stimuli are interpreted. Emotional primes must be designed with, not just in, context. Our study’s failure is instructive, not because it was rare, but because it was predictable. Learning from such failures is essential to building more robust, context-aware research practices.

1 Supplementary Information

References

Albertson, B., & Gadarian, S. K. (2016). Did that Scare You? Tips on Creating Emotion in Experimental Subjects. Political Analysis, 24(4), 485–491. https://doi.org/10.1093/pan/mpw022. →

Amano, K., Dominguez, M. S., Fraser, T., Gagnon, E., Incerti, T., Jang, J., McClean, C. T., Mitchell, A. M., Miyano, S., Moreshead, C., Saijo, H., Stanescu, D., Teraoka, A., Yamagishi, H., Willis, C. N., Woo, Y., & Crabtree, C. (2023). Field Research When There Is Limited Access to the Field: Lessons from Japan. PS: Political Science & Politics, 56(1), 99–105. https://doi.org/10.1017/S1049096522000932. →

Benoit, K., Watanabe, K., Wang, H., Nulty, P., Obeng, A., Müller, S., & Matsuo, A. (2018). quanteda: An R package for the quantitative analysis of textual data. Journal of Open Source Software, 3(30), 774. https://doi.org/10.21105/joss.00774. →

Blair, J., & Conrad, F. G. (2011). Sample Size for Cognitive Interview Pretesting. Public Opinion Quarterly, 75(4), 636–658. https://doi.org/10.1093/poq/nfr035. →

Fellows, I. (2011). wordcloud: Word Clouds. Version 2.6. https://doi.org/10.32614/CRAN.package.wordcloud. →

Höhne, J. K., Krebs, D., & Kühnel, S. M. (2021). Measurement Properties of Completely and End Labeled Unipolar and Bipolar Scales in Likert-Type Questions on Income (In)Equality. Social Science Research, 97, 102544. https://doi.org/10.1016/j.ssresearch.2021.102544. →

Kam, C. D., Wilking, J. R., & Zechmeister, E. J. (2007). Beyond the “Narrow Data Base”: Another Convenience Sample for Experimental Research. Political Behavior, 29(4), 415–440. https://doi.org/10.1007/s11109-007-9037-6. →

Koskey, K. L. K. (2016). Using the Cognitive Pretesting Method to Gain Insight Into Participants’ Experiences: An Illustration and Methodological Reflection. International Journal of Qualitative Methods 15(1). https://doi.org/10.1177/1609406915624577. →

Kurdi, B., Lozano, S., & Banaji, M. R. (2017). Introducing the Open Affective Standardized Image Set (OASIS). Behavior Research Methods, 49(2), 457–470. https://doi.org/10.3758/s13428-016-0715-3. →

Manski, C. F., & Pepper, J. V. (2000). Monotone Instrumental Variables: With an Application to the Returns to Schooling. Econometrica, 68(4), 997–1010. https://doi.org/10.1111/1468-0262.t01-1-00144a. →

Marcus, G. E. (2000). Emotions in Politics. Annual Review of Political Science, 3(1), 221–250. https://doi.org/10.1146/annurev.polisci.3.1.221. →

Mayer, S. J., & Nguyen, C. (2021). Angry Reactionary Narcissists? Anger Activates the Link Between Narcissism and Right-Populist Party Support. Politics and Governance, 9(3), 248–259. https://doi.org/10.17645/pag.v9i3.4000. →

Mayer, S. J., Dollmann, J., & Meth, A. (2024). Re-UP—Panel Study of Ukrainians Unpublished data set. →

Mohammad, S. M., & Turney, P. D. (2013). Crowdsourcing a Word-Emotion Association Lexicon. Computational Intelligence, 29(3), 436–465. https://doi.org/10.1111/j.1467-8640.2012.00460.x. →

Perneger, T. V., Courvoisier, D. S., Hudelson, P. M., & Gayet-Ageron, A. (2015). Sample Size for Pre-Tests of Questionnaires. Quality of Life Research, 24(1), 147–151. https://doi.org/10.1007/s11136-014-0752-2. →

R Core Team (2023). Version 4.5.3. R: A Language and Environment for Statistical Computing. https://www.r-project.org/. →

Sears, D. O. (1986). College Sophomores in the Laboratory: Influences of a Narrow Data Base on Social Psychology’s View of Human Nature. Journal of Personality and Social Psychology, 51(3), 515–530. https://doi.org/10.1037/0022-3514.51.3.515. →

Siedlecka, E., & Denson, T. F. (2019). Experimental Methods for Inducing Basic Emotions: A Qualitative Review. Emotion Review, 11(1), 87–97. https://doi.org/10.1177/1754073917749016. a, b

Silge, J., & Robinson, D. (2016). tidytext: Text Mining and Analysis Using Tidy Data Principles in R. The Journal of Open Source Software, 1(3), 37–37. https://doi.org/10.21105/joss.00037. →

Walter, L., Mayer, S. J., Dollmann, J., Jacobsen, J., & Meth, A. (2022). High Levels of Support for European Union Accession in Ukraine During the War in 2022. An Analysis Based on the ReUP Study. Deutsches Zentrum für Integrations- und Migrationsforschung (DeZIM). https://doi.org/10.31235/osf.io/zqn9j. →

Webster, S. W., & Albertson, B. (2022). Emotion and Politics: Noncognitive Psychological Biases in Public Opinion. Annual Review of Political Science, 25(1), 401–418. https://doi.org/10.1146/annurev-polisci-051120-105353. a, b

Webster, S. W., Connors, E. C., & Sinclair, B. (2022). The Social Consequences of Political Anger. The Journal of Politics, 84(3), 1292–1305. https://doi.org/10.1086/718979. a, b, c

Yu, X., Wojcieszak, M., Lee, S., Casas, A., Azrout, R., & Gackowski, T. (2021). The (Null) Effects of Happiness on Affective Polarization, Conspiracy Endorsement, and Deep Fake Recognition: Evidence from Five Survey Experiments in Three Countries. Political Behavior, 43(3), 1265–1287. https://doi.org/10.1007/s11109-021-09701-1. a, b, c, d, e

Zhang, X., Yu, H. W., & Barrett, L. F. (2014). How does this make you feel? A comparison of four affect induction procedures. Frontiers in Psychology 5:689. https://doi.org/10.3389/fpsyg.2014.00689. →