Triggering without Deceiving—An Attempt at Designing a Non-Deceptive Male Identity Threat Treatment

The rise of right-wing populism has been linked to threats to male identity from cultural and economic change. We report a survey experiment (N = 468 adult U.S. men) deploying a novel, non-deceptive treatment designed to induce male identity threat. We find no evidence that our treatment affected the importance men attach to gender identity or downstream outcomes including attitudes toward transgender individuals and ethnic and partisan in- and out-groups. Either our treatment failed to threaten male identities or effects are too subtle to detect. We draw two lessons for experimental design: adequate statistical power and credible manipulation checks are essential for distinguishing manipulation failure from situations where data reject the theory. We discuss potential reasons for manipulation failure in our case – omitting explicit performance standards for idealized masculine behavior and our choice of masculinity dimension – and highlight how descriptive research can inform future non-deceptive identity threat experiments.

Right-wing populism is on the rise. Brexit, Donald Trump, and the success of European far-right parties are prominent examples. A common explanation points to discontent among men arising from economic and cultural changes. Globalization, de-industrialization, and increasing gender equality are said to have threatened male identities. Evidence for identity threat’s role comes in two forms. First, studies link community-level economic changes to electoral support for right-wing populist candidates (Autor et al. 2020; Baccini and Weymouth 2021; Becker et al. 2017; Clark et al. 2022; Colantone and Stanig 2018a; Colantone and Stanig 2018b; Dippel et al. 2022), credibly documenting a causal link between economic shocks and populism but struggling to isolate identity threat from other mechanisms like economic frustration (e.g., Clark et al. 2022). Second, surveys document a link between a sense of marginalization and support for right-wing populism (e.g., Anduiza and Rico 2024; Gidron and Hall 2017; Guiso et al. 2024; Norris and Inglehart 2016; Valentino et al. 2018), enabling the study of mechanisms like identity threat (Gidron and Hall 2017), but raising concerns about reverse causality and unobserved confounders.

A survey experiment that randomly threatens male identities could provide identified evidence of effects on proximate outcomes like feelings toward out-groups and illuminate the psychological mechanisms underlying macro-relationships (Ludwig et al. 2011). Yet, identity threat treatments must be subtle enough to avoid ethical concerns but strong enough to create temporary effects. Moreover, it can be difficult to conceive of non-deceptive treatments.

A large literature studies threats to male identities through deceptive treatments, mostly in the lab (see Dahl et al. 2015; Maass et al. 2003; Rudman and Fairchild 2004; Vandello et al. 2008; Vescio et al. 2021; Vescio et al. 2025a, b; Willer et al. 2013). We sought to create a non-deceptive treatment, since deception is particularly problematic in online survey experiments, where verifying debriefing is difficult. Moreover, online experiments typically use panels of paid respondents who regularly participate in surveys. Deception may undermine their trust in future experiments. We also worry about psychological risks. A widely used design involves a quiz about “gendered” knowledge and randomly informs men that their score falls into a range typical for women—regardless of actual performance (Rudman and Fairchild 2004; Vandello et al. 2008; Vescio et al. 2021; Vescio et al. 2025a, b). Participants thus may receive false information contradicting their self-beliefs,¹ which may induce psychological distress even after debriefing.

We report on a pre-registered online survey experiment with N = 468 men in the U.S.—to our knowledge, the first attempt at a non-deceptive male identity threat treatment.² The literature identifies several dimensions of masculinity common across Western societies: embodying cultural ideals requires being “(a) high in power, status, and dominance, (b) emotionally, physically, and mentally tough, and (c) reject and distance from all that is feminine, gay, or otherwise unmanly” (Vescio et al. 2021). Maass et al. (2003) distinguish between different identity threat types: group value threats undermine the in-group’s status, distinctiveness threats challenge between-group differences, legitimacy threats question the acceptability of cross-group status hierarchies, and acceptance threats erode one’s sense of being a prototypical group member.

Our operationalization was guided by our goal to provide micro-evidence linking economic shocks, men’s inability to fulfill gendered expectations, and right-wing populism. We thus designed our treatment to threaten men’s sense of acceptance regarding economic success—a typical element of masculinity’s power, status, and dominance dimension.

Drawing on social identity theory, we hypothesized men would resolve the resulting frustration by reaffirming their male identity (Elsbach 2003; Elsbach and Kramer 1996) or re-categorizing themselves via non-threatened identities like race or partisanship (Sambanis and Shayo 2013; Shayo 2009). Men attaching more importance to their masculinity may display less favorable views toward transgender individuals, whom many cis-men see as an out-group. Those attaching less importance to gender but more to race or partisanship may express more favorable views toward ethnic and partisan in-groups and less favorable views toward out-groups (Shayo 2009; Tajfel 2010; Turner 1975).

We find no evidence of treatment effects on men’s gender identity or downstream outcomes, consistent with our treatment failing to induce appreciable identity threat. However, limitations such as low statistical power and few manipulation checks prevent us from ruling out subtle identity threat effects we fail to detect. We discuss general experimental design lessons as well as key learnings for the design of non-deceptive identity threat treatments: the importance of performance standards, the target dimension of masculinity, and prior descriptive research.

1 Experimental Design

1.1 Treatment

Table 1 summarizes the experimental design. Respondents were randomly assigned to treatment or control via simple randomization upon reaching the experiment. Table A3 provides evidence of covariate balance. All respondents first viewed an introductory screen designed to create the impression that men excel at logical reasoning and to link this ability to business success. The screen displayed the gender distribution of investment bankers as a pie chart (see Figure A1), accompanied by the following text:

	Treatment Group	Control Group
Step 1	Introductory screen (Fig. A1)	Introductory screen (Fig. A1)
Step 2	Logical reasoning task	Logical reasoning task
Step 3	Truthful feedback	–
Step 4	Outcome questions	Outcome questions

On the next screen, respondents were asked to solve a logical reasoning task from a financial sector blog (Verlet 2021):

The correct answer is 24 minutes. Control respondents received no feedback and proceeded with the survey. Treated respondents read either “Congratulations, you are correct!” if they answered 24 minutes or “That answer is incorrect. The correct answer is 24 minutes,” if they had answered something else or left the field blank.

Because treatment and control experiences were identical until the point of feedback, treatment assignment could not affect task performance. Indeed, we find no evidence of a treatment effect on the likelihood of answering correctly (Table A4). We can therefore restrict the analysis to male respondents who answered incorrectly (N = 413) and identify the effect of negative feedback, which we expected to induce a sense of identity threat among men.

We sought to temporarily highlight a genuine failure to meet gendered expectations. Our treatment is less likely to contravene participants’ self-image than deceptive treatments, reducing psychological risks but potentially limiting effectiveness. However, the literature describes masculinity as easily threatened (e.g., Berdahl et al. 2018; Vandello et al. 2008) because it prescribes difficult-to-attain ideals—power, status, and toughness (Vescio et al. 2025b)—and is inherently relative, requiring men to continuously out-compete others (Willer et al. 2013). It thus seemed conceivable that even a subtle non-deceptive treatment may induce a sense of threat.

Our approach had additional advantages. First, logical reasoning is a specific skill unlikely to be salient for all respondents, making it plausible that our treatment could shift self-perceptions about it. Second, performance on the logical reasoning task correlates weakly negatively with male identity strength in our sample (). This benefits our design: prior work shows identity threats are especially consequential for individuals strongly committed to an identity (Maass et al. 2003). The fact that men with a stronger masculine identity were, if anything, more likely to fail the task allows us to study the effect of truthful negative feedback among them.³ Finally, statistical power increases when most respondents give an incorrect answer, since we estimate effects only among this group. Pre-testing confirmed the task’s difficulty: Figure A2 shows almost 90% of men gave no or incorrect answers.

1.2 Measurement

Next, we measure gender identity strength by asking “How important is being a man to your identity?” with five response options ranging from “not at all important” to “extremely important.” The question is similar but not identical to our pre-treatment measure to prevent survey fatigue and avoid connecting the earlier gender module to the experiment.⁴ Similar questions are commonly used to measure social identity strength (e.g., Smith et al. 2019).

Missingness never exceeds 2% of responses in any outcome and is imputed via chained equations using the R package mice. We exclude respondents with undefined outcomes (e.g., ambiguous race precludes defining a racial in-group) from the respective analyses.

1.3 Ethics

1.4 Estimation

y_i is respondent i’s outcome, α an intercept, τ the average treatment effect, z_i the treatment assignment indicator, a vector of 13 pre-registered de-meaned covariates with coefficient vector , a vector of treatment-by-covariate interactions with coefficient-vector , and a heteroskedasticity-robust error term. Treatment-by-covariate interactions ensure covariate adjustment cannot hurt asymptotic precision (Lin 2013). Supplementary Information A.4 provides details on covariates; section C.1 shows results without covariates.

We focus on men who answered the logical reasoning task incorrectly or did not respond. Supplementary Information C.2 shows identical results excluding respondents who gave no answer. We calculate exact p-values via randomization inference, a nonparametric approach relying on the known random assignment mechanism rather than large-sample or distributional assumptions (Fisher 1935; Gerber and Green 2012). We permute treatment assignment 10,000 times to simulate the sampling distribution under the sharp null hypothesis of no (positive or negative) effect for any unit and compare the observed estimate to this distribution. Hypothesis test directions are pre-registered.

2 Results

Fig. 1 shows estimated effects of negative feedback on gender identity strength. We find no evidence of treatment effects among all men who gave incorrect or no answers, nor across four pre-registered subgroups that we expected to be most susceptible to male identity threat. Consistent with the apparent absence of a first stage effect, we find no evidence of effects on downstream outcomes among any group (Fig. 2 and Supplementary Information C.1).

Fig. 1 Effect of negative feedback on the strength of men’s gender identity (first stage). Dots indicate point estimates in control-group standard deviation units, and bars 90% (thick line) and 95% (thin line) parametric confidence intervals. p-values are two-tailed and stem from randomization inference. Outcomes are standardized using the control-group mean and standard deviation. Table A5 in the Supplementary Information presents the results in tabular form, section A.4 provides estimation details, and section D the question wording.

Fig. 2 Effects of negative feedback on downstream outcomes among men. Dots indicate point estimates in control-group standard deviation units, and bars 90% (thick line) and 95% (thin line) parametric confidence intervals. p-values stem from randomization inference. Tests are two-tailed for “Gender Identity” and “Warmth Transgender”, one-tailed for other outcomes. Outcomes are standardized using the control-group mean and standard deviation. Table A5 in the Supplementary Information presents the results in tabular form, section A.4 provides estimation details, and section D the question wording.

3 Discussion

Our results permit different interpretations. One is that our treatment simply did not threaten male identities and thus cannot speak to whether identity threats affect downstream outcomes like out-group attitudes. Alternatively, the treatment may have threatened identities in ways we fail to capture. First, it may have increased identity commitment among some men while decreasing it among others, yielding a null average effect, though this seems less plausible given the apparent absence of effects across all pre-registered sub-groups. Second, our study may be underpowered: Table A7 shows we could detect effects of 0.235–0.272 control group standard deviations, yet many interventions produce smaller effects (Kraft 2020; Lovakov and Agadullina 2021). Third, our single measure of our first stage outcome—gender identity salience—may not fully capture how men experience or express identity threat.

Assuming our treatment created identity threat, the apparent absence of downstream effects would suggest the mechanisms implied by social identity theory may be less powerful than assumed—though, power limitations raise the possibility that downstream effects were too subtle for detection as well.

Given these observationally equivalent possibilities, we hesitate to draw strong substantive conclusions from our null results. Instead, we focus on how to design experiments that are more informative. Moreover, prior research successfully induces male identity threat, even if using deception and without examining our downstream outcomes. We therefore do not abandon the underlying research agenda but conclude by outlining suggestions for developing more effective non-deceptive identity threat treatments.

3.1 Lessons for Experimental Design

A first perhaps obvious lesson is the need for larger samples enabling detection of smaller effects. This consideration is especially important for treatments carrying psychological risks. Researchers studying identity threat must design manipulations strong enough to generate measurable effects yet subtle enough to avoid harming participants. Similar trade-offs arise in survey experiments examining negative emotions (Searles and Mattes 2015; Webster and Albertson 2022), misinformation exposure (Murphy and Greene 2023), or polarizing messages (Aruguete et al. 2021). A commitment to non-deceptive treatments makes it even more likely that interventions will be subtle. Larger samples help balance ethical restraint with inferential power and disentangle observationally equivalent explanations for null findings.

A second lesson concerns credible manipulation checks. Our reliance on a single gender identity strength measure limits our ability to determine whether our treatment failed to induce identity threat or whether identity threat didn’t affect downstream outcomes. Additional post-treatment measures of masculinity salience and perceived threat would have helped us distinguish more clearly between these possibilities. This limitation is not unique to our study. Researchers often infer that a manipulation successfully activated a psychological mechanism when observing effects on final outcomes (Fiedler et al. 2021). Our null findings illustrate this logic’s limits. Without independent, reliable mechanism measures, interpreting null results becomes difficult.

3.2 Lessons for the Study of Male Identity Threat

A comparison with existing deceptive treatments suggests several ways to improve our non-deceptive manipulation. First, existing treatments judge men against a benchmark. Willer et al. (2013) inform men their gendered knowledge quiz score falls within ranges atypical for men and typical for women. Maass et al. (2003) tell men their gendered self-description score falls within ranges typical for female students. Dahl et al. (2015) inform men they underperformed relative to a female teammate on a test predicting “success in action-oriented and competitive environments.”

We informed respondents they failed the logical reasoning task without mentioning a benchmark. We sought to create an implicit performance standard through the introductory screen that connects being male, logical reasoning and financial success. But this prime was more subtle than the explicit comparisons in existing studies. Moreover, our respondents may not view business elites as a reference group. Feedback on performance relative to the average man—or, more potently, woman—may have been more effective.

To avoid deception, we would require information about average men’s and women’s performance on our task. While impossible with our one-shot survey, future work could conduct two surveys: one eliciting the performance distribution, and another assessing effects of truthful feedback about participants’ position in this distribution. Moreover, we could not truthfully tell any participant they performed below average, since the average man and woman both failed our task. The task’s difficulty enhanced statistical power but may have undermined the expectation that men perform well, contributing to the ineffectiveness of the treatment. Future work may use multiple tasks of varying difficulty to generate semi-continuous performance measures balancing power concerns with truthful feedback requirements.

Second, logical reasoning may not connect closely enough with masculinity to create identity threat, and finance may be too removed from respondents’ lives. Other non-deceptive treatments targeting the economic dimension of masculinity are possible. Dahl et al. (2015) design an effective—though deceptive—treatment based on a task framed as predictive of “success in action-oriented and competitive environments.” Future work could develop non-deceptive treatments using tasks that can be truthfully framed in this way. For example, respondents could report their income and randomly receive feedback on how their income compares to the average man or woman in their industry. This approach may target a dimension respondents care about while also providing new information. In short, targeting other economic aspects of masculinity is a fruitful avenue for future work. The ideal dimension is strongly linked to masculinity, important to respondents yet not so salient they’re fully aware of their relative performance, weakly correlated with masculine identity, and can be elicited in a survey for comparison with reference groups.

Designing non-deceptive identity threat treatments requires additional descriptive research. We designed this treatment deductively, with pre-testing limited to assessing task difficulty among few respondents. Qualitative work and surveys could elicit which masculinity dimensions matter most to respondents, men’s and women’s performance along these dimensions, how performance correlates with gender identities, respondents’ beliefs about reference groups, and reference group importance. While assessing the effectiveness of identity threat treatments ultimately requires well-powered experiments, descriptive research can help design treatments likely to be effective.

1 Supplementary Information

Supplementary material

Acknowledgements

We thank the Weidenbaum Center on the Economy, Government, and Public Policy at WashingtonUniversity in St. Louis for giving us the opportunity to run this experiment. The pre-analysis plan forthis study can be found at https://osf.io/8nvkg. The data and replication materials are available athttps://doi.org/10.7910/DVN/BVEAIR.

References

American National Election Studies (2022). ANES cumulative data file 1948–2020. https://electionstudies.org/data-center/anes-time-series-cumulative-data-file/ →

Anduiza, E., & Rico, G. (2024). Sexism and the far-right vote: The individual dynamics of gender backlash. American Journal of Political Science, 68(2), 478–493. →

Aruguete, N., Calvo, E., Scartascini, C., & Ventura, T. (2021). Trustful voters, trustworthy politicians: A survey experiment on the influence of social media in politics. IDB-WP-1169. Inter-American Development Bank. Department of Research and Chief Economist. https://doi.org/10.18235/0003389 →

Autor, D., Dorn, D., Hanson, G., & Majlesi, K. (2020). Importing political polarization? The electoral consequences of rising trade exposure. American Economic Review, 110(10), 3139–3183. →

Baccini, L., & Weymouth, S. (2021). Gone for good: Deindustrialization, white voter backlash, and US presidential voting. American Political Science Review, 115(2), 550–567. →

Becker, S., Fetzer, T., & Novy, D. (2017). Who voted for Brexit? A comprehensive district-level analysis. Economic Policy, 32(92), 601–650. →

Berdahl, J., Cooper, M., Glick, P., Livingston, R., & Williams, J. (2018). Work as a masculinity contest. Journal of Social Issues, 74(3), 422–448. →

Broockman, D., Kalla, J., & Westwood, S. (2023). Does affective polarization undermine democratic norms or accountability? Maybe not. American Journal of Political Science, 67(3), 808–828. →

Clark, R., Khoban, R., & Zucker, N. (2022). Breadwinner backlash: The gendered effects of industrial decline. American Journal of Political Science 1–19. a, b

Colantone, I., & Stanig, P. (2018a). Global competition and Brexit. American Political Science Review, 112(2), 201–218. →

Colantone, I., & Stanig, P. (2018b). The trade origins of economic nationalism: Import competition and voting behavior in Western Europe. American Journal of Political Science, 62(4), 936–953. →

Dahl, J., Vescio, T., & Weaver, K. (2015). How threats to masculinity sequentially cause public discomfort, anger, and ideological dominance over women. Social Psychology, 46(4), 242–254. a, b, c, d

Dippel, C., Gold, R., Heblich, S., & Pinto, R. (2022). The effect of trade on workers and voters. The Economic Journal, 132(641), 199–217. →

Elsbach, K. (2003). Relating physical environment to self-categorizations: Identity threat and affirmation in a non-territorial office space. Administrative Science Quarterly, 48(4), 622–654. →

Elsbach, K., & Kramer, R. (1996). Members’ responses to organizational identity threats: Encountering and countering the Business Week rankings. Administrative Science Quarterly, 41(3), 442–476. →

Fiedler, K., McCaughey, L., & Prager, J. (2021). Quo vadis, methodology? The key role of manipulation checks for validity control and quality of science. Perspectives on Psychological Science, 16(4), 816–826. →

Fisher, R. (1935). The design of experiments. Edinburgh: Oliver and Boyd. →

Gerber, A., & Green, D. (2012). Field experiments: design, analysis, and interpretation. New York: W. W. Norton & Company. →

Gidron, N., & Hall, P. (2017). The politics of social status: Economic and cultural roots of the populist right. The British Journal of Sociology, 68, S57–S84. a, b

Guiso, L., Herrera, H., Morelli, M., & Sonno, T. (2024). Economic insecurity and the demand for populism in Europe. Economica, 91(362), 588–620. →

Kraft, M. (2020). Interpreting effect sizes of education interventions. Educational Researcher, 49(4), 241–253. →

Lin, W. (2013). Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique. The Annals of Applied Statistics, 7(1), 295–318. →

Lovakov, A., & Agadullina, E. (2021). Empirically derived guidelines for effect size interpretation in social psychology. European Journal of Social Psychology, 51(3), 485–504. →

Ludwig, J., Kling, J., & Mullainathan, S. (2011). Mechanism experiments and policy evaluations. Journal of Economic Perspectives, 25(3), 17–38. →

Maass, A., Cadinu, M., Guarnieri, G., & Grasselli, A. (2003). Sexual harassment under social identity threat: The computer harassment paradigm. Journal of Personality and Social Psychology, 85(5), 853–870. a, b, c, d

Murphy, G., & Greene, C. (2023). Conducting ethical misinformation research: Deception, dialogue, and debriefing. Current Opinion in Psychology, 54, 101713 →

Norris, Pippa and Ronald Inglehart. 2016. “Trump, Brexit, and the rise of populism: Economic have-nots and cultural backlash.” Harvard JFK School of Government Faculty Working Papers Series pp. 1–52. →

Rudman, L., & Fairchild, K. (2004). Reactions to counterstereotypic behavior: The role of backlash in cultural stereotype maintenance. Journal of Personality and Social Psychology, 87(2), 157–176. a, b

Sambanis, N., & Shayo, M. (2013). Social identification and ethnic conflict. American Political Science Review, 107(2), 294–325. →

Searles, K., & Mattes, K. (2015). It’s a mad, mad world: Using emotion inductions in a survey. Journal of Experimental Political Science, 2(2), 172–182. →

Shayo, M. (2009). A model of social identity with an application to political economy: Nation, class, and redistribution. American Political Science Review, 103(2), 147–174. a, b

Smith, T., Davern, M., Bautista, R., Freese, J., & Morgan, S. (2019). General social surveys, 1972–2018: Cumulative codebook. https://gss.norc.org/content/dam/gss/get-documentation/pdf/codebook/GSS_Codebook.pdf →

Tajfel, H. (2010). Social identity and intergroup relations. Vol. 7. Cambridge University Press. →

Turner, J. (1975). Social comparison and social identity: Some prospects for intergroup behaviour. European Journal of Social Psychology, 5(1), 1–34. →

Valentino, N., Wayne, C., & Oceno, M. (2018). Mobilizing sexism: The interaction of emotion and gender attitudes in the 2016 US presidential election. Public Opinion Quarterly, 82(S1), 799–821. →

Vandello, J., Bosson, J., Cohen, D., Burnaford, R., & Weaver, J. (2008). Precarious manhood. Journal of Personality and Social Psychology, 95(6), 1325–1339. a, b, c

Verlet, A. (2021). Classic brain teasers from real-life interviews. https://www.simtrade.fr/blog_simtrade/classic-brain-teasers-real-life-interviews/ →

Vescio, T., Schermerhorn, N., Gallegos, J., & Laubach, M. (2021). The affective consequences of threats to masculinity. Journal of Experimental Social Psychology, 97, 104195 a, b, c

Vescio, T., Schermerhorn, N., Lewis, K., Yamaguchi-Pedroza, K., & Loviscky, A. (2025a). Masculinity threats sequentially arouse public discomfort, anger, and positive attitudes toward sexual violence. Personality and Social Psychology Bulletin, 51(1), 96–109. a, b

Vescio, T., Yamaguchi-Pedroza, K., Schermerhorn, N., & Loviscky, A. (2025b). Threats to masculinity evoke status-quo-reinforcing racism, xenophobia, and Islamophobia. Frontiers in Social Psychology, 3, 1494928 →

Webster, S., & Albertson, B. (2022). Emotion and politics: Noncognitive psychological biases in public opinion. Annual Review of Political Science, 25, 401–418. →

Weidenbaum Center (2023). The American Social Survey (TASS), Wave 11: policy and current events winter 2023 National survey administered via NORC AmeriSpeak. →

Willer, R., Rogalin, C., Conlon, B., & Wojnowicz, M. (2013). Overdoing gender: A test of the masculine overcompensation thesis. American Journal of Sociology, 118(4), 980–1022. a, b, c, d

Zippia. Investment Banker Demographics and Statistics in the US. Last modified September 9, 2022. https://www.zippia.com/investment-banking-analyst-jobs/demographics/ →