Pseudo-Opinions in Online Surveys: Evidence to Recontextualize the Imputed Meaning Hypothesis

Abstract

Pseudo-opinions and nonattitudes describe when survey respondents give substantive answers to things that they know little to nothing about. They can be seen as a threat to the validity and reliability of survey data as they represent deviations from the ‘true’ answer: “I don’t know”. Two competing hypotheses are often put forth to explain pseudo-opinions. The mental coin-flip hypothesis is associated with satisficing response behavior and states respondents choose an answer at random while the imputed meaning hypothesis is associated more with optimizing and states that respondents attempt to find meaning in the unfamiliar question and answer accordingly. While the imputed meaning hypothesis is currently favored in the literature, we argue it is implausible in online surveys employing ‘professional’ panelists. Namely, the lack of an interviewer should reduce the pressure to look informed, and extrinsic incentives should reduce the respondent’s motivation. Therefore, we believe online surveys are particularly susceptible to a satisficing style of pseudo-opinions. To investigate this, we conduct a large online survey and use response latencies as an indicator of satisficing along with fictitious issues, i.e., questions about nonexistent things. The results of fixed effects probit regression models confirm that the probability for a pseudo-opinion tends to be higher for very fast (satisficing) responses. However, the inclusion of the squared response latency term reveals that pseudo-opinions are predicted by both very fast and very slow average responses. This shows that very fast responses should be scrutinized. Comparatively slow answers indicate respondents trying to put in the effort to give a valid answer. Two experimental conditions reveal instructions to take one’s time and deliberate before answering reduce pseudo-opinions slightly, and the inclusion of an explicit “don’t know” category greatly reduces pseudo-opinions.