Visual Design, Order Effects, and Respondent Characteristics in a Self-Administered Survey

Abstract

Recent survey design research has shown that small changes in the structure and visual layout of questions can affect respondents' answers. While the findings have provided strong evidence of such effects, they are limited by the homogeneity of their samples, in that many of these studies have used random samples of college students. In this paper, we examine the effects of seven experimental alterations in question format and visual design using data from a general population survey that allows us to examine the effects of demographic differences among respondents. Results from a 2005 random sample mail survey of 1,315 households in a small metropolitan region of the United States suggest that the visual layout of survey questions affects different demographic groups in similar ways.