Are Final Comments in Web Survey Panels Associated with Next-Wave Attrition?

Abstract

Near the end of a web survey respondents are often asked whether they have further comments. Such final comments are usually ignored, in part because open-ended questions are challenging to analyse. We explored whether final comments are associated with next-wave attrition in survey panels. We categorized a random sample of final comments in the Longitudinal Studies for the Social Sciences (LISS) panel and Dutch Immigrant panel into one of eight categories (neutral, positive, six subcategories of negative) and regressed the indicator of next-wave attrition on comment length, comment category and socio-demographic variables. In the Immigrant panel we found shorter final comments (<30 words) are associated with increased next-wave attrition, and longer final comments (>55 words) with decreased next-wave attrition relative to making no comment. Comments about unclear survey questions quadruple the odds of attrition and “other” (uncategorized) negative comments almost double the odds of attrition. In the LISS panel, making a comment (vs. not) and comment length are not associated with attrition. However, when specifying individual comment categories, neutral comments are associated with half the odds of attrition relative to not making a comment.