Response Patterns in a Multi-day Diary Survey: Implications for Adaptive Survey Design

Abstract

In multi-day diary surveys, respondents make participation decisions every day. Some respondents remain committed throughout, whereas others drop out after the first few days or in the later days of the survey, leading to item nonresponse. Such item nonresponse at the day level can introduce nonresponse and underreporting error, reduce statistical power and bias survey estimates. Despite its critical influence on survey data quality, the important issue of day-level item nonresponse in diary surveys has received surprisingly little attention. This study evaluates different response patterns in a seven-day diary survey and considers how these patterns might inform adaptive designs for future diary surveys. We analyzed data from the U.S. National Household Food Acquisition and Purchase Survey (FoodAPS), a nationally representative survey designed to collect comprehensive data on household food purchases and acquisitions during one-week time periods. In total, there were 4,826 households with 14,317 individuals that responded to the survey. To evaluate how response patterns differed across respondents and across the diary period, we employed a latent class growth analysis (LCGA), which enables the identification of different groups of respondents based on their reporting patterns. Our analysis identified five classes of respondents, ranging from highly-motivated respondents to those exhibiting minimal effort. We also compared the identified classes in terms of covariate profiles and distributions on key FoodAPS outcomes. Respondents who showed low-motivated response patterns were found to record fewer events in the diary for the key variables. To inform adaptive designs in future diary data collections, we examined respondent characteristics that were known before the diary portion of the survey and significantly associated with class membership. Several respondent characteristics (e.g., low education) and paradata features (e.g., longer initial interviews) were linked with the probability of having low-motivation response patterns. Our findings have implications for future designs of multi-day diary surveys.