Harmonizing Data From Open-Ended and Closed-Ended Quantity Questions with Observed Score Equating

Abstract

Many surveys ask respondents about manifest quantities, such as income, age, weight, or their height. Surveys often either use open-ended questions, where respondents report the quantity directly as an integer value (e.g., “56”), or closed-ended quantity questions where respondents select from a set of discrete interval response options (e.g., “51 to 100”). Quantity data gathered with different response schemes thus becomes hard to compare or to harmonize to be used in integrative analyses. We compare two approaches to harmonizing quantity question data. Firstly, the widely used middle of category (MOC) interpolation. Secondly, Observed Score Equating in a Random Groups Design (OSE-RG). OSE-RG is originally an approach to harmonize measures for latent constructs. However, the equipercentile OSE-RG algorithm lends itself well to quantity questions. To test the performance of both algorithms, we gathered experimental data (N = 3484) on the number of books possessed as an example quantity, where we varied the quantity-question response scheme. We show that OSE-RG often outperforms or at least matches MOC when harmonizing closed-ended questions towards an open-ended format, or when harmonizing different closed-ended response formats amongst each other. Notably, OSE-RG is also less susceptible to response biases induced by different close-ended interval response schemes.