Is Vague Valid? The Comparative Predictive Validity of Vague Quantifiers and Numeric Response Options

Abstract

A number of surveys, including many student surveys, rely on vague quantifiers to measure behaviors important in evaluation. The ability of vague quantifiers to provide valid information, particularly compared to other measures of behaviors, has been questioned both within both survey research generally and educational research specifically. Still, there is a dearth of research on whether vague quantifiers or numeric responses perform better in regards to validity. This study examines measurement properties of frequency estimation questions through the assessment of predictive validity, which has also been shown to be important in examining measurement properties of competing question formats. Data from the National Survey of Student Engagement (NSSE), a preeminent survey of university students, is analyzed, in which two psychometrically tested benchmark scales, active and collaborative learning and student-faculty interaction, are measured through both vague quantifier and numeric responses. Predictive validity is assessed through correlations and regression models relating both vague and numeric scales to grades in school and two education experience satisfaction measures. Results support the view that the predictive validity is higher for vague quantifier scales, and hence better measurement properties, compared to numeric responses. These results are discussed in light of other findings on measurement properties of vague quantifiers and numeric responses, suggesting that vague quantifiers may be a useful measurement tool for behavioral data, particularly when it is the relationship between variables that are of interest.