The Response Entropy Index: Comparative Assessment of Performance and Cultural Bias across Indices of Careless Responding

Abstract

The response entropy (RE) index calculates the balance of proportions of values endorsed by participants on Likert-scaled surveys, and is proposed as a new method for flagging careless response patterns.  In the first study, performance of the RE index was compared to other commonly used post hoc indices for detecting careless responding (CR) such as the Mahalanobis distance (MD) and the psychometric synonym (PS) index.  Three different types of Bogus Scores (BS) were generated: 1) uniform random values produced by computer (n = 100); 2) normally distributed random values produced by computer (n = 100); and 3) purposefully careless responses produced by human participants (n = 100).  The BS data were then implanted in a true, cleaned social science dataset (n = 500).  Multinomial logistic regression determined that the RE index made independent contributions from other indices to the prediction of BS.  Latent variable analyses suggest that the variability type RE index may be tapping distinct constructs from regression type indices such as the PS index.  In study 2, potential cultural bias in CR indices was examined with a true social science dataset (n = 302) comprised of racially diverse participants.  Unlike other post hoc indices of CR, the RE index was unrelated to participant race.  Further analyses demonstrated that racial differences on other indices of CR could be accounted for by culturally different styles of survey responding.  For example, Asian participants’ higher MD scores relative to White participants’ was mediated by a culturally specific acquiescent survey response style.