Evaluating Item Content and Scale Characteristics Using a Pretrained Neural Network Model

Authors

  • Jeffrey Stanton Syracuse University
  • Angela Ramnarine-Rieks Syracuse University
  • Yisi Sang Syracuse University

DOI:

https://doi.org/10.18148/srm/2024.v18i2.8240

Keywords:

Cronbach’s alpha, Answer behavior, Emotion prediction, Microphone, Natural Language Processing, Open-ended questions, Smartphone, Voice recordings, Neural network, rating scale

Abstract

Multi-item scales are widely used in social research. The psychometric characteristics of a scale and the successful use of a scale in research depend in part on item wording. This article demonstrates a method for using natural language processing (NLP) tools to assist with the item development process, by showing that numeric embedding representations of items are useful in predicting the characteristics of a scale. NLP comprises a set of algorithmic techniques for analysing words, phrases, and larger units of written language. We used NLP tools to create and analyse semantic summaries of the item texts for n=386 previously published multi-item scales. Results showed that semantic representations of items connect to scale characteristics such as Cronbach's alpha internal consistency.

Downloads

Published

2024-08-07

How to Cite

Stanton, J., Ramnarine-Rieks, A., & Sang, Y. (2024). Evaluating Item Content and Scale Characteristics Using a Pretrained Neural Network Model. Survey Research Methods, 18(2), 153–165. https://doi.org/10.18148/srm/2024.v18i2.8240

Issue

Section

Articles

Similar Articles

1 2 3 4 5 6 7 8 9 10 > >> 

You may also start an advanced similarity search for this article.