Evaluating Item Content and Scale Characteristics Using a Pretrained Neural Network Model
DOI:
https://doi.org/10.18148/srm/2024.v18i2.8240Keywords:
Cronbach’s alpha, Answer behavior, Emotion prediction, Microphone, Natural Language Processing, Open-ended questions, Smartphone, Voice recordings, Neural network, rating scaleAbstract
Multi-item scales are widely used in social research. The psychometric characteristics of a scale and the successful use of a scale in research depend in part on item wording. This article demonstrates a method for using natural language processing (NLP) tools to assist with the item development process, by showing that numeric embedding representations of items are useful in predicting the characteristics of a scale. NLP comprises a set of algorithmic techniques for analysing words, phrases, and larger units of written language. We used NLP tools to create and analyse semantic summaries of the item texts for n=386 previously published multi-item scales. Results showed that semantic representations of items connect to scale characteristics such as Cronbach's alpha internal consistency.Published
2024-08-07
How to Cite
Stanton, J., Ramnarine-Rieks, A., & Sang, Y. (2024). Evaluating Item Content and Scale Characteristics Using a Pretrained Neural Network Model. Survey Research Methods, 18(2), 153–165. https://doi.org/10.18148/srm/2024.v18i2.8240
Issue
Section
Articles
License
Copyright (c) 2024 Jeffrey Stanton, Angela Ramnarine-Rieks, Yisi Sang
![Creative Commons License](http://i.creativecommons.org/l/by/4.0/88x31.png)
This work is licensed under a Creative Commons Attribution 4.0 International License.