The Three-Step Test-Interview (TSTI): An observation-based method for pretesting self-completion questionnaires
Keywords: cognitive pre-testing, self-administered questionnaire, think aloud, protocol analysis, observation
AbstractThree-Step Test-Interview (TSTI) is a method for pretesting a self-completion questionnaire by first observing actual instances of interaction between the instrument and respondents (the response process) before exploring the reasons for this behavior. The TSTI consists of the following three steps: 1. (Respondent-driven) observation of response behavior. 2. (Interviewer-driven) follow-up probing aimed at remedying gaps in observational data. 3. (Interviewer-driven) debriefing aimed at eliciting experiences and opinions. We describe the aims and the techniques of these three steps, and then discuss pilot studies in which we tested the feasibility and the productivity of the TSTI by applying it in testing three rather different types of questionnaires. In the first study, the quality of a set of questions about alcohol consumption was assessed. The TSTI proved to be productive in identifying problems that resulted from a mismatch between the ‘theory’ underlying the questions on the one hand, and features of a respondent’s actual behavior and biography on the other hand. In the second pilot study, Dutch and Norwegian versions of an attitude scale, the 20-item Illegal Aliens Scale, were tested. The TSTI appeared to be productive in identifying problems that resulted from different ‘response strategies’. In the third pilot, a two-year longitudinal study, the TSTI appeared to be an effective method for documenting processes of ‘response shift’ in repeated measurements of health-related Quality of Life (QoL).
How to Cite
Hak, T., Veer, K. van der, & Jansen, H. (2008). The Three-Step Test-Interview (TSTI): An observation-based method for pretesting self-completion questionnaires. Survey Research Methods, 2(3), 143-150. https://doi.org/10.18148/srm/2008.v2i3.1669
Copyright for articles published in this journal is retained by the authors, with first publication rights granted to the journal. By virtue of their appearance in this open access journal, users can use, reuse and build upon the material published in the journal but only for non-commercial purposes and with proper attribution.