Multivariate Tests for Phase Capacity

Abstract

To combat the potentially detrimental effects of nonresponse, most surveys repeatedly follow-up with nonrespondents, often targeting a response rate or predetermined number of completes. Each additional recruitment attempt generally brings in a new wave of data, but returns gradually diminish over the course of a static data collection protocol. This is because each subsequent wave tends to contain fewer and fewer new responses, thereby rendering smaller and smaller changes in point estimates. Consequently, point estimates calculated from the accumulating data begin to stabilize. This is the notion of phase capacity, suggesting some form of design change is warranted, such as switching modes, increasing the incentive, or simply discontinuing nonrespondent follow-up. Phase capacity testing methods that have appeared in the literature to date are generally only applicable to a single point estimate. It is unclear how to proceed if conflicting results are obtained following independent tests on two or more point estimates. The purpose of this paper is to introduce two multivariate phase capacity tests, each designed with the aim of providing a universal, yes-or-no phase capacity determination for a battery of point estimates. The two competing methods’ performance is compared via simulation and application using data from the 2011 Federal Employee Viewpoint Survey.