Model Based Survey Design Using Logits: Estimating Lost Statistical Power from Random Alternative Sampling

Abstract

McFadden’s random alternative sampling conditional logit estimator permits researchers and survey designers to estimate a random utility choice model observing information about a subset of available alternatives. We quantify the extent to which a small sample size and a reduction in the number of sample alternatives lead to bias and loss of statistical power. The sample size must be small and choice probabilities must be weakly correlated with choice characteristics for there to be substantial bias and low power. Finally, we find that there is a sharply decreasing marginal gain from increasing the number of sampled alternatives. We provide an empirical example on the choice of health insurance plans that verifies our conclusions.