UC Davis Agricultural and Resource Economics

Dalia Ghanem, University of California, Davis

Testing Attrition Bias in Field Experiments (with Sarojini Hirshleifer and Karen Ortiz-Becerra)

Date and Location

Tuesday, November 6, 2018, 4:10 PM - 5:30 PM
ARE Library Conference Room, 4101 Social Sciences and Humanities


Attrition is a common threat to the internal validity of field experiments in economics. This paper brings insights from the nonparametric identification literature in panel models to address the question of how to formally test for attrition bias in the presence of baseline outcome data. We first conduct a systematic review of the field experiment literature in economics and find that there is no consensus on how to test for attrition bias. Of the field experiments we reviewed: 81% test for differential attrition rates, but only 59% conduct some type of test for internal validity conditional on response status. We then formally demonstrate that: (1) the differential attrition rate test does not control size in general, and (2) internal validity has a sharp testable restriction on the baseline outcome distribution. We show that this restriction implies joint (as opposed to simple) tests of distributional equality, which are not widely used in the literature. We also demonstrate that the restriction differs depending on whether the authors’ object of interest is the treatment effect for respondents or for the population selected for the evaluation. We propose randomization procedures to obtain p-values for Kolmogorov-Smirnov- and Cramer-von-Mises-type statistics of the joint hypothesis. We further extend our framework to testing internal validity in the case of stratified randomization. Finally, we include simulation experiments to illustrate the implications of these results for empirical practice.

Contact Us

2116 Social Sciences and Humanities
University of California, Davis
One Shields Avenue
Davis, CA 95616

Main Office: 530-752-1515
Student Advising Services: 530-754-9536
DeLoach Conference Room: 530-752-2916
Main Conference Room: 530-754-1850