More Random Than Not? A Review of the Logic of Inference in Experimental Public Administration

Abstract

The emphasis on causal inference within public administration research has spurred a proliferation of experimental studies, primarily due to the internal validity attributed to effects inferred from random assignment of treatment conditions. Still, many experimental studies include findings based not only random assignment, but also on a variety of other logics of inference. This study examines the logic of inference in experimental public administration. Drawing on a sample of experimental studies, we find that 57.5% of the findings rely solely on randomized inference, 18.8% involve an interaction between randomly and non-randomly assigned factors, and 23.7% are based on a logic unrelated to random assignment of a treatment condition. We further investigate how these “upstream” findings are interpreted in the “downstream” studies that cite them. We find that 77.4% of downstream citations use the upstream findings to support causal claims. Of those, 41.6% are not rooted in a logic based on randomization, suggesting a misalignment between the logic of inference underlying results and their utilization by citing researchers. This misalignment has important implications for the body of evidence relied upon for testing, developing, and refining theories central to public administration and management scholarship and is likely to worsen if not addressed