When possible, report a Fisher-exact <i>P</i> value and display its underlying null randomization distribution
Marie‐Abèle Bind, Donald B. Rubin
Abstract
Significance Statistical analyses of randomized experiments often rely on asymptotic P values instead of using the actual randomization procedure that led to the observed data. Fisher-exact and asymptotic P values can differ dramatically: The former should be preferred because it is calculated using the exact null randomization distribution, which, in small samples, can substantially differ from its approximating Student’s t distribution. Moreover, we may learn something scientifically interesting from examining the shape of the null randomization distribution.
Topics & Concepts
RandomizationNull (SQL)Null hypothesisNull distributionp-valueExact testMathematicsStatisticsDistribution (mathematics)Statistical hypothesis testingStatistical significanceApplied mathematicsComputer scienceRandomized controlled trialData miningMedicineTest statisticMathematical analysisSurgeryStatistical Methods and InferenceStatistical Methods in Clinical TrialsStatistical Methods and Bayesian Inference