Bayesian Estimation of Single-Test Reliability Coefficients
Julius M. Pfadt, Don van den Bergh, Klaas Sijtsma, Morten Moshagen, Eric‐Jan Wagenmakers
Abstract
Popular measures of reliability for a single-test administration include coefficient a, coefficient k 2 , the greatest lower bound (glb), and coefficient x. First, we show how these measures can be easily estimated within a Bayesian framework. Specifically, the posterior distribution for these measures can be obtained through Gibbs samplingfor coefficients a, k 2 , and the glb one can sample the covariance matrix from an inverse Wishart distribution; for coefficient x one samples the conditional posterior distributions from a single-factor CFA-model. Simulations show thatunder relatively uninformative priorsthe 95% Bayesian credible intervals are highly similar to the 95% frequentist bootstrap confidence intervals. In addition, the posterior distribution can be used to address practically relevant questions, such as "what is the probability that the reliability of this test is between .70 and .90?", or, "how likely is it that the reliability of this test is higher than .80?" In general, the use of a posterior distribution highlights the inherent uncertainty with respect to the estimation of reliability measures.