Weighted Bayesian bootstrap for scalable posterior distributions
Michael A. Newton, Nicholas G. Polson, Jianeng Xu
Abstract
Abstract We introduce and develop a weighted Bayesian bootstrap (WBB) for machine learning and statistics. WBB provides uncertainty quantification by sampling from a high dimensional posterior distribution. WBB is computationally fast and scalable using only off‐the‐shelf optimization software. First‐order asymptotic analysis provides a theoretical justification under suitable regularity conditions on the statistical model. We illustrate the proposed methodology in regularized regression, trend filtering and deep learning and conclude with directions for future research.
Topics & Concepts
Bayesian probabilityComputer scienceScalabilityPosterior probabilityArtificial intelligenceSampling (signal processing)Uncertainty quantificationAlgorithmMachine learningBayesian inferenceMathematicsStatistical learningData miningBayesian statisticsPattern recognition (psychology)Statistical modelBayes' theoremBayesian networkImportance samplingScale (ratio)Statistical analysisRecursive Bayesian estimationBayesian experimental designMathematical optimizationProbability distributionBayesian linear regressionInterpretabilityGaussian Processes and Bayesian InferenceMarkov Chains and Monte Carlo MethodsStochastic Gradient Optimization Techniques