Free Energy Minimization: A Unified Framework for Modeling, Inference, Learning, and Optimization [Lecture Notes]
Sharu Theresa Jose, Osvaldo Simeone
Abstract
The goal of this lecture note is to review the problem of free energy minimization as a unified framework underlying the definition of maximum entropy modeling, generalized Bayesian inference, learning with latent variables, the statistical learning analysis of generalization, and local optimization. Free energy minimization is first introduced, here and historically, as a thermodynamic principle. Then, it is described mathematically in the context of Fenchel duality. Finally, the applications to modeling, inference, learning, and optimization are covered, starting from basic principles.
Topics & Concepts
InferenceFree energy principleComputer scienceEnergy minimizationBayesian inferencePrinciple of maximum entropyGeneralizationEntropy (arrow of time)Mathematical optimizationArtificial intelligenceStatistical inferenceMinificationLatent variableContext (archaeology)Machine learningBayesian probabilityMathematicsMathematical analysisChemistryStatisticsBiologyPhysicsPaleontologyQuantum mechanicsComputational chemistryStatistical Mechanics and EntropyGaussian Processes and Bayesian InferenceProbabilistic and Robust Engineering Design