Momentum and stochastic momentum for stochastic gradient, Newton, proximal point and subspace descent methods
Nicolas Loizou, Peter Richtárik
Topics & Concepts
Iterated functionStochastic gradient descentMomentum (technical analysis)MathematicsRate of convergenceStochastic optimizationApplied mathematicsBall (mathematics)Stochastic approximationMathematical optimizationMathematical analysisComputer scienceFinanceArtificial neural networkChannel (broadcasting)Computer networkKey (lock)EconomicsComputer securityMachine learningStochastic Gradient Optimization TechniquesSparse and Compressive Sensing TechniquesAdvanced Bandit Algorithms Research