AdaSAM: Boosting sharpness-aware minimization with adaptive learning rate and momentum for training deep neural networks
Hao Sun, Li Shen, Qihuang Zhong, Liang Ding, Shixiang Chen, Jingwei Sun, Jing Li, Guangzhong Sun, Dacheng Tao
Topics & Concepts
SpeedupComputer scienceRate of convergenceArtificial neural networkBoosting (machine learning)Momentum (technical analysis)MinificationArtificial intelligenceDeep learningAccelerationAlgorithmMachine learningPerturbation (astronomy)Deep neural networksMathematical optimizationMathematicsPhysicsComputer networkFinanceQuantum mechanicsProgramming languageChannel (broadcasting)Operating systemClassical mechanicsEconomicsAdvanced Neural Network ApplicationsStochastic Gradient Optimization TechniquesMachine Learning and ELM