AdaDerivative optimizer: Adapting step-sizes by the derivative term in past gradient information
Weidong Zou, Yuanqing Xia, Weipeng Cao
Topics & Concepts
Computer scienceSmoothingOvershoot (microwave communication)Term (time)Key (lock)Artificial intelligenceGradient descentArtificial neural networkExponential smoothingReliability (semiconductor)Image (mathematics)Object (grammar)Node (physics)Noise (video)AlgorithmPattern recognition (psychology)Computer visionPower (physics)PhysicsComputer securityEngineeringStructural engineeringQuantum mechanicsTelecommunicationsAdvanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningMachine Learning and ELM