Robust Standard Gradient Descent Algorithm for ARX Models Using Aitken Acceleration Technique
Jing Chen, Min Gan, Quanmin Zhu, Pritesh Narayan, Yanjun Liu
Abstract
A robust standard gradient descent (SGD) algorithm for ARX models using the Aitken acceleration method is developed. Considering that the SGD algorithm has slow convergence rates and is sensitive to the step size, a robust and accelerative SGD (RA-SGD) algorithm is derived. This algorithm is based on the Aitken acceleration method, and its convergence rate is improved from linear convergence to at least quadratic convergence in general. Furthermore, the RA-SGD algorithm is always convergent with no limitation of the step size. Both the convergence analysis and the simulation examples demonstrate that the presented algorithm is effective.
Topics & Concepts
AccelerationConvergence (economics)Rate of convergenceAlgorithmGradient descentMathematicsComputer scienceMathematical optimizationArtificial intelligenceArtificial neural networkPhysicsChannel (broadcasting)Economic growthEconomicsClassical mechanicsComputer networkAdvanced Adaptive Filtering TechniquesSparse and Compressive Sensing TechniquesTarget Tracking and Data Fusion in Sensor Networks