Benign Non-Convex Optimization Techniques for Training Neuro-Inspired Architectures
Vandana Roy, S Vinod Kumar, Vijilius Helena Raj, Sorabh Lakhanpal, Dinesh Kumar Yadav, Rafil A. Alzuhairi
Abstract
In non-convex optimization contexts, neuronal network optimization is difficult yet crucial. The novel Adaptive Ensemble of Benign Non-Convex Optimization Algorithms (ABNOA) improves neuron-inspired structure training. SGD is trapped in local minima when there are non-convex loss surfaces; therefore, it doesn’t always work. ABNOA solves these issues by dynamically selecting, combining, and fine-tuning optimization algorithms. This builds a versatile, dependable optimization mechanism. We tested ABNOA against typical speed evaluation methods using several metrics. Our analysis indicated that ABNOA performs better in several key areas. It lets you choose a method based on the optimization circumstances. This accelerates convergence. ABNOA’s Adaptive Parameter Tuning (APT) fine-tunes hyperparameters in real time, improving algorithm efficiency and flexibility. Accelerating convergence improves training efficiency. It can manage complex loss landscapes, escape local minima, and finetune hyperparameters, giving it a full solution for non-convex neural network improvement for students and practitioners. According to the findings, ABNOA might transform complicated neural network training and advance AI.