DC-DistADMM: ADMM Algorithm for Constrained Optimization Over Directed Graphs
Vivek Khatana, Murti V. Salapaka
Abstract
This article reports an algorithm for multiagent distributed optimization problems with a common decision variable, local linear equality, and inequality, constraints and set constraints with convergence rate guarantees. The algorithm accrues all the benefits of the alternating direction method of multipliers (ADMM) approach. It also overcomes the limitations of existing methods on convex optimization problems with linear inequality, equality, and set constraints by allowing directed communication topologies. Moreover, the algorithm can be synthesized distributively. The developed algorithm has: first, a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$O(1/k) $</tex-math></inline-formula> rate of convergence, where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> is the iteration counter, when individual functions are convex but not-necessarily differentiable, and second, a geometric rate of convergence to any arbitrary small neighborhood of the optimal solution, when the objective functions are smooth and restricted strongly convex at the optimal solution. The efficacy of the algorithm is evaluated by a comparison with state-of-the-art constrained optimization algorithms in solving a constrained distributed <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> -regularized logistic regression problem, and unconstrained optimization algorithms in solving a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> -regularized Huber loss minimization problem. Additionally, a comparison of the algorithm's performance with other algorithms in the literature that utilize multiple communication steps is provided.