Linearly-Involved Moreau-Enhanced-Over-Subspace Model: Debiased Sparse Modeling and Stable Outlier-Robust Regression
Masahiro Yukawa, Hiroyuki Kaneko, Kyohei Suzuki, Isao Yamada
Abstract
We present an efficient mathematical framework to derive promising methods that enjoy “enhanced” desirable properties. The popular minimax concave penalty for sparse modeling subtracts, from the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> norm, its Moreau envelope, inducing nearly unbiased estimates and thus yielding considerable performance enhancements. To extend it to underdetermined linear systems, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">the projective minimax concave penalty</i> , which leads to “enhanced” sparseness over the input subspace. We also present a promising regression method which has an “enhanced” robustness and substantial stability by distinguishing outlier and noise explicitly. The proposed framework, named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">the linearly-involved Moreau-enhanced-over-subspace (LiMES) model</i> , encompasses those two specific examples as well as two others: stable principal component pursuit and robust classification. The LiMES function involved in the model is an “additively nonseparable” weakly convex function, while the ‘inner’ objective function to define the Moreau envelope is “separable”. This <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mixed nature of separability and nonseparability</i> allows an application of the LiMES model to the underdetermined case with an efficient algorithmic implementation. Two linear/affine operators play key roles in the model: one corresponds to the projection mentioned above and the other takes care of robust regression/classification. A necessary and sufficient condition for convexity of the smooth part of the objective function is studied. Numerical examples show the efficacy of LiMES in applications to sparse modeling and robust regression.