Revisiting <i>L</i> <sub>2,1</sub>-Norm Robustness With Vector Outlier Regularization
Bo Jiang, Chris Ding
Abstract
In many real-world applications, data usually contain outliers. One popular approach is to use the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm function as a robust loss/error function. However, the robustness of the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm function is not well understood so far. In this brief, we propose a new vector outlier regularization (VOR) framework to understand and analyze the robustness of the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm function. Our VOR function defines a data point to be the outlier if it is outside a threshold with respect to a theoretical prediction, and regularizes it, i.e., pull it back to the threshold line. Thus, in the VOR function, how far an outlier lies away from its theoretical predicted value does not affect the final regularization and analysis results. One important aspect of the VOR function is that it has an equivalent continuous formulation, based on which we can prove that the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm function is the limiting case of the proposed VOR function. Based on this theoretical result, we thus provide a new and intuitive explanation for the robustness property of the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm function. As an example, we use the VOR function to matrix factorization and propose a VOR principal component analysis (PCA) (VORPCA). We show some benefits of VORPCA on data reconstruction and clustering tasks.