Detecting Meaningful Clusters From High-Dimensional Data: A Strongly Consistent Sparse Center-Based Clustering Approach
Saptarshi Chakraborty, Swagatam Das
Abstract
In context to high-dimensional clustering, the concept of <i>feature weighting</i> has gained considerable importance over the years to capture the relative degrees of importance of different features in revealing the cluster structure of the dataset. However, the popular techniques in this area either fail to perform feature selection or do not preserve the simplicity of Lloyd’s heuristic to solve the <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -means problem and the like. In this paper, we propose a Lasso Weighted <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -means ( <inline-formula><tex-math notation="LaTeX">$LW$</tex-math></inline-formula> - <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -means) algorithm, as a simple yet efficient sparse clustering procedure for high-dimensional data where the number of features ( <inline-formula><tex-math notation="LaTeX">$p$</tex-math></inline-formula> ) can be much higher than the number of observations ( <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> ). The <inline-formula><tex-math notation="LaTeX">$LW$</tex-math></inline-formula> - <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -means method imposes an <inline-formula><tex-math notation="LaTeX">$\ell _1$</tex-math></inline-formula> regularization term involving the feature weights directly to induce feature selection in a sparse clustering framework. We develop a simple block-coordinate descent type algorithm with time-complexity resembling that of Lloyd’s method, to optimize the proposed objective. In addition, we establish the strong consistency of the <inline-formula><tex-math notation="LaTeX">$LW$</tex-math></inline-formula> - <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -means procedure. Such an analysis of the large sample properties is not available for the conventional sparse <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -means algorithms, in general. <inline-formula><tex-math notation="LaTeX">$LW$</tex-math></inline-formula> - <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -means is tested on a number of synthetic and real-life datasets and through a detailed experimental analysis, we find that the performance of the method is highly competitive against the baselines as well as the state-of-the-art procedures for center-based high-dimensional clustering, not only in terms of clustering accuracy but also with respect to computational time.