Rethinking Embedded Unsupervised Feature Selection: A Simple Joint Approach
Heng Chang, Jun Guo, Wenwu Zhu
Abstract
Recently, various embedded methods for unsupervised feature selection have been put forward. However, most of them adopt a two-step strategy, i.e., selecting <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> top-ranked dimensions according to a learned order of all features, then conducting K-means clustering for evaluation. This commonly used strategy usually results in a group of sub-optimal features, because the selected <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> top-ranked features are seldom the desired top- <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> dimensions. To address this problem, we rethink the two steps in a joint manner and propose a simple yet effective approach called <b>U</b> nsupervised <b>F</b> eature <b>S</b> election with <b>S</b> eparability ( <b>UFS<inline-formula><tex-math notation="LaTeX">$^{\mathbf{2}}$</tex-math><alternatives><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msup><mml:mrow/><mml:mn mathvariant="bold">2</mml:mn></mml:msup></mml:math><inline-graphic xlink:href="chang-ieq4-3178715.gif" xmlns:xlink="http://www.w3.org/1999/xlink"/></alternatives></inline-formula></b> ) to simultaneously select features and cluster data. More specifically, a binary vector is seamlessly integrated into K-means to select an exact number of features for clustering. Different from previous embedded methods involving <inline-formula><tex-math notation="LaTeX">$l_{2,1}$</tex-math></inline-formula> -norm, our joint model explicitly uses the parameter <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> (i.e., the number of selected features). Afterwards, a customized term for the binary vector is designed to maximize the separability among selected feature dimensions. In order to solve the formulated 0-1 integer programming problem, an iterative algorithm is developed. Finally, we evaluate the proposed approach extensively on different datasets. Despite the relative simplicity, UFS <inline-formula><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> remarkably and generally outperforms state-of-the-art baselines.