Estimating the number of clusters via a corrected clustering instability
Jonas M B Haslbeck, Dirk U. Wulff
Abstract
Abstract We improve instability-based methods for the selection of the number of clusters k in cluster analysis by developing a corrected clustering distance that corrects for the unwanted influence of the distribution of cluster sizes on cluster instability. We show that our corrected instability measure outperforms current instability-based measures across the whole sequence of possible k , overcoming limitations of current insability-based methods for large k . We also compare, for the first time, model-based and model-free approaches to determining cluster-instability and find their performance to be comparable. We make our method available in the R-package .
Topics & Concepts
InstabilityCluster analysisCluster (spacecraft)Stability (learning theory)Measure (data warehouse)Computer scienceSequence (biology)Selection (genetic algorithm)MathematicsStatistical physicsStatisticsData miningPhysicsArtificial intelligenceMachine learningChemistryMechanicsProgramming languageBiochemistryAdvanced Clustering Algorithms ResearchBayesian Methods and Mixture ModelsData Analysis with R