Self-Organizing Map algorithm as a tool for outlier detection
Agnieszka Nowak - Brzezińska, Czesław Horyń
Abstract
The research addresses the problem of outlier detection using the (unsupervised) self-organizing map (S OM) algorithm introduced by Kohonen [1]. Since the outliers can indicate something scientifically interesting, we need to analyze these issue deeply. In this research, we do not assume that after outliers are detected, they are removed. We hand them over to domain experts asking for further exploration. We have experience applying the LOF (local outlier factor) algorithm to detect outliers in real data. Now we wanted to check how a new algorithm, an S OM algorithm, will behave. We compare the results of outlier detection process using the LOF algorithm with the results of using the S OM algorithm. We were interested whether a data type significantly influences the efficiency of the process of outlier detection. It is obvious that qualitative data is much more difficult to explore than quantitative data. We chose two various dataset (quantitive and qualitative). We kept changing the learning parameters: a learning rate, a neighborhood radius, an activation function, a distance measure and others and analyzed the accuracy of the outlier detection process. The accuracy of the S OM algorithm for outlier detection is satisfying. This algorithm is faster than the LOF algoritm. The results of the experiments show that there is a strong negative correlation between the learning parameters and the training time of an SOM map and the QE quantization error computing.