Detection of outliers with respect to a MUSIC geotechnical database
Jianye Ching, Kok‐Kwang Phoon, Pengsheng Huang
Abstract
This study proposes a novel method that addresses a nontraditional class of outlier detection problems. The purpose of most outlier detection methods in the literature is to detect outliers within a dataset. A record can be considered an outlier if it is distinct from the regular records in the dataset. However, the purpose of the novel outlier detection method proposed in this study is to detect outlier data groups (a data group may denote a site or a project) with respect to a soil/rock property "MUSIC" database. A data group is an outlier group if its characteristics (mean, variance, correlation, or higher order dependency) are distinct from the regular data groups in the database. This study frames the outlier detection problem into a formal hypothesis testing problems with the null hypothesis that “the target data group is identically distributed as the regular groups in the database.” With the hierarchical Bayesian model previously developed by the first two authors, the p-value for this hypothesis testing problem can be estimated rigorously. Numerical and real examples show that the p-value can effectively detect outlier data groups as well as outlier records with respect to a database.