Classifying rockburst with confidence: A novel conformal prediction approach
Bemah Ibrahim, Isaac Ahenkorah
Abstract
The scientific community recognizes the seriousness of rockbursts and the need for effective mitigation measures. The literature reports various successful applications of machine learning (ML) models for rockburst assessment; however, a significant question remains unanswered: How reliable are these models, and at what confidence level are classifications made? Typically, ML models output single rockburst grade even in the face of intricate and out-of-distribution samples, without any associated confidence value. Given the susceptibility of ML models to errors, it becomes imperative to quantify their uncertainty to prevent consequential failures. To address this issue, we propose a conformal prediction (CP) framework built on traditional ML models (extreme gradient boosting and random forest) to generate valid classifications of rockburst while producing a measure of confidence for its output. The proposed framework guarantees marginal coverage and, in most cases, conditional coverage on the test dataset. The CP was evaluated on a rockburst case in the Sanshandao Gold Mine in China, where it achieved high coverage and efficiency at applicable confidence levels. Significantly, the CP identified several “confident” classifications from the traditional ML model as unreliable, necessitating expert verification for informed decision-making. The proposed framework improves the reliability and accuracy of rockburst assessments, with the potential to bolster user confidence.