PerMetrics: A Framework of Performance Metrics forMachine Learning Models

Abstract

Performance metrics are pivotal in machine learning field, especially for tasks like regression, classification, and clustering (Saura, 2021).They offer quantitative measures to assess the accuracy and efficacy of models, aiding researchers and practitioners in evaluating, contrasting, and enhancing algorithms and models.In regression tasks, where continuous predictions are made, metrics such as mean squared error (MSE), root mean square error (RMSE), and Coefficient of Determination (COD) (Nguyen et al., 2018; Nguyen, Nguyen, & Nguyen, 2019) can reveal how well models capture data patterns.In classification tasks, metrics such as accuracy, precision, recall, F1-score, and AUC-ROC (Luque et al., 2019) assess a model's ability to classify instances correctly, detect false results, and gauge overall predictive performance.Clustering tasks aim to discover inherent patterns and structures within unlabeled data by grouping similar instances together.Metrics like Silhouette coefficient, Davies-Bouldin index, and Calinski-Harabasz index (Nainggolan et al., 2019) measure clustering quality, helping evaluate how well algorithms capture data distribution and assign instances to clusters.In general, performance metrics serve multiple purposes.They enable researchers to compare different models and algorithms (Ahmed et al., 2021), identify strengths and weaknesses (Nguyen, Nguyen, Nguyen, & Nguyen, 2019), and make informed decisions about model selection and parameter tuning (Nguyen, Hoang, et al., 2020).Moreover, it also plays a crucial role in the iterative process of model development and improvement.By quantifying the model's performance, metrics guide the optimization process (Van Thieu, Deb Barma, et al., 2023), allowing researchers to fine-tune algorithms, explore feature engineering techniques (Nguyen et al., 2021), and address issues such as overfitting, underfitting, and bias (Nguyen, Nguyen, et al., 2020).This paper introduces a Python framework named PerMetrics (PERformance METRICS), designed to offer comprehensive performance metrics for machine learning models.The library, packaged as permetrics, is open-source and written in Python.It provides a wide number of metrics to enable users to evaluate their models effectively.permetrics is hosted on GitHub and is under continuous development and maintenance by the dedicated team.The framework is accompanied by comprehensive documentation, examples, and test cases, facilitating easy comprehension and integration into users' workflows.

Topics & Concepts