Optimized Breast Cancer Classification Using PCA-LASSO Feature Selection and Ensemble Learning Strategies With Optuna Optimization
Prabhat Kumar Sahu, Taiyaba Fatma
Abstract
Breast cancer is one of the most prevalent and life-threatening diseases among women worldwide, making early and accurate detection crucial for effective treatment. This study presents a novel and optimized breast cancer classification system using machine learning models enhanced through advanced hyperparameter tuning techniques and statistical validation methods. The publicly available Breast Cancer Wisconsin (Diagnostic) dataset was utilized and underwent preprocessing steps, including dimensionality reduction and feature selection methods such as Principal Component Analysis (PCA) combined with Least Absolute Shrinkage and Selection Operator (LASSO). The classifiers employed include Random Forest, Support Vector Machine (SVM), Gradient Boosting, and Logistic Regression, which were further refined using GridSearchCV, RandomizedSearchCV, and Optuna, with 3-fold cross-validation implemented to ensure robust evaluation of model performance. The novelty of this work lies in integrating advanced feature selection methods with Optuna-driven optimization and ensemble learning strategies to achieve superior accuracy. A comprehensive evaluation of the proposed models was performed using key classification metrics such as precision, recall, F1-score, and accuracy, with the Optuna-optimized ensemble model achieving the highest accuracy of 99.423%. Statistical validation of the results was conducted using ANOVA tests, confirming the significance of the improvements achieved through hyperparameter optimization. Additionally, feature importance scores for Random Forest and Gradient Boosting provide insights into the most influential factors in the classification process. This work demonstrates the efficacy of combining dimensionality reduction, ensemble learning, and Optuna optimization, validated through 3-fold cross-validation, offering a novel and reliable approach to interpretable breast cancer detection systems.