Performance evaluation of optimal ensemble learning approaches with PCA and LDA-based feature extraction for heart disease prediction
Md Masud Karim Rabbi, MA Bari, Tanoy Debnath, Anichur Rahman, Avik Kumar Das, Md. Parvez Hossain, Ghulam Muhammad
Abstract
Heart disease is a global health concern with a high mortality rate , necessitating early, accurate, and reliable prediction methods for effective prevention and control. In this research, we combine principal component analysis and linear discriminant analysis to reduce dataset complexity and enhance the performance of heart disease classification models by selecting the most relevant features. We address the class imbalance by employing two balancing techniques: oversampling and the synthetic minority oversampling technique, which ensures a more representative dataset, leading to more accurate predictions. Our study develops a novel ensemble approach, utilizing a combination of random forest , support vector machine , K-nearest neighbors, logistic regression , decision tree , and Gaussian naive Bayes to significantly improve heart disease prediction accuracy. Furthermore, we implement advanced ensemble learning techniques, such as Stacking, Bagging, Voting, and Boosting, to achieve early and precise prediction of heart disease. The performance evaluation is conducted on three datasets: Cleveland Heart Disease, Framingham Heart Disease, and Indicators of Heart Disease Dataset (2020), ensuring a robust validation of our methods. The results demonstrate that the voting ensemble machine learning algorithm (VEMLA) achieved 92% accuracy on the Cleveland Heart Disease dataset, while the bagging ensemble machine learning algorithm (BEMLA) achieved 97% accuracy on both the Framingham Heart Disease and Indicators of Heart Disease (2020) datasets. Notably, the proposed BEMLA consistently outperformed other methods, showcasing its superiority in heart disease prediction. This study contributes a comprehensive and effective approach to heart disease diagnosis, outperforming individual classifiers and providing valuable insights for practical clinical applications.