Dimensionality Reduction based on SHAP Analysis: A Simple and Trustworthy Approach
Chejarla Santosh Kumar, Movva Naga Sumanth Choudary, Vinay Babu Bommineni, Grandhi Tarun, T Anjali
Abstract
In this 21st century the world is driven by data, analysis, and predictions based on this data is substantial. However, these predictions that have an immense impact on our daily life comes with an overhead of complex data mining and large datasets. With this paper, we will suggest a way to reduce the dimensionality of the dataset without a great loss of accuracy and reduce the necessity for complex data mining, by analyzing the features based on their SHAP - SHapley Additive explanation, values we prioritize the features and discard the features of unsubstantial relevance to the accuracy of the model.
Topics & Concepts
Dimensionality reductionOverhead (engineering)Computer scienceRelevance (law)TrustworthinessCurse of dimensionalityData miningSimple (philosophy)Data modelingArtificial intelligenceMachine learningDatabaseComputer securityEpistemologyPolitical sciencePhilosophyLawOperating systemExplainable Artificial Intelligence (XAI)Machine Learning and Data ClassificationMachine Learning in Healthcare