Performance analysis of hyperparameter optimization methods for ensemble learning with small and medium sized medical datasets
Vinod J. Kadam, Shivajirao M. Jadhav
Abstract
The accuracy of the various classifiers depends mainly on good hyper-parameter and consequently on the scheme (hyper-parameter tuning algorithm) adopted to estimate these values. Currently, the hyper-parameter tuning for Ensemble classifiers (which involves a number of hyper-parameters) is receiving a lot of attention. This paper aims to analyze the effectiveness of three optimization methods: Grid Search (GS), Random Search (RS) and Bayesian Optimization (BO) on small and medium sized medical datasets to select a set of optimal hyper parameters for an ensemble classifier (here we used decision tree ensemble using AdaBoost). 5 fold CV wasemployed to evaluate the generalization performance.
Topics & Concepts
HyperparameterHyperparameter optimizationComputer scienceBayesian optimizationEnsemble learningAdaBoostMachine learningArtificial intelligenceDecision treeClassifier (UML)Naive Bayes classifierRandom forestGeneralizationGridPattern recognition (psychology)MathematicsSupport vector machineMathematical analysisGeometryMachine Learning and Data ClassificationArtificial Intelligence in HealthcareData Mining Algorithms and Applications