Optimal ratio for data splitting
V. Roshan Joseph
2022Statistical Analysis and Data Mining The ASA Data Science Journal757 citationsDOIOpen Access PDF
Abstract
Abstract It is common to split a dataset into training and testing sets before fitting a statistical or machine learning model. However, there is no clear guidance on how much data should be used for training and testing. In this article, we show that the optimal training/testing splitting ratio is , where is the number of parameters in a linear regression model that explains the data well.
Topics & Concepts
Computer scienceTraining setLinear regressionMachine learningData miningStatistical hypothesis testingArtificial intelligenceStatisticsAlgorithmPattern recognition (psychology)MathematicsGaussian Processes and Bayesian InferenceNeural Networks and ApplicationsAdvanced Statistical Methods and Models