How Can We Ensure Reproducibility and Clinical Translation of Machine Learning Applications in Laboratory Medicine?

Abstract

Recent studies demonstrating problems with COVID (1) and sepsis (2) prediction models have helped raise awareness (3) about the need for better practices in developing and reporting machine learning (ML; or artificial intelligence) methods in healthcare. This so-called reproducibility (or replication) crisis has been recognized and extends beyond clinical applications (4). In fact, this issue is not specific to ML. Many scientific journals, including Clinical Chemistry, have adopted principles (specified in the article submission guidelines) to facilitate reproducibility, rigor, and transparency in published findings. Although there are common elements that support transparency and rigor in science, each technology has its own set of pitfalls that must be addressed. This is true for the rapidly developing field of ML. The growth and development of open-source software and digitized and publicly available data sources have made the application of ML methods highly accessible. While this has facilitated a surge in interest and publications, it has reduced the requirement for developers to have the necessary foundational or subject matter knowledge needed for quality publications and innovations. When ML methods are published in clinical journals, peer reviewers and editors may lack the expertise to appropriately evaluate technical aspects of submissions. Thus, we need best practices that help educate clinical experts and govern how ML for laboratory medicine should be developed and communicated. This is critical for ensuring practicality and reproducibility of ML applications and, ultimately, their successful translation to clinical practice. Regulatory guidance for ML-based diagnostics is evolving. For example, the US Food and Drug Administration recently outlined its plan for regulating artificial intelligence/ML-based software as a medical device (5). The Food and Drug Administration maintains a list of approved artificial intelligence/ML-enabled devices (6). The current reality in laboratory medicine (and many other clinical specialties outside of radiology) is that there are relatively few examples of ML-based products that have gone through the Food and Drug Administration’s proposed review pathway or have been implemented as lab developed tests or decision support tools following some type of validation. Most ML applications are limited to publication in scientific journals, and several groups have proposed checklists that describe the minimum information required when reporting ML methods and points to be considered in their review (4, 7–11). Here, we draw from this prior work to summarize practices (Table 1) that should be applied in the development, reporting, and review of ML applications. Summary of recommended practices for ML model development activities. State the objective of the ML problem, specifying inputs and outputs, with its prediction methodology. Provide adequate detail about the clinical scenario and population where the model applies. Describe details of data collection/aggregation, preparation, and case labeling steps. Use large and diverse data sets that are representative of the intended population and scenario. Provide descriptive statistics with demographics before and after processing steps for training and validation data sets. Use data preparation methods that prevent outcome and validation data leakage. Perform 5- or 10-fold cross validation versus simple train/test splits. Investigate a range of model types and tune hyperparameters. Provide rationale for final selection. Report performance metrics from validation set predictions. Evaluate model performance and clinical utility or impact. Report metrics appropriate for the clinical decision (e.g., area under the precision recall curve, positive predictive value, negative predictive value, number needed to alert). Include external validation for final method performance evaluation and note any recalibration strategies used or indicated. Interpret the results and performance of the selected model. Address performance in relevant subgroups and clinical scenarios. Conduct post-hoc model examination using global and local interpretability methods suitable for the selected model. Make data and code accessible. State the objective of the ML problem, specifying inputs and outputs, with its prediction methodology. Provide adequate detail about the clinical scenario and population where the model applies. Describe details of data collection/aggregation, preparation, and case labeling steps. Use large and diverse data sets that are representative of the intended population and scenario. Provide descriptive statistics with demographics before and after processing steps for training and validation data sets. Use data preparation methods that prevent outcome and validation data leakage. Perform 5- or 10-fold cross validation versus simple train/test splits. Investigate a range of model types and tune hyperparameters. Provide rationale for final selection. Report performance metrics from validation set predictions. Evaluate model performance and clinical utility or impact. Report metrics appropriate for the clinical decision (e.g., area under the precision recall curve, positive predictive value, negative predictive value, number needed to alert). Include external validation for final method performance evaluation and note any recalibration strategies used or indicated. Interpret the results and performance of the selected model. Address performance in relevant subgroups and clinical scenarios. Conduct post-hoc model examination using global and local interpretability methods suitable for the selected model. Make data and code accessible. Summary of recommended practices for ML model development activities. State the objective of the ML problem, specifying inputs and outputs, with its prediction methodology. Provide adequate detail about the clinical scenario and population where the model applies. Describe details of data collection/aggregation, preparation, and case labeling steps. Use large and diverse data sets that are representative of the intended population and scenario. Provide descriptive statistics with demographics before and after processing steps for training and validation data sets. Use data preparation methods that prevent outcome and validation data leakage. Perform 5- or 10-fold cross validation versus simple train/test splits. Investigate a range of model types and tune hyperparameters. Provide rationale for final selection. Report performance metrics from validation set predictions. Evaluate model performance and clinical utility or impact. Report metrics appropriate for the clinical decision (e.g., area under the precision recall curve, positive predictive value, negative predictive value, number needed to alert). Include external validation for final method performance evaluation and note any recalibration strategies used or indicated. Interpret the results and performance of the selected model. Address performance in relevant subgroups and clinical scenarios. Conduct post-hoc model examination using global and local interpretability methods suitable for the selected model. Make data and code accessible. State the objective of the ML problem, specifying inputs and outputs, with its prediction methodology. Provide adequate detail about the clinical scenario and population where the model applies. Describe details of data collection/aggregation, preparation, and case labeling steps. Use large and diverse data sets that are representative of the intended population and scenario. Provide descriptive statistics with demographics before and after processing steps for training and validation data sets. Use data preparation methods that prevent outcome and validation data leakage. Perform 5- or 10-fold cross validation versus simple train/test splits. Investigate a range of model types and tune hyperparameters. Provide rationale for final selection. Report performance metrics from validation set predictions. Evaluate model performance and clinical utility or impact. Report metrics appropriate for the clinical decision (e.g., area under the precision recall curve, positive predictive value, negative predictive value, number needed to alert). Include external validation for final method performance evaluation and note any recalibration strategies used or indicated. Interpret the results and performance of the selected model. Address performance in relevant subgroups and clinical scenarios. Conduct post-hoc model examination using global and local interpretability methods suitable for the selected model. Make data and code accessible. Although ML seems to be ubiquitous, not every problem is suitable for ML, and not all ML predictions are feasible or useful in practice. Development of ML-based tools should begin with a properly framed question that is representative of and practical to the workflow or process for which it is designed. This is often easier said than done, and there are numerous examples where poorly formulated ML problems led to bias or other unintended and harmful consequences from seemingly reasonable uses of ML. This includes a recent discovery of racial bias in an algorithm for assessing suitability for high-risk care management programs based on a patient’s predicted healthcare needs, although it specifically excluded race as a feature (12). Other issues arise when problem formulations allow algorithms to utilize aspects that may not be relevant and are unlikely to be generalizable, particularly when they reflect practice- or policy-based patterns. Authors and reviewers must be aware of the troubles that may arise during this stage of development. Reports should clearly state the objective of the ML problem with its prediction methodology and provide adequate detail about the clinical scenario and population where the model applies. Any proxy or surrogate outcome markers or labels should be explicitly defined and justified. Authors should describe input data to demonstrate their quality, reliability, and suitability for the problem. This includes descriptions of the procedures for data collection or aggregation, inclusion/exclusion, preprocessing, and case labeling. Preprocessing is often necessary with laboratory data, as issues such as outliers, skewness, value censoring, class imbalance, and missingness are common. Descriptive statistics should be reported for the data set before and after processing, comparing the training and validation sets. Supervised learning approaches require data that are accurately labeled. Any potential sources of bias in data selection, preparation, or labeling should be identified and mitigated. The lack of large and diverse data sets is a major problem in ML. Underrepresentation of racial, ethnic, sex, and geographic groups and discriminatory biases in training data are increasingly being identified as issues hampering ML applications. The predictor and response variables should be defined, with steps taken to guarantee their separation and to prevent data leakage. Data leakage is a phenomenon in ML that leads to overoptimistic performance. It occurs when variables in the training set can easily be used to infer outcomes, or when data from the training set overlaps with that in the test or validation sets. One common mistake that allows for data leakage is performing data preparation steps on the entire data set before splitting for training and validation. Because it is not possible to know a priori the most effective ML algorithm for a given problem, it is best practice to compare models and optimize the parameters controlling the way in which they learn (so-called hyperparameters). All model comparisons and hyperparameter tuning should be performed using cross-validation (typically 5- or 10-fold) from the training set. More important, under no circumstances should validation data ever be used as part of a model selection or hyperparameter tuning process. The final performance of a trained ML model should be assessed using a validation data set that has not been used for any portion of the optimization or training process. In many cases, this represents a held-out portion of the original data set. However, when available, it is preferable to have an external validation data set of adequate size that was independently collected to demonstrate that the final model is generalizable with consistent predictor effects. External validation may identify the need for model recalibration when applied to new data, as performance may vary across settings, populations, and time. ML performance metrics should be reported from the predictions within the validation set. Given the limitations of the area under the ROC curve when applied to many clinical scenarios, developers should also calculate and report metrics appropriate for medical decision making and describe performance on relevant subpopulations (13). When a holdout or independent validation cohort is not available due to an extremely limited data set, it may be acceptable to report preliminary results based on cross-validation. However, it is critical that all aspects of variable selection and model development are independently performed using only the appropriate subset of training data for each iteration of the cross-validation. While this approach may be necessary in the case of limited data, it is a much weaker result than the use of a true validation set, and limitations such as the possibility of overfitting should be clearly described. Further, investigators should be clear that this approach only provides proof-of-concept evidence that the given ML approach may work, rather than a finalized model. Although a black-box approach to ML may be attractive for certain complex tasks, it carries substantial risks for many applications in laboratory medicine. Specifically, it is important to examine particular variables that drive predictions, in part, to make sure unanticipated problems such as obvious mistakes of causality are not missed. For example, 1 published attempt to create an ML prediction of death due to pneumonia identified asthma as a protective factor; further investigation, however, revealed that this was actually due to aggressive treatment of known asthma patients, rather than a function of asthma itself (14). This cautionary incident underscores the need for assessment and discussion of variables and interactions that contribute to the outcome of models in general (global) and to individual predictions (local). A number of interpretability approaches (15), such as sensitivity analysis and SHapley Additive exPlanations (SHAP), are now available to assist with such evaluations and should be strongly encouraged. The complex nature of bioinformatics and ML workflows has led to an increased appreciation of the need for reproducible workflows. Methods sections of traditional manuscripts are not sufficient to allow critical examination of the way in which data have been handled and the model has been trained and validated. For this reason, we strongly advocate that authors provide access to the underlying code and data, as this can help verify best practices were used and is a critical step in replicating the results. This is the strongest way to ensure the ongoing integrity of the literature for ML. In our experience, authors are sometimes reluctant to provide this information due to concerns about intellectual property or inadvertent disclosure of protected patient information. It is likely that journal editors will need to mandate that authors provide the elements of a reproducible workflow to adequately address these issues. ML-based methods are appearing with increasing frequency in scientific research reports, including those in laboratory medicine. As with all emerging technologies, best practices are required to ensure appropriate rigor and transparency of these reports, which will lead to their reproducibility, replicability, and clinical translation. We have summarized our opinions on best practices, garnered from prior work in general and clinical ML fields, to outline how investigators should develop, validate, and report ML-based methods in laboratory medicine. We hope this report also provides guidance for all journal editors and reviewers when evaluating submitted papers. All authors confirmed they have contributed to the intellectual content of this paper and have met the following 4 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved. Upon manuscript submission, all authors completed the author disclosure form. Disclosures and/or potential conflicts of interest: S.R. Master, AACC; S. Haymond, AACC. S.R. Master, Indigo BioAutomation. None declared. S.R. Master, Korean Society for Laboratory Medicine; S. Haymond, AACC. S.R. Master, National Institutes of Health. S.R. Master, US District Attorney, Northern California. None declared. S. Haymond, support for attending meetings and/or travel from AACC; S.R. Master, support for attending meetings and/or travel from AACC.

Topics & Concepts