Scorecard for synthetic medical data evaluation
Ghada Zamzmi, Adarsh Subbaswamy, Elena Sizikova, Edward Margerrison, Jana G. Delfino, Aldo Badano
Abstract
Although the interest in synthetic medical data (SMD) for developing and testing artificial intelligence (AI) methods is growing, the absence of a comprehensive framework to evaluate the quality and applicability of SMD hinders its wider adoption. Here, we outline an evaluation framework designed to meet the unique requirements of medical applications. We also introduce SMD scorecard, a comprehensive report accompanying artificially generated datasets. This scorecard provides a quantitative assessment of SMD across seven criteria (7 Cs), complemented by a descriptive section that contains all relevant information about the dataset. The SMD scorecard provides a practical framework for evaluating and reporting the quality of synthetic data, which can benefit SMD developers and users. The use of synthetic medical data (SMD) in AI development is on the rise, but its broader application is limited by the lack of a comprehensive evaluation framework. Here, Ghada Zamzmi and colleagues present a novel evaluation framework tailored for medical applications, along with an SMD scorecard that quantitatively assesses synthetic datasets across seven key criteria.