Navigating Data-Centric Artificial Intelligence With DC-Check: Advances, Challenges, and Opportunities
Nabeel Seedat, Fergus Imrie, Mihaela van der Schaar
Abstract
Data-centric AI is an emerging paradigm that emphasizes the critical role of data in real-world machine learning (ML) systems—as a complement to model development. However, data-centric AI is still in its infancy, lacking a standardized framework that outlines necessary data-centric considerations at various stages of the ML pipeline: <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Data, Training, Testing, and Deployment</b> . This lack of guidance hampers effective communication and design of data-centric driven ML systems. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">To address this critical gap, we introduce DC-Check, an actionable checklist-style framework that encapsulates data-centric considerations for ML systems. DC-Check is aimed at both practitioners and researchers to serve as a reference guide to data-centric AI development. Around each question in DC-Check, we discuss the applicability of different approaches, survey the state of the art, and highlight specific data-centric AI challenges and research opportunities. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">While developing DC-Check, we also undertook an analysis of the current data-centric AI landscape. The insights obtained from this exploration support the DC-Check framework, reinforcing its utility and relevance in the rapidly evolving field. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">To make DC-Check and related resources easily accessible, we provide a DC-Check companion website (<uri>https://www.vanderschaar-lab.com/dc-check/</uri>), which will serve as a living resource, updated as methods and tools evolve.