Experimental evidence of effective human–AI collaboration in medical decision-making
Carlo Reverberi, Tommaso Rigon, Aldo Solari, Cesare Hassan, Paolo Cherubini, GI Genius CADx Study Group, Giulio Antonelli, Halim Awadie, Sebastian Bernhofer, Sabela Carballal, Mário Dinis‐Ribeiro, A Fernández-Clotet, Glòria Fernández‐Esparrach, Ian M. Gralnek, Yuta Higasa, Taku Hirabayashi, Tatsuki Hirai, Mineo Iwatate, Miki Kawano, Markus Mader, A Maieron, Sebastian Mattes, Tastuya Nakai, Íngrid Ordás, Raquel Ortigão, Oswaldo Ortiz Zúñiga, María Pellisé, Cláudia Lúcia de Oliveira Pinto, Florian Riedl, Ariadna Sánchez, Emanuel Steiner, Yukari Tanaka, Andrea Cherubini
Abstract
Artificial Intelligence (AI) systems are precious support for decision-making, with many applications also in the medical domain. The interaction between MDs and AI enjoys a renewed interest following the increased possibilities of deep learning devices. However, we still have limited evidence-based knowledge of the context, design, and psychological mechanisms that craft an optimal human-AI collaboration. In this multicentric study, 21 endoscopists reviewed 504 videos of lesions prospectively acquired from real colonoscopies. They were asked to provide an optical diagnosis with and without the assistance of an AI support system. Endoscopists were influenced by AI ([Formula: see text]), but not erratically: they followed the AI advice more when it was correct ([Formula: see text]) than incorrect ([Formula: see text]). Endoscopists achieved this outcome through a weighted integration of their and the AI opinions, considering the case-by-case estimations of the two reliabilities. This Bayesian-like rational behavior allowed the human-AI hybrid team to outperform both agents taken alone. We discuss the features of the human-AI interaction that determined this favorable outcome.