AI in the hands of imperfect users
Kristin M. Kostick, Sara Gerke
Abstract
As the use of artificial intelligence and machine learning (AI/ML) continues to expand in healthcare, much attention has been given to mitigating bias in algorithms to ensure they are employed fairly and transparently. Less attention has fallen to addressing potential bias among AI/ML's human users or factors that influence user reliance. We argue for a systematic approach to identifying the existence and impacts of user biases while using AI/ML tools and call for the development of embedded interface design features, drawing on insights from decision science and behavioral economics, to nudge users towards more critical and reflective decision making using AI/ML.
Topics & Concepts
ImperfectComputer scienceArtificial intelligenceHuman–computer interactionData scienceMachine learningPhilosophyLinguisticsArtificial Intelligence in Healthcare and EducationEthics and Social Impacts of AIHealthcare cost, quality, practices