ThirdEye: Attention Maps for Safe Autonomous Driving Systems
Andrea Stocco, Paulo J. Nunes, Marcelo d’Amorim, Paolo Tonella
Abstract
Automated online recognition of unexpected conditions is an indispensable component of autonomous vehicles to ensure safety even in unknown and uncertain situations. In this paper we propose a runtime monitoring technique rooted in the attention maps computed by explainable artificial intelligence techniques. Our approach, implemented in a tool called ThirdEye, turns attention maps into confidence scores that are used to discriminate safe from unsafe driving behaviours. The intuition is that uncommon attention maps are associated with unexpected runtime conditions.
Topics & Concepts
IntuitionComputer scienceArtificial intelligenceComponent (thermodynamics)Machine learningHuman–computer interactionPsychologyCognitive scienceThermodynamicsPhysicsAdversarial Robustness in Machine LearningAnomaly Detection Techniques and ApplicationsExplainable Artificial Intelligence (XAI)