Quantitative Metrics for Benchmarking Human-Aware Robot Navigation
Jarosław Karwowski, Wojciech Szynkiewicz
Abstract
Social robots have recently gained popularity, and many human-aware navigation approaches have emerged. This work presents a comprehensive benchmark for quantitatively assessing robot navigation methods. As an automated quantitative approach, our Social Robot Planner Benchmark (SRPB) produces invariable indicators of the algorithm’s performance that can assist the system designer in selecting the best method for the specific application. Our benchmark extends state-of-the-art task performance scores and proposes novel social metrics regarding robot motion naturalness and the perceived safety of humans surrounding the robot. Our social metrics take human tracking reliability into account. Using the SRPB integrated with the TIAGo robot, we assessed the robot’s behaviour operating with traditional and human-aware trajectory planners in simulated and real-world environments. Our experiments tested whether state-of-the-art human-aware trajectory planners significantly improve human-awareness indicators over traditional approaches yet still maintain reasonable navigation performance. An open-source implementation of our benchmark, compatible with the Robot Operating System, is provided.