Benchmarking Off-The-Shelf Solutions to Robotic Assembly Tasks
Wenzhao Lian, Tim Kelch, Dirk Holz, Adam Norton, Stefan Schaal
Abstract
In recent years, many learning based approaches have been studied to realize robotic manipulation and assembly tasks, often including vision and force/tactile feedback. How-ever, it is unclear what the baseline state-of-the-art performance is and what the bottleneck problems are. In this work, we evaluate off-the-shelf (OTS) industrial solutions on a recently introduced benchmark, the National Institute of Standards and Technology (NIST) Assembly Task Board. A set of assembly tasks is introduced and baseline methods are provided to understand their intrinsic difficulty. Multiple sensor-based robotic solutions are then evaluated, including hybrid force/motion control and 2D/3D pattern matching. An end-to-end integrated solution that accomplishes the tasks is also provided.The results and findings throughout the study reveal a few noticeable factors that impede the adoptions of the OTS solutions: dependency on expertise, limited applicability, lack of interoperability, no scene awareness or error recovery mechanisms, and high cost. This paper also provides a first attempt of an objective benchmark performance on the NIST Assembly Task Boards as a reference comparison for future works on this problem.