Integrating Reinforcement Learning and Learning From Demonstrations to Learn Nonprehensile Manipulation
Xilong Sun, Jiqing Li, Anna Kovalenko, Wei Feng, Yongsheng Ou
Abstract
Motor skills are essential for robots to accomplish complicated and dexterous manipulation tasks, which are difficult to be mastered through traditional controller designs. Currently, robots learning from demonstrations enable them to learn control policies automatically from human motor demonstrations. However, the nonlinearity and instantaneousness of the demonstrated forces prohibit robots from fully mastering the motor skill features by simply exploiting force examples. Therefore, a self-improvement learning scheme is required to refine the control policy further until satisfactory motor skills are acquired. Hence, this paper combines learning from demonstrations and reinforcement learning to learn a controller for complex motor skills. The proposed method is validated on an IIWA KUKA robot, performing a specified nonprehensile manipulation task. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The motivation of this paper originates from the requirement to develop an efficient and fast learning algorithm that improves the robot skill learning efficiency. Specifically, our research focuses on the nonprehensile manipulation task, easily subject to environmental changes. Therefore, the robot must continuously interact with the environment to master the skill. To accelerate the skill learning process, learning from demonstrations initializes the control policies, and then the robot starts to practice the demonstrated skill. After each practice round, the robot receives a reward from the environment, and based on the reinforcement learning algorithm limited up to 100 trials, the robot masters the nonprehensile manipulation skill.