A knowledge-based task planning approach for robot multi-task manipulation
Deshuai Zheng, Yan Jin, Tao Xue, Yong Liu
Abstract
Abstract Task planning is a crucial component in facilitating robot multi-task manipulations. Language-based task planning methods offer practicality in receiving commands from humans in real-life scenarios and require only low-cost labeled data. However, existing methods often rely on sequence models for planning, which primarily focus on mapping language to sequences of sub-tasks while neglecting the knowledge about tasks and objects. To overcome these limitations, we propose a knowledge-based task planning approach called Recurrent Graph Convolutional Network (RGCN). It is devised with a novel structure that combined GCN (Kipf and Welling in International Conference on Learning Representations (ICLR), 2017) and LSTM (Hochreiter and chmidhuber in Neural Comput 9 (8): 1735-1780, 1997. https://doi.org/10.1162/neco.1997.9.8.1735 ) which enables it to leverage knowledge graph data and historical predictions. The experimental results demonstrate that our approach achieves the impressive task planning success rate of $${95.7\%}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>95.7</mml:mn> <mml:mo>%</mml:mo> </mml:mrow> </mml:math> , surpassing the best baseline method significantly, which achieves $${78.7\%}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>78.7</mml:mn> <mml:mo>%</mml:mo> </mml:mrow> </mml:math> . Furthermore, we evaluate the performance of multi-task manipulation across a specific set of 20 tasks within a simulated environment. Notably, RGCN combined with pre-trained primitive tasks exhibits the highest success rate compared with state-of-art multi-task learning methods. Our method is proven to be significant for language-conditioned task planning and is qualified for instructing robots for multi-task manipulation.