Large Language Model-based Human-Agent Collaboration for Complex Task Solving
Xueyang Feng, Zhiyuan Chen, Yujia Qin, Yankai Lin, Chen Xu, Zhiyuan Liu, Ji-Rong Wen
Abstract
In recent developments within the research community, the integration of Large Language Models (LLMs) in creating fully autonomous agents has garnered significant interest.Despite this, LLM-based agents frequently demonstrate notable shortcomings in adjusting to dynamic environments and fully grasping human needs.In this work, we introduce the problem of LLM-based human-agent collaboration for complex task-solving, exploring their synergistic potential.To tackle the problem, we propose a Reinforcement Learning-based Human-Agent Collaboration method, ReHAC, which trains a policy model designed to determine the most opportune stages for human intervention within the task-solving process.We conduct experiments under real and simulated human-agent collaboration scenarios.Experimental results demonstrate that the synergistic efforts of humans and LLM-based agents significantly improve performance in complex tasks, primarily through well-planned, limited human intervention.Datasets and code are available at: