Sensitivity Adaptation of Lower-Limb Exoskeleton for Human Performance Augmentation Based on Deep Reinforcement Learning
Ranran Zheng, Zhiyuan Yu, Hongwei Liu, Zhe Zhao, Jing Chen, Longfei Jia
Abstract
The lower-limb exoskeleton for human performance augmentation (LEHPA) in sensitivity amplification control (SAC) is vulnerable to model parameter uncertainties and unmodeled dynamics due to its large sensitivity to external disturbances. This paper proposes sensitivity adaptation based on deep reinforcement learning (SADRL) to reduce the dependence on the model accuracy and tackle the ever-changing human-exoskeleton interaction (HEI) dynamics by interpreting the sensitivity adjustment as a Markov Decision Process (MDP). The agent learns an appropriate sensitivity for each exoskeleton state. To train the control policy safely and efficiently, a multibody simulation environment is created to implement the training process, accompanied by a novel hybrid inverse-forward dynamics simulation method to carry out the simulation. For comparison purposes, the SAC controller is introduced as a benchmark. A novel performance evaluation method based on the HEI forces at the back, thighs, and shanks is proposed to evaluate the control effect of the trained SADRL controller quantitatively. The SADRL controller is compared with the SAC controller at five specified walking speeds, resulting in a lumped HEI force ratio of 0.54. The total decrease of HEI forces demonstrates the superior control effect of the SADRL strategy.