Tactile Gloves Predict Load Weight During Lifting With Deep Neural Networks
Guoyang Zhou, Ming‐Lun Lu, Denny Yu
Abstract
Overexertion in lifting tasks is one of the leading causes of occupational injuries. The load weight is the key information required to evaluate the risk of a lifting task. However, weight varies across different objects and is unknown in many circumstances. Existing methods of estimating the load weight without manual weighing focused on analyzing body kinematics or muscle activations, which either utilize indirect indicators or require intrusive sensors. This study proposed using tactile gloves as a new modality to predict the load weight. Hand pressure data measured by tactile gloves during each lift were formulated as a 2-D matrix containing spatial and temporal information. Different types of deep neural networks were adopted, and a ResNet 18 regression model achieved the best performance. Specifically, it achieved a predicted <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${R}$ </tex-math></inline-formula> -squared of 0.821 and a mean absolute error of 1.579 kg. In addition, to understand the model’s decision-making logic and the hand force exertion pattern during lifting, the Shapley additive explanations (SHAPs) technique was utilized to determine the importance of each sensor at each frame. The results demonstrated that the right hand was more important than the left hand for the model to predict the load weight. Additionally, fingers were more important than palms, and the middle phase of a lifting task was more important than its beginning and ending phases. Overall, this study demonstrated the feasibility of using tactile gloves to predict the load weight and provided new scientific insights on hand force exertion patterns during lifting.