Egocentric Human Pose Estimation using Head-mounted mmWave Radar
Wenwei Li, Ruofeng Liu, Shuai Wang, Dongjiang Cao, Wenchao Jiang
Abstract
3D human pose plays a critical role in human behavior understanding and has many applications (e.g., VR/AR). Conventional pose estimations deploy sensors as fixed infrastructure, which significantly restrains the mobility of the user. Inspired by the emerging head-mounted devices (e.g., VR/AR glasses) and the recent advance in low-cost mmWave radar, we present mmEgo, the first egocentric human pose estimation design using a head-mounted mmWave radar, which offers ubiquitous pose tracking with high mobility, robustness to complex environments, and privacy preservation. To tackle the unique challenges of radar sensing from the egocentric perspective (e.g., random radar motion and the scarcity of information on the lower body), we propose several technical designs, including root-relative radar motion tracking for radar motion decoupling and a two-stage pose estimator that incorporates human kinematics priors. Extensive experiments and case studies show that our method can reduce the joint localization error by 44.2% and potentially enable a wide spectrum of applications.