ThuMouse: A Micro-gesture Cursor Input through mmWave Radar-based Interaction
Ziheng Li, Zhenyuan Lei, An Yan, Erin Solovey, Kaveh Pahlavan
Abstract
In this paper, we propose ThuMouse, a novel interaction paradigm aimed to create a gesture-based and touch-free cursor interaction that accurately tracks the motion of fingers in real-time. ThuMouse enables users to move the cursor using frequency-modulated continuous-wave (FMCW) radar. While previous work with FMCW radar in human-computer-interfaces (HCI) has focused on classifying a set of predefined hand gestures, ThuMouse regressively tracks the position of a finger, which allows for finer-grained interaction. This paper presents the gesture sensing pipeline we built, with regressive tracking through deep neural networks, data augmentation for robustness, and computer vision as a training base. We also report on a proof-of-concept demonstration shows how our system can function as a mouse, and identify areas for future work. This work builds a foundation for designing finer micro gesture-based interactions, allowing the finger to emulate external input devices such as a joystick and touch-pad.