Watching Your Phone's Back
Lei Wang, Xiang Zhang, Yuanshuang Jiang, Yong Zhang, Chenren Xu, Ruiyang Gao, Daqing Zhang
Abstract
Gesture recognition on the back surface of mobile phone, not limited to the touch screen, is an enabling Human-Computer Interaction (HCI) mechanism which enriches the user interaction experiences. However, there are two main limitations in the existing Back-of-Device (BoD) gesture recognition systems. They can only handle coarse-grained gesture recognition such as tap detection and cannot avoid the air-borne propagation suffering from the interference in the air. In this paper, we propose StruGesture, a fine-grained gesture recognition system using the back of mobile phones with ultrasonic signals. The key technique is to use the structure-borne sounds (i.e., sound propagation via structure of the device) to recognize sliding gestures on the back of mobile phones. StruGesture can fully extract the structure-borne component from the hybrid Channel Impulse Response (CIR) based on Peak Selection Algorithm. We develop a deep adversarial learning architecture to learn the gesture-specific representation for robust and effective recognition. Extensive experiments are designed to evaluate the robustness over nine deployment scenarios. The results show that StruGesture outperforms the competitive state-of-the-art classifiers by achieving an average recognition accuracy of 99.5% over 10 gestures.