Noncontact 3D gesture recognition enabled VR human–machine interface via electret-nanofiber-based triboelectric sensor
Lijun Lu, Jing Wu, Yujie Zhang, Chang Liu, Yun Yan Hu, Baojin Chen, Yan Zhu, Yanchao Mao
Abstract
Virtual reality (VR) is an emerging communication means and creates extensive opportunities in interacting scenarios such as remote collaboration and metaverse. Human-machine interfaces (HMIs) play important roles in VR as they provide interaction platforms between users and virtual environments. However, traditional VR HMIs based on handheld devices or keyboards cannot recognize diverse three-dimensional (3D) gestures, which results in limited freedom of VR interactions. Here, we report a noncontact VR HMI enabled by an electret-nanofiber-based triboelectric sensor (ETS), which is fabricated by the electrospun polylactic acid/thermoplastic polyurethane (PLA/TPU) electret nanofiber film. The nanofiber structure of PLA/TPU electret enhanced the charge retention ability of triboelectric sensor and thus significantly improved its signal strength and stability. Integrated with a deep learning-based multilayer perceptron neural network, the ETS realizes the recognition of 18 different types of 3D gestures with a high average accuracy of 97.3%. An intelligent noncontact VR interactive system based on the ETS is further developed, which is used to manipulate game characters for performing different actions by 3D gestures. Compared with traditional VR HMIs, the proposed VR HMI based on PLA/TPU electret nanofiber film can detect various 3D gestures and offers a superior interaction freedom. This work for the first time introduces the triboelectric 3D gesture recognition method to the VR HMIs, and could make the interaction between human and virtual environments become more efficient and fascinating.