CrowdBERT: Crowdsourcing Indoor Positioning via Semi-Supervised BERT With Masking
Han Yu, Zan Li, Zhongliang Zhao, Torsten Braun
Abstract
As a mature indoor positioning solution, fingerprint-based positioning has been widely applied. However, traditional fingerprint positioning schemes still face the problems of limited hidden spatial feature extraction ability and insufficient fingerprint calibration with unlabeled crowdsourcing data. In order to address the above problems, we refer to the transformer-based deep learning model in natural language processing (NLP) and propose a crowdsourcing indoor positioning model via semi-supervised bidirectional encoder representation for transformer with masking, namely CrowdBERT. First, we tokenize the fingerprint data to adapt to the input form of the model. Then, we design a spatial fingerprint attention encoder as a feature extractor, which internal multihead attention mechanism combined with three-layer spatial feature embedding can fully capture the spatial features of fingerprint sequences. Meanwhile, we propose received signal strength-token masking to help the model perform bidirectional feature extraction so that the pretraining can more efficient use of hidden features from unlabeled crowdsourcing fingerprints. Finally, the limited labeled fingerprints is used to fine tune the downstream network structure to further improve the positioning accuracy. To evaluate our proposed positioning system, we conduct a set of comprehensive experiments on the three different data sets and evaluation results demonstrate that the CrowdBERT model significantly outperforms the other traditional positioning algorithms, such as K nearest neighbor, DNN, residual network, stacked autoencoder, and variational autoencoder.