Toward Real-World Applicability: Lightweight Underwater Acoustic Localization Model Through Knowledge Distillation
Runze Hu, Xiaohui Chu, Daowei Dou, Xiaogang Liu, Yining Liu, Bingbing Qi
Abstract
Deep learning (DL) approaches in underwater acoustic localization (UAL) have gained a great deal of popularity. While numerous works are devoted to improving the localization precision, they neglect another critical challenge inherent in the DL-based UAL problem, i.e., the model's practicality. Advanced DL models generally exhibit extremely high complexity, requiring a large amount of computational resources and resulting in slow inference time. Unfortunately, the limited processing power and real-time demands in oceanic applications make the deployment of complex DL models exceedingly challenging. To address this challenge, this article proposes a lightweight UAL framework based on knowledge distillation (KD) techniques, which effectively reduces the size of a deep UAL model while maintaining competitive performance. Specifically, a dedicated teacher network is designed using attention mechanisms and convolutional neural networks (CNNs). Then, the KD is performed to distill the knowledge from the teacher network into a lightweight student model, such as a three-layer CNN. In practical deployment, only the lightweight student model will be utilized. With the proposed lightweight framework, the student model has 98.68% fewer model parameters and is 87.4% faster in inference time compared to the teacher network, while the prediction accuracy drops to only 1.07% (97.55% <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\rightarrow$</tex-math></inline-formula> 96.48%). In addition, the generalization ability of the student model is examined through transfer learning, where the model is transferred between two different ocean environments. The student model demonstrates a stronger generalization ability compared to the model without the KD process, as it can quickly adapt itself to a new application environment using just 10% of the data.