Weight-adaptive joint mixed-precision quantization and pruning for neural network-based equalization in short-reach direct detection links
Zhaopeng Xu, Qi Wu, Weiqi Lu, Honglin Ji, Hui Chen, Tonghui Ji, Yu Yang, Gang Qiao, Jianwei Tang, Chen Cheng, Lulu Liu, Shangcheng Wang, Junpeng Liang, Jinlong Wei, Weisheng Hu, William Shieh
Abstract
Neural network (NN)-based equalizers have been widely applied for dealing with nonlinear impairments in intensity-modulated direct detection (IM/DD) systems due to their excellent performance. However, the computational complexity (CC) is a major concern that limits the real-time application of NN-based receivers. In this Letter, we propose, to our knowledge, a novel weight-adaptive joint mixed-precision quantization and pruning approach to reduce the CC of NN-based equalizers, where only integer arithmetic is taken into account instead of floating-point operations. The NN connections are either directly cutoff or represented by a proper number of quantization bits by weight partitioning, leading to a hybrid compressed sparse network that computes much faster and consumes less hardware resources. The proposed approach is verified in a 50-Gb/s 25-km pulse amplitude modulation (PAM)-4 IM/DD link using a directly modulated laser (DML) in the C-band. Compared with the traditional fully connected NN-based equalizer operated with standard floating-point arithmetic, about 80% memory can be saved at a minimum network size without degrading the system performance. Quantization is also shown to be more suitable to over-parameterized NN-based equalizers compared with NNs selected at a minimum size.