Configurable 2D–3D CNNs Accelerator for FPGA-Based Hyperspectral Imagery Classification
Wenjing He, Yuesong Yang, Shaohui Mei, Jian Hu, Wanqiu Xu, Shiqi Hao
Abstract
Convolutional neural network (CNN) is used efficiently for classification of hyperspectral imagery (HSI). Both 3D CNN and hybrid 2D-3D CNN have better performance due to the full extraction of joint spatial-spectral feature. Most of previous acceleration researches on field-programmable gate array (FPGA) has concentrated on 2D CNN, few of which was made for 3D CNN acceleration. More importantly, it's notable that the CNN models for HSI classification have some unique characteristics of computation and memory, different from that for object detection. Thus, the conventional accelerating approaches may be not applicable. To address this problem, we propose a dynamic configurable accelerator architecture suitable for both 2D and 3D CNN, ensuring fast development for various networks. The multiple nested loops are optimized delicately and the convolutional layers are fully parameterized, making it easy to scale up the network. Furthermore, we develop the parallelism-oriented memory pattern and data access strategy to optimize the data path and the local buffer. Finally, we implement the proposed architecture for 2D,3D and hybrid CNNs, validating its effectiveness and reconfigurability. To demonstrate its extensibility, we also prototype the accelerator on two FPGA platforms. We achieve the average and maximum throughput of 18.2/166.18 giga operations per second (GOPS) for HybridSN. To compare with previous FPGA accelerators, a typical baseline 3D model (C3D) is also implemented with the average and maximum throughput 160.25/225.87 GOPS, and the performance efficiency achieves up to 1.23 with 10.7% power efficiency gain.