DT-CNN: An Energy-Efficient Dilated and Transposed Convolutional Neural Network Processor for Region of Interest Based Image Segmentation
Dongseok Im, Donghyeon Han, Sungpill Choi, Sanghoon Kang, Hoi‐Jun Yoo
Abstract
An energy-efficient convolutional neural network (CNN) processor is proposed for real-time image segmentation on mobile devices. The proposed processor utilizes Region of Interest (ROI) based image segmentation to speed up the process and reduce the overall external memory access. Although the ROI based image segmentation degrades the segmentation accuracy, the proposed dilation rate adjustment algorithm, which regulates the receptive field depending on the ROI resolution during dilated convolution, compensates for the accuracy degradation up to 0.2310 mean Intersection over Union (mIoU). In addition, the processor accelerates the dilated and transposed convolution by skipping the redundant zero computations with the proposed delay cells. As a result, the throughput of dilated and transposed convolution is increased up to ×159 and ×3.84 . The delay cells can also support the variable dilation rates in dilated convolution caused by the dilation rate adjustment algorithm. Moreover, the processor selects the operating frequency based on the ROI resolution to save power consumption up to 81.2%. The processor is simulated in 65 nm CMOS technology, and the 6.8 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> processor consumes the 206 mW power consumption with the 4.66 ms of processing time and 3.22 TOPS/W energy-efficiency at the target image segmentation dataset.