Reformulating the direct convolution for high-performance deep learning inference on ARM processors
Sergio Barrachina, Adrián Castelló, Manuel F. Dolz, Tze Meng Low, Hèctor Martínez, Enrique S. Quintana–Ort́ı, Upasana Sridhar, Andrés E. Tomás
Abstract
We present two high-performance implementations of the convolution operator via the direct algorithm that outperform the so-called lowering approach based on the im2col transform plus the gemm kernel on an ARMv8-based processor. One of our methods presents the additional advantage of zero-memory overhead while the other employs an additional yet rather moderate workspace, substantially smaller than that required by the im2col+gemm solution. In contrast with a previous implementation of a similar zero-memory overhead direct convolution, this work exhibits the key advantage of preserving the conventional NHWC data layout for the input/output activations of the convolution layers.