Memory Access Characteristics of Neural Network Workloads and Their Implications
Soyeon Park, Hyokyung Bahn
Abstract
With the recent advances in machine learning and many-core computing technologies, neural networks are widely used in various service domains of the 4th industrial revolution. As the data set of neural network is becoming increasingly large, it is important to analyze the memory access characteristics of neural network workloads. In this paper, we perform a comprehensive analysis of memory access behaviors in four types of neural network configurations, i.e., CNN (convolutional neural networks), RNN (recurrent neural networks), DNN (deep neural networks), and ANN (artificial neural networks). From this analysis, we observe the following characteristics, which are quite different from traditional desktop and smartphone memory accesses. First, we analyze the access bias of memory locations and find that most memory accesses occur in a certain limited memory locations. Second, the identity of these hot locations is the data and heap regions, which account for over 90% of total memory accesses. Third, the bias of memory access in neural network workloads is relatively weaker than other desktop or smartphone workloads, specially for write operations. Fourth, write operations account for about twice of read operations regardless of neural network types. Fifth, in predicting re-access likelihood, temporal locality provides better information than access frequency in read operations, but combining the two properties is necessary for accurate estimation in write operations. We anticipate that the analysis of this study will be a good guideline for designing an efficient memory system for neural network workloads.