A Reinforcement Learning-Based Framework for Solving the IP Mapping Problem
Qingkun Chen, Wenjin Huang, Yuze Peng, Yihua Huang
Abstract
In network-on-chip (NoC) designs, the intellectual property (IP) mapping problem is a critical issue and is usually solved by heuristic searches. However, heuristic searches suffer from the problem of easily falling into the local optimum. To tackle this problem, this article proposes a reinforcement learning-based framework (RLF), which enhances the performance of heuristic searches through the neural network-based probability model. Within this framework, first, a neural network-based probability model for IP mapping is built and trained by reinforcement learning instead of supervised learning to overcome the difficulty of obtaining a high-quality labeled training set. Second, based on the pretrained probability model, the model-based heuristic uses the probability model to generate the initial population and then employs heuristic searches to find the optimal solution. Two model-based heuristics, i.e., the message passing neural network-pointer network-based genetic algorithm (MPN-GA) and the message passing neural network-pointer network-based PSMAP (MPN-PSMAP), are proposed as specific instances. Simulation results show that the MPN-GA reduces the communication cost by an average of 9.32% than the genetic algorithm (GA). The MPN-PSMAP achieves an average reduction in the communication cost of 8.37% than the PSMAP. Finally, two extensions are given as examples to show the good extensibility of this framework.