Policy-Based Reinforcement Learning for Through Silicon Via Array Design in High-Bandwidth Memory Considering Signal Integrity
Keunwoo Kim, Hyunwook Park, Seongguk Kim, Youngwoo Kim, Kyungjune Son, Daehwan Lho, Keeyoung Son, Taein Shin, Boogyo Sim, Joonsang Park, Shinyoung Park, Joungho Kim
Abstract
In this article, a policy-based reinforcement learning (RL) method for optimizing through silicon via (TSV) array design in high-bandwidth memory (HBM) considering signal integrity is proposed. The proposed method can provide an optimal TSV-array signal/ground pattern design to maximize the eye opening (EO), which determines the bandwidth of the high-speed TSV channel. The proposed method adopts the proximal policy optimization algorithm, which directly trains the optimal policy, providing efficient handling of large action spaces rather than value-based RL. The convolutional neural network is used as a feature extractor to extract the location information of the TSV-array. To overcome the computational cost of the reward estimation, a fast EO estimation method is developed based on the equivalent circuit modeling and peak distortion analysis. The proposed method is applied to optimize 1-byte of TSV-array in a 16-high HBM and showed an 18.2% increase in EO compared with the initial design. The optimality performance of the proposed method is compared with deep q-network and random search algorithm, and the proposed method shows 3.4% and 9.6% better optimality, respectively.