Explainable Reinforcement Learning(XRL)-Based Decap Placement Optimization for High Bandwidth Memory (HBM)

• Main Authors: Kim, Keunwoo, Park, Hyunwook, Son, Keeyoung, Choi, Seonguk, Shin, Taein, Lee, Junghyun, Yoon, Jiwon, An, Hyunjun, Kim, Haeyeon, Choi, Wooshin, Choi, Jung-Hwan, Kim, Joungho
• Format: Conference Proceeding
• Language: English
• Subjects: Bandwidth; Electronics packaging; explainable reinforcement learning (XRL); high-bandwidth memory (HBM); Optimization methods; Power integrity (PI); Reinforcement learning; Training; Vectors
• ISSN: 2165-4115
• DOI: 10.1109/EPEPS61853.2024.10754045

In this paper, for the first time, we propose an explainable reinforcement learning (XRL)-based decap placement optimization method for high bandwidth memory (HBM) considering power integrity (PI). The proposed XRL-based method enhances explainability by transforming the sum of various types of rewards into a vector sum operation for the trained model. A CNN-based network was used for training, with each reward considered from a multi-objective RL perspective. To verify the proposed method, we applied it to solve the problem of placing decaps at VDDQ domain of HBM3 module. In this paper, rewards were set as the suppression of self-impedance and transfer-impedance at each probing port. The proposed method achieved improvements of 2.8% compared to usage of general scalar sum reward. Ultimately, the vector differences in the Q-value for different actions provided grounds for action taken and allowed for the evaluation of whether the model was well-trained.