Distributed Deep Learning Optimization of Heat Equation Inverse Problem Solvers

• Published in: IEEE transactions on computer-aided design of integrated circuits and systems 2023-12, Vol.42 (12), p.1-1
• Main Authors: Wang, Zhuowei, Yang, Le, Lin, Haoran, Zhao, Genping, Liu, Zixuan, Song, Xiaoyu
• Format: Article
• Language: English
• Subjects: Computational modeling; Computer architecture; concurrent computation; Cyber-physical systems; Data communication; Data models; Deep learning; Distributed deep learning; gradient optimization; Graphics processing units; heat equation; Heating systems; Industrial applications; Inverse problems; Linear programming; Mathematical models; Optimization; Parallel processing; Partial differential equations; Solvers; Thermodynamics; Training
• ISSN: 0278-0070; 1937-4151
• DOI: 10.1109/TCAD.2023.3296370

The inversion problem of partial differential equation plays a crucial role in cyber-physical systems applications. This paper presents a novel deep learning optimization approach to constructing a solver of heat equation inversion. To improve the computational efficiency in large-scale industrial applications, data and model parallelisms are incorporated on a platform of multiple GPUs. The advanced Ring-AllReduce architecture is harnessed to achieve an acceleration ratio of 3.46. Then a new multi-GPUs distributed optimization method GradReduce is proposed based on Ring-AllReduce architecture. This method optimizes the original data communication mechanism based on mechanical time and frequency by introducing the gradient transmission scheme solved by linear programming. The experimental results show that the proposed method can achieve an acceleration ratio of 3.84 on a heterogeneous system platform with two CPUs and four GPUs.