Pace v0.2: a Python-based performance-portable atmospheric model

• Published in: Geoscientific Model Development 2023-05, Vol.16 (9), p.2719-2736
• Main Authors: Dahm, Johann, Davis, Eddie, Deconinck, Florian, Elbert, Oliver, George, Rhea, McGibbon, Jeremy, Wicky, Tobias, Wu, Elynn, Kung, Christopher, Ben-Nun, Tal, Harris, Lucas, Groner, Linus, Fuhrer, Oliver
• Format: Article
• Language: English
• Subjects: Algorithms; Analysis; Atmospheric models; Atmospheric physics; Climate models; Cloud microphysics; Design; Domain specific languages; Driving ability; FORTRAN; Hardware; High level languages; High performance computing; Language; Machine learning; Maintainability; Microphysics; Moore's law; New technology; Optimization; Productivity; Simulation; Software; Supercomputers
• ISSN: 1991-9603; 1991-959X; 1991-962X; 1991-9603; 1991-962X
• DOI: 10.5194/gmd-16-2719-2023

Progress in leveraging current and emerging high-performance computing infrastructures using traditional weather and climate models has been slow. This has become known more broadly as the software productivity gap. With the end of Moore's law driving forward rapid specialization of hardware architectures, building simulation codes on a low-level language with hardware-specific optimizations is a significant risk. As a solution, we present Pace, an implementation of the nonhydrostatic FV3 dynamical core and GFDL cloud microphysics scheme which is entirely Python-based. In order to achieve high performance on a diverse set of hardware architectures, Pace is written using the GT4Py domain-specific language. We demonstrate that with this approach we can achieve portability and performance, while significantly improving the readability and maintainability of the code as compared to the Fortran reference implementation. We show that Pace can run at scale on leadership-class supercomputers and achieve performance speeds 3.5–4 times faster than the Fortran code on GPU-accelerated supercomputers. Furthermore, we demonstrate how a Python-based simulation code facilitates existing or enables entirely new use cases and workflows. Pace demonstrates how a high-level language can insulate us from disruptive changes, provide a more productive development environment, and facilitate the integration with new technologies such as machine learning.