Opening the Black Box of the Radiation Belt Machine Learning Model

• Published in: Space Weather 2023-04, Vol.21 (4), p.n/a
• Main Authors: Ma, Donglai, Bortnik, Jacob, Chu, Xiangning, Claudepierre, Seth G., Ma, Qianli, Kellerman, Adam
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Black boxes; Earth; Earth magnetosphere; Electron flux; Fluctuations; Machine learning; Mathematical models; Model accuracy; Modelling; Neural networks; Outer radiation belt; Radiation; Radiation belt electrons; Radiation belts; Solar wind; Storms
• ISSN: 1542-7390; 1539-4964; 1542-7390
• DOI: 10.1029/2022SW003339

Many Machine Learning (ML) systems, especially deep neural networks, are fundamentally regarded as black boxes since it is difficult to fully grasp how they function once they have been trained. Here, we tackle the issue of the interpretability of a high‐accuracy ML model created to model the flux of Earth's radiation belt electrons. The Outer RadIation belt Electron Neural net (ORIENT) model uses only solar wind conditions and geomagnetic indices as input features. Using the Deep SHAPley additive explanations (DeepSHAP) method, for the first time, we show that the “black box” ORIENT model can be successfully explained. Two significant electron flux enhancement events observed by Van Allen Probes during the storm interval of 17–18 March 2013 and non‐storm interval of 19–20 September 2013 are investigated using the DeepSHAP method. The results show that the feature importance calculated from the purely data‐driven ORIENT model identifies physically meaningful behavior consistent with current physical understanding. This work not only demonstrates that the physics of the radiation belt was captured in the training of our previous model, but that this method can also be applied generally to other similar models to better explain the results and to potentially discover new physical mechanisms. Plain Language Summary A neural network is regarded as a black box model since it can approximate any function but its structure won't give any insights on the nature of the function being approximated. A set of neural network models named Outer RadIation belt Electron Neural net have been developed previously to model the electron flux of the outer radiation belt. In this work, we demonstrate the general flow of explaining the machine learning (ML) model of radiation belts and investigate two typical events during the storm and non‐storm times. The results identify physically meaningful behavior and are consistent with current physical understanding, additionally providing new insight into radiation belt dynamics. Furthermore, the proposed framework can be generalized for a variety of other ML models, including various plasma parameters in the Earth's magnetosphere. Key Points We demonstrate the feature attribution method for a machine learning model of electron flux We quantify the effects of geomagnetic indices and solar wind parameters on electron flux during a storm time event and a non‐storm event Our feature importance results identify physical effects that are co