MagPySV: A Python Package for Processing and Denoising Geomagnetic Observatory Data

• Published in: Geochemistry, geophysics, geosystems : G3 geophysics, geosystems : G3, 2018-09, Vol.19 (9), p.3347-3363
• Main Authors: Cox, G. A., Brown, W. J., Billingham, L., Holme, R.
• Format: Article
• Language: English
• Subjects: Case studies; Corrections; Current data; Data analysis; Data processing; Geomagnetic field; Geomagnetic storms; Observatories; Principal components analysis; Regions; Resolution; Solar activity; Statistical methods; Storms
• ISSN: 1525-2027; 1525-2027
• DOI: 10.1029/2018GC007714

Measurements obtained at ground‐based observatories are crucial to understanding the geomagnetic field and its secular variation (SV). However, current data processing methods rely on piecemeal closed‐source codes or are performed on an ad hoc basis, hampering efforts to reproduce data sets underlying published results. We present MagPySV, an open‐source Python package designed to provide a consistent and automated means of generating high‐resolution SV data sets from hourly means distributed by the Edinburgh World Data Centre. It applies corrections for documented baseline changes, and optionally, data may be excluded using the ap index, which removes effects from documented high solar activity periods such as geomagnetic storms. Robust statistics are used to identify and remove outliers. Developing existing denoising methods, we use principal component analysis of the covariance matrix of residuals between observed SV and that predicted by a global field model to remove a proxy for external field contamination from observations. This method creates a single covariance matrix for all observatories of interest combined and applies the denoising to all locations simultaneously, resulting in cleaner time series of the internally generated SV. In our case studies, we present cleaned data in two geographic regions: monthly first differences are used to investigate geomagnetic jerk morphology in Europe, an area previously well‐studied at lower resolution, and annual differences are investigated for northern high latitude regions, which are often neglected due to their high noise content. MagPySV may be run on the command line or within an interactive Jupyter notebook; two notebooks reproducing the case studies are supplied. Key Points MagPySV is an open‐source Python package for creating reproducible high‐resolution time series of internal secular variation Implemented denoising method uses principal component analysis to characterize external contamination in different geographic regions Denoised data from MagPySV and their application to geomagnetic jerks presented in case studies for Europe and high northern latitudes