MemXCT: Design, Optimization, Scaling, and Reproducibility of X-Ray Tomography Imaging

• Published in: IEEE transactions on parallel and distributed systems 2021-11, Vol.33 (9)
• Main Authors: Hidayetoglu, Mert, Bicer, Tekin, de Gonzalo, Simon Garcia, Ren, Bin, Gursoy, Doga, Kettimuthu, Rajkumar, Foster, Ian T., Hwu, Wen-Mei W.
• Format: Article
• Language: English
• Subjects: cache utilization; GPU computing; Knights Landing; OTHER INSTRUMENTATION; space-filling curves; SpMV; x-ray tomography
• ISSN: 1045-9219; 1558-2183

Here, this work extends our previous research entitled "MemXCT: Memory-centric X-ray CT Reconstruction with Massive Parallelization" that was originally published at SC19 conference (Hidayetoglu et al., 2019) with reproducibility of the computational imaging performance. X-ray computed tomography (XCT) is regularly used at synchrotron light sources to study the internal morphology of materials at high resolution. However, experimental constraints, such as radiation sensitivity, can result in noisy or undersampled measurements. Further, depending on the resolution, sample size and data acquisition rates, the resulting noisy dataset can be in the order of terabytes. Advanced iterative reconstruction techniques can produce high-quality images from noisy measurements, but their computational requirements have made their use an exception rather than the rule. We propose a novel memory-centric approach that avoids redundant computations at the expense of additional memory complexity. We develop a memory-centric iterative reconstruction system, MemXCT, that uses an optimized SpMV implementation with two-level pseudo-Hilbert ordering and multi-stage input buffering. We evaluate MemXCT on various supercomputer architectures involving KNL and GPU. MemXCT can reconstruct a large (11Kx11K) mouse brain tomogram in 10 seconds using 4096 KNL nodes (256K cores). The results presented in our original article at the SC19 were based on large-scale supercomputing resources. The MemXCT application was selected for the Student Cluster Competition (SCC) Reproducibility Challenge and evaluated on a variety of cloud computing resources by universities around the world in the SC20 conference. We summarize the results of the top-ranked SCC Reproducibility Challenge teams and identify the most pertinent measures for ensuring the reproducibility of our experiments in this article.