Visualizing probabilistic models and data with Intensive Principal Component Analysis

• Published in: Proceedings of the National Academy of Sciences - PNAS 2019-07, Vol.116 (28), p.13762-13767
• Main Authors: Quinn, Katherine N., Clement, Colin B., De Bernardis, Francesco, Niemack, Michael D., Sethna, James P.
• Format: Article
• Language: English
• Subjects: Big Bang theory; Cold dark matter; Cold spinning; Cosmic microwave background; Dark energy; Dark matter; Data points; Embedding; Ising model; Isometric; Mathematical models; Neural networks; Physical Sciences; Principal components analysis; Probabilistic models; Statistical analysis; Statistical mechanics
• ISSN: 0027-8424; 1091-6490
• DOI: 10.1073/pnas.1817218116

Unsupervised learning makes manifest the underlying structure of data without curated training and specific problem definitions. However, the inference of relationships between data points is frustrated by the “curse of dimensionality” in high dimensions. Inspired by replica theory from statistical mechanics, we consider replicas of the system to tune the dimensionality and take the limit as the number of replicas goes to zero. The result is intensive embedding, which not only is isometric (preserving local distances) but also allows global structure to be more transparently visualized. We develop the Intensive Principal Component Analysis (InPCA) and demonstrate clear improvements in visualizations of the Ising model of magnetic spins, a neural network, and the dark energy cold dark matter (ΛCDM) model as applied to the cosmic microwave background.