Deep Learning and Symbolic Regression for Discovering Parametric Equations

• Published in: IEEE transaction on neural networks and learning systems 2024-11, Vol.35 (11), p.16775-16787
• Main Authors: Zhang, Michael, Kim, Samuel, Lu, Peter Y., Soljacic, Marin
• Format: Article
• Language: English
• Subjects: Deep learning; equation discovery; high-dimensional; Logic gates; Mathematical models; Neural networks; partial differential equation (PDE); Partial differential equations; Physics; Training
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2023.3297978

Symbolic regression is a machine learning technique that can learn the equations governing data and thus has the potential to transform scientific discovery. However, symbolic regression is still limited in the complexity and dimensionality of the systems that it can analyze. Deep learning, on the other hand, has transformed machine learning in its ability to analyze extremely complex and high-dimensional datasets. We propose a neural network architecture to extend symbolic regression to parametric systems where some coefficient may vary, but the structure of the underlying governing equation remains constant. We demonstrate our method on various analytic expressions and partial differential equations (PDEs) with varying coefficients and show that it extrapolates well outside of the training domain. The proposed neural-network-based architecture can also be enhanced by integrating with other deep learning architectures such that it can analyze high-dimensional data while being trained end-to-end. To this end, we demonstrate the scalability of our architecture by incorporating a convolutional encoder to analyze 1-D images of varying spring systems.