Differentiable Programming for Differential Equations: A Review

The differentiable programming paradigm is a cornerstone of modern scientific computing. It refers to numerical methods for computing the gradient of a numerical model's output. Many scientific models are based on differential equations, where differentiable programming plays a crucial role in...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-06
Main Authors: Sapienza, Facundo, Bolibar, Jordi, Schäfer, Frank, Groenke, Brian, Pal, Avik, Boussange, Victor, Heimbach, Patrick, Hooker, Giles, Pérez, Fernando, Persson, Per-Olof, Rackauckas, Christopher
Format: Article
Language:English
Subjects:
Computation
Differential equations
Machine learning
Mathematical analysis
Numerical methods
Numerical models
Parameter sensitivity
Programming
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The differentiable programming paradigm is a cornerstone of modern scientific computing. It refers to numerical methods for computing the gradient of a numerical model's output. Many scientific models are based on differential equations, where differentiable programming plays a crucial role in calculating model sensitivities, inverting model parameters, and training hybrid models that combine differential equations with data-driven approaches. Furthermore, recognizing the strong synergies between inverse methods and machine learning offers the opportunity to establish a coherent framework applicable to both fields. Differentiating functions based on the numerical solution of differential equations is non-trivial. Numerous methods based on a wide variety of paradigms have been proposed in the literature, each with pros and cons specific to the type of problem investigated. Here, we provide a comprehensive review of existing techniques to compute derivatives of numerical solutions of differential equations. We first discuss the importance of gradients of solutions of differential equations in a variety of scientific domains. Second, we lay out the mathematical foundations of the various approaches and compare them with each other. Third, we cover the computational considerations and explore the solutions available in modern scientific software. Last but not least, we provide best-practices and recommendations for practitioners. We hope that this work accelerates the fusion of scientific models and data, and fosters a modern approach to scientific modelling.
ISSN:2331-8422