H-Revolve: A Framework for Adjoint Computation on Synchronous Hierarchical Platforms

We study the problem of checkpointing strategies for adjoint computation on synchronous hierarchicalplatforms, specifically computational platforms with several levels of storage with different writing andreading costs. When reversing a large adjoint chain, choosing which data to checkpoint and wher...

Full description

Saved in:
Bibliographic Details
Published in:ACM transactions on mathematical software 2020-06
Main Authors: Herrmann, Julien, Pallez, Guillaume
Format: Article
Language:English
Subjects:
Computer Science
Mathematical Software
Numerical Analysis
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We study the problem of checkpointing strategies for adjoint computation on synchronous hierarchicalplatforms, specifically computational platforms with several levels of storage with different writing andreading costs. When reversing a large adjoint chain, choosing which data to checkpoint and where is a criticaldecision for the overall performance of the computation. We introduce H-Revolve, an optimal algorithm forthis problem. We make it available in a public Python library along with the implementation of several state-of-the-art algorithms for the variant of the problem with two levels of storage. We provide a detailed descriptionof how one can use this library in an adjoint computation software in the field of automatic differentiationor backpropagation. Finally, we evaluate the performance of H-Revolve and other checkpointing heuristicsthough an extensive campaign of simulation.
ISSN:0098-3500
DOI:10.1145/3378672