Inexact Derivative-Free Optimization for Bilevel Learning

• Published in: Journal of mathematical imaging and vision 2021-06, Vol.63 (5), p.580-600
• Main Authors: Ehrhardt, Matthias J., Roberts, Lindon
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; Applications of Mathematics; Computer Science; Exact solutions; Image Processing and Computer Vision; Learning; Mathematical Methods in Physics; Noise reduction; Optimization; Parameters; Regularization; Signal,Image and Speech Processing
• ISSN: 0924-9907; 1573-7683
• DOI: 10.1007/s10851-021-01020-8

Variational regularization techniques are dominant in the field of mathematical imaging. A drawback of these techniques is that they are dependent on a number of parameters which have to be set by the user. A by-now common strategy to resolve this issue is to learn these parameters from data. While mathematically appealing, this strategy leads to a nested optimization problem (known as bilevel optimization) which is computationally very difficult to handle. It is common when solving the upper-level problem to assume access to exact solutions of the lower-level problem, which is practically infeasible. In this work we propose to solve these problems using inexact derivative-free optimization algorithms which never require exact lower-level problem solutions, but instead assume access to approximate solutions with controllable accuracy, which is achievable in practice. We prove global convergence and a worst-case complexity bound for our approach. We test our proposed framework on ROF denoising and learning MRI sampling patterns. Dynamically adjusting the lower-level accuracy yields learned parameters with similar reconstruction quality as high-accuracy evaluations but with dramatic reductions in computational work (up to 100 times faster in some cases).