Impact of non-smooth observation operators on variational and sequential data assimilation for a limited-area shallow-water equation model

• Published in: Quarterly journal of the Royal Meteorological Society 2012-01, Vol.138 (663), p.323-339
• Main Authors: Steward, J. L., Navon, I. M., Zupanski, M., Karmitsa, N.
• Format: Article
• Language: English
• Subjects: 4D-Var; Earth, ocean, space; Exact sciences and technology; External geophysics; L-BFGS; LMBM; Meteorology; MLEF; non-smooth optimization; Physics of the high neutral atmosphere
• ISSN: 0035-9009; 1477-870X
• DOI: 10.1002/qj.935

We investigate the issue of variational and sequential data assimilation with nonlinear and non‐smooth observation operators using a two‐dimensional limited‐area shallow‐water equation model and its adjoint. The performance of the four‐dimensional variational approach (4D‐Var: two dimensions plus time) compared with that of the maximum‐likelihood ensemble filter (MLEF), a hybrid ensemble/variational method, is tested in the presence of non‐smooth observation operators. Following the work of Lewis & Overton and Karmitsa, we investigate minimization of the data‐assimilation cost functional using the limited‐memory Broyden–Fletcher–Goldfarb–Shanno (L‐BFGS) quasi‐Newton algorithm originally intended for smooth optimization and the limited‐memory bundle method (LMBM) algorithm specifically designed to address large‐scale non‐smooth minimization problems. Numerical results obtained for the MLEF method show that the LMBM algorithm yields results superior to the L‐BFGS method. Results for 4D‐Var suggest that L‐BFGS performs well when the non‐smoothness is not extreme, but fails for non‐smooth functions with large Lipschitz constants. The LMBM method is found to be a suitable choice for large‐scale non‐smooth optimization, although additional work is needed to improve its numerical stability. Finally, the results and methodologies of 4D‐Var and MLEF are compared and contrasted. Copyright © 2011 Royal Meteorological Society