Application of deep learning to the estimation of normalization coefficients in diffusion-based covariance models

Abstract: Variational data assimilation in ocean models depends on the ability to model general correlation operators in the presence of coastlines. Grid-point filters based on diffusion operators are widely used for this purpose, but come with a computational bottleneck - the costly estimation of normalization factors for every model grid point. In this paper, we show that a simple convolutional neural network can effectively learn these normalization factors with better accuracy than the current operational methods. Our network is tested with a two-dimensional diffusion operator from the NEMOVAR ocean data assimilation system, applied to a global ocean grid with approximately one degree horizontal resolution. The network is trained on exact normalization factors estimated by a brute-force method. Knowing that convolutional networks can only model translation-equivariant functions, we ensure that the normalization estimation problem is indeed translation-equivariant. Specifically, we show how the number of inputs of this problem can be reduced while preserving translation equivariance. Adding the distance to the coastline as an input channel is found to improve the performance of the network around coastlines. Extensions to three-dimensional diffusion and to higher horizontal resolutions are discussed. Removing the computational bottleneck associated with normalization opens the way to using adaptive correlation models for operational ocean data assimilation. The code for this work is publicly available at https://github.com/FolkeKS/DL-normalization/tree/core-features