TVSpecNET: Deep TV Spectral Decomposition

• TVSpecNET is a deep CNN designed to approximate nonlinear total variation (TV) spectral decompositions, achieving near-perfect fidelity to model-based methods.
• It leverages learned representations to recover all spectral bands in real time (~0.002 s per 1-megapixel image), offering up to 10,000× speedup over traditional solvers.
• TVSpecNET enables practical applications in filtering, feature transfer, and image fusion by capturing image structures at multiple nonlinear scales.

TVSpecNET is a deep convolutional neural network designed to approximate non-linear spectral decompositions of images derived from one-homogeneous functionals, specifically total variation (TV). By learning to produce total variation (TV) spectral bands from raw images, TVSpecNET achieves up to four orders of magnitude speedup (≈10,000×) over traditional GPU-based gradient-flow solvers while maintaining near-perfect fidelity to model-based non-linear spectral TV decompositions. TVSpecNET recovers all spectral bands in real time (≈0.002 s for a 1-megapixel image) and enables applications in filtering, feature transfer, and image fusion, which rely on the representation of image structure at multiple nonlinearly defined "scales" or spectral components (Grossmann et al., 2020).

1. Total Variation Spectral Decomposition

Non-linear spectral TV decomposition provides a fully non-linear “frequency-like” representation of images, which differs fundamentally from classical linear transformations (such as Fourier or wavelet decompositions). The formulation is based on the total variation functional $J_{TV}(u) = \int_\Omega |Du|$, where $\Omega \subset \mathbb{R}^N$ denotes the image domain.

The TV scale-space of an image $f$ is generated via the TV-flow (gradient flow of $J_{TV}$):

$$u_t(t,x) \in -\partial J_{TV}(u(t,x)), \quad u(0,x) = f(x)$$

or, in its regularized PDE form,

$$\partial_t u = \text{div}\left(\frac{\nabla u}{|\nabla u|}\right)$$

The spectral response is parameterized as

$\varphi(t, x) = u_{tt}(t, x) \cdot t$

The original image can be perfectly reconstructed by integrating the spectrum:

$$f(x) = \int_{0}^{\infty} \varphi(t, x)\, dt + \bar{f}$$

where $\bar{f}$ is the