Diffeomorphic Flows via ResNet Discretization

• The topic is defined as a continuous diffeomorphic flow parameterized by a residual network, where the Euler discretization links deep network blocks to smooth invertible transformations.
• Key methodologies include enforcing Lipschitz continuity and regularization constraints to maintain global invertibility and geometric stability across applications.
• Practical applications span shape registration, time-series alignment, and generative modeling, with empirical benchmarks demonstrating improved reconstruction and topology preservation.

A diffeomorphic flow parameterized by a residual network is a mathematical and computational framework in which the traditional residual network (ResNet) architecture is interpreted as a discretization of a continuous-time flow of diffeomorphisms. This conceptual link provides a rigorous foundation for understanding the invertibility, stability, and geometric characteristics of deep networks, especially in structured domains such as shape analysis, generative modeling, registration, and time series alignment.

1. Continuous-Time Formulation and Discretization

Diffeomorphic flows arise from the integration of a time-dependent smooth vector field $v_t : \mathbb{R}^N \to \mathbb{R}^N$ through the ordinary differential equation (ODE)

$$\frac{d}{dt}\,\phi_t(x) = v_t(\phi_t(x)), \quad \phi_0(x) = x,$$

where $\phi_t$ is the flow at time $t$, and under standard smoothness and Lipschitz assumptions on $v_t$, each $\phi_t$ is a diffeomorphism.

The standard forward Euler discretization for step size $h$ yields

$$\phi_{k+1}(x) = \phi_k(x) + h\, v_{t_k}(\phi_k(x)), \quad \phi_0(x)=x.$$

This iteration has exactly the form of a ResNet update: $x_{k+1} = x_k + F(x_k, \theta_k),$ where the residual block $F(x, \theta_k) \approx h\, v_{t_k}(x)$ is parameterized by $\theta_k$ (Rousseau et al., 2018). Thus, an $L$-layer ResNet is an $L$-step explicit Euler discretization of a continuous diffeomorphic flow.

2. Conditions Ensuring Diffeomorphic Properties

Invertibility and smoothness of the learned mapping are critical in applications such as probabilistic flows, shape registration, and temporal alignment. Sufficient conditions for the flow $\phi_t$ to remain a global diffeomorphism throughout the integration are:

• $v_t(x)$ is $C^1$ in both $t$ and $x$;
• Uniform Lipschitz bound: $\|D v_t(x)\| \leq K < \infty$;
• Integrability in time to ensure the flow exists for the full interval.

For residual networks used in generative flows, an additional strict Lipschitz bound $\|g\|_{\text{Lip}} < 1$ on the non-identity part $g(x)$ ensures global invertibility via the Banach fixed-point theorem, and both the forward and inverse mapping are continuously differentiable—ensuring $f(x) = x + g(x)$ is a diffeomorphism (Chen et al., 2019, Koenen et al., 2021).

Statically parameterized flows—i.e., shared-weight ResNets—correspond to flows generated by stationary vector fields, linking the network's mapping to exponential diffeomorphic operators (Rousseau et al., 2018).

3. Geometric and Metric Structure: Riemannian and Sub-Riemannian Flows

The geometric characterization of diffeomorphic flows leverages a (sub-)Riemannian metric on the space of diffeomorphisms. The "energy" of a flow is defined as

$$E[\phi] = \frac{1}{2} \int_0^1 \|v_t\|^2_{\mathcal{V}}\,dt,$$

where $\mathcal{V}$ is a Hilbert space of velocity fields with inner product $$\langle u, w \rangle_{\mathcal V} = \int_{\mathbb R^N} (L u(x)) \cdot w(x)\,dx$$ and a positive differential operator $L$. Minimizing regularized loss functions with this energy as a penalty enforces geodesics in the diffeomorphism group (Rousseau et al., 2018, Jansson et al., 2022).

In landmark matching and LDDMM-based registration, the velocity field is often restricted to a kernel-induced subspace, and the ODE system and its discretization naturally admit a ResNet structure: $$q^{k+1}_i = q^k_i + h\,v_{t_k}(q^k_i), \quad v_{t_k}(x) = \sum_{j} K(x, q^k_j) u^k_j.$$ This interpretation directly connects optimal control problems in sub-Riemannian geometry to a ResNet recursion (Jansson et al., 2022).

4. Computational Realizations and Regularization

To ensure stable, invertible flows via residual networks, step-size constraints on the Euler discretization (e.g., $h < 1/K$ where $K$ is a global Lipschitz constant) are necessary to prevent folding or numerical blowup (Rousseau et al., 2018). Depth and block size trade off accuracy versus computational cost; increasing layers while keeping total flow time fixed improves approximation without compromising geometry.

Regularization schemes include:

• Explicit $\ell_2$ penalties to control vector field norms and enforce smoothness;
• Jacobian or geodesic-length penalties for topology preservation and minimal-energy deformation;
• Inverse consistency terms (e.g., $\|\phi_1 \circ \phi_{-1} - \mathrm{Id}\|^2$) to ensure numerical reversibility (Rousseau et al., 2018, Salman et al., 2018).

Over-regularization can render deformations too rigid (underfitting), while under-regularization can result in non-diffeomorphic (unstable) mappings (Jansson et al., 2022).

5. Extensions and Application Domains

The framework of diffeomorphic flows parameterized by residual networks admits extensive applications:

• Shape registration and medical image analysis: Flows represented via NODE or residual blocks integrate topology-preserving deformations between anatomical or geometric templates and observations (Amor et al., 2021, Sun et al., 2022, Joshi et al., 2023).
• Time-series alignment: Diffeomorphic ResNets align temporal signals without loss of invertibility or monotonicity (Huang et al., 2021).
• Generative modeling: Density functions are learned as push-forwards under invertible ResNets, with unbiased log-determinant estimation via trace-series (Russian roulette and Hutchinson estimators) (Chen et al., 2019).
• Mesh generation: Diffeomorphic flows coded as residual ODEs yield genus-preserving, manifold meshes, outperforming discrete methods in topological robustness (Gupta et al., 2020).
• Control-theoretic universal approximation: Any diffeomorphism homotopic to the identity can be approximated by a flow of a linear control system, discretized into residual blocks (Scagliotti, 2021).

6. Theoretical Generalizations and Practical Algorithmic Considerations

The requirement of smooth diffeomorphisms in normalizing flows and geometrically-constrained architectures can be relaxed to the notion of $\mathcal{L}$-diffeomorphisms, where invertibility and differentiability hold almost everywhere, allowing for ReLU-based or piecewise-smooth activations. Composition and density transformation laws continue to hold in this setting, broadening the admissible architecture set for diffeomorphic flows (Koenen et al., 2021).

Key implementation recipes include spectral or mixed-norm regularization of all linear layers to enforce contractivity, the use of non-saturating activations (e.g., LipSwish), explicit computation or stochastic estimation of log-Jacobian determinants, and backpropagation formulas that decouple memory cost from network depth or power-series truncation (Chen et al., 2019, Koenen et al., 2021).

7. Empirical Evidence and Benchmarks

Empirical studies across domains document that residual-network parameterizations of diffeomorphic flows:

• Achieve state-of-the-art or near-optimal performance in shape registration, density estimation, and classification (Amor et al., 2021, Salman et al., 2018, Chen et al., 2019, Gupta et al., 2020);
• Produce mappings with strictly positive Jacobian determinants, strictly invertible, and topology-preserving, as verified in synthetic and real datasets (Rousseau et al., 2018, Gupta et al., 2020);
• Require minimal or no explicit topology regularization when the continuous flow formulation is enforced by network and integration design (Sun et al., 2022, Gupta et al., 2020);
• Exhibit improved sample quality, lower data-fitting error, and accurate correspondences and reconstructions compared to non-diffeomorphic or discrete analogs.

In sum, viewing ResNets and other deep blocks as discretized diffeomorphic flows yields both practical and theoretical benefits. It enables invertible, smooth, and geometrically meaningful transformations, supporting a wide array of applications from machine learning and computational anatomy to physics-based simulation (Rousseau et al., 2018, Amor et al., 2021, Sun et al., 2022, Chen et al., 2019, Gupta et al., 2020).