---
title: Backward-Coherence Regularisation Overview
url: https://www.emergentmind.com/topics/backward-coherence-regularisation
type: topic
---

# Backward-Coherence Regularisation Overview

to=arxiv_search.search  天天送json  {
  "query": "\"backward coherence\" regularisation OR \"backward-coherence\"",
  "max_results": 10,
  "sort_by": "submittedDate"
}
to=arxiv_search.search  天天中彩票这个json  {
  "query": "\"inverse consistency\" registration regularization",
  "max_results": 10,
  "sort_by": "submittedDate"
}
Backward-Coherence Regularisation denotes a family of regularisation strategies in which model admissibility is constrained by consistency under a backward relation. In the available literature, that backward relation appears as inverse-consistent composition of spatial maps, bidirectional temporal prediction obtained by swapping baseline and follow-up scans, reconstructibility of an RNN hidden state from its successor, preservation of instantaneous phase while regularising only magnitude in coherent inverse problems, and coherence-enhancing constraints that stabilise backward reconstruction in diffusion MRI [2105.04459] [2509.03141] [2606.08934] [2410.22161] [1412.1439].

## 1. Scope and common structure

Across domains, backward-coherence regularisation is not a single canonical penalty but a design pattern. The regulariser couples a forward object to a backward map or backward direction: \(\phi\) to \(\psi\) in deformable registration, \(I_{T_a}\) to \(I_{T_b}\) and back in longitudinal MRI prediction, \(h_t\) to \(h_{t+1}\) in recurrent dynamics, and \(z\) to \(|z|\) with phase preserved in coherent imaging [2105.04459] [2509.03141] [2606.08934] [2410.22161]. This suggests a unifying description: the regularisation acts by shrinking the feasible set to configurations that remain mutually compatible under inversion, reversal, or backward projection.

| Setting | Backward relation | Regularisation form |
|---|---|---|
| Deformable registration | \(\phi \circ \psi \approx Id\), \(\psi \circ \phi \approx Id\) | \(L_{ic}\) |
| Longitudinal diffusion MRI | baseline/follow-up swap, \(\Delta \mapsto -\Delta\) | BITR |
| RNN hidden states | \(h_t \approx g_\phi(h_{t+1})\) | \(L_{\mathrm{bc}}\) |
| Coherent inverse problems | preserve \(\arg(z)\), regularise \(|z|\) | \(\operatorname{prox}_{H(|\cdot|)}\) |

A recurrent feature is that backward coherence is embedded inside the optimisation or training objective rather than applied as post-processing. In registration, it is part of the loss; in longitudinal diffusion, it is a training-time direction swap; in RNNs, it augments the task objective; in coherent inverse problems, it changes the proximal step itself. The result is a regulariser whose semantics are task-specific but whose operational role is to suppress implausible forward solutions that cannot be reconciled with an associated backward operation.

## 2. Inverse consistency in deformable registration

ICON defines backward-coherence as inverse consistency between a forward map \(\phi : \Omega \to \Omega\) and a backward map \(\psi : \Omega \to \Omega\), with \(\Omega = [0,1]^d\) and displacement parameterisation \(\phi = Id + D^{AB}\), \(\psi = Id + D^{BA}\) [2105.04459]. The core loss is
\[
L_{ic}(\phi,\psi) = \|\phi \circ \psi - Id\|_2^2 + \|\psi \circ \phi - Id\|_2^2,
\]
and, because ICON composes maps via interpolation, it is evaluated in displacement form as
\[
L_{ic}(\phi,\psi) = \|D^{AB} \circ \psi + D^{BA}\|_2^2 + \|D^{BA} \circ \phi + D^{AB}\|_2^2.
\]
This penalises departures from mutual invertibility: when \(L_{ic} \to 0\), \(\psi \approx \phi^{-1}\) and \(\phi \approx \psi^{-1}\).

A crucial implementation detail is randomized off-grid evaluation. ICON samples points \(x_i + \epsilon_i\), with \(\epsilon_i\) drawn from a multivariate Gaussian whose per-dimension standard deviation equals the pixel or voxel spacing, and evaluates the inverse-consistency residual away from lattice centres. The stated rationale is that on-grid evaluation alone allows a “swap” of neighbouring grid points in a textureless region to compose to the identity at those centres while still folding between them. Off-grid sampling exposes this non-invertibility within cells and raises \(L_{ic}\).

ICON also supplies a theoretical intuition for why inverse consistency enforces spatial regularity. Under small, independent spatial white-noise perturbations of the output maps before composition, the expected inverse-consistency term expands into the original coherence loss plus terms proportional to
\[
\epsilon^2 \|D\psi \sqrt{Jac(\psi)}\|_2^2 + \epsilon^2 \|D\phi \sqrt{Jac(\phi)}\|_2^2.
\]
The paper interprets this as an implicit \(H^1\)-type regularisation weighted by local volume change. Even without explicit injected noise, ICON argues that training error in inverse consistency acts similarly because finite network capacity and stochastic training prevent exact satisfaction of the inverse-consistency equations across a population.

The full training objective is
\[
L(\theta; I_A, I_B) = \|I_A \circ \phi - I_B\|_2^2 + \|I_B \circ \psi - I_A\|_2^2 + \lambda \big[\|\phi \circ \psi - Id\|_2^2 + \|\psi \circ \phi - Id\|_2^2\big].
\]
ICON does not add tuned smoothness, elastic, diffusion, or bending-energy penalties such as \(\int_\Omega \|\nabla \phi\|^2\). Continuity and differentiability follow from linear interpolation except on a measure-zero set, approximate invertibility is enforced by \(L_{ic}\), and folding is quantified via the fraction of voxels where \(\det(D\phi) \le 0\).

Empirically, inverse consistency plus off-grid interpolation reduces folds compared to similarity alone. On synthetic shapes, U-Net achieved Dice \(\approx 0.95\)–\(0.98\) with folds reduced to \(\approx 0.09\)–\(1.59\%\) as \(\lambda\) grows; on the 3D OAI knee dataset, ICON (2 half-res + 2 full-res) reported Dice \(68.29\), folds \(118.4\), and time \(1.06\) s, while SyN (CC) achieved Dice \(65.71\) with \(0\) folds but required \(1330\) s. ICON therefore presents backward coherence as a standalone regulariser for approximately diffeomorphic maps, while also stating that backward-coherence alone does not strictly guarantee diffeomorphisms at test time.

## 3. Bidirectional temporal regularisation in longitudinal diffusion models

In TADM-3D, backward-coherence regularisation appears as Back-In-Time Regularisation (BITR), a training strategy for a 3D Denoising Diffusion Probabilistic Model that predicts a future MRI from a baseline volume and a desired time interval \(\Delta_{a,b} = T_b - T_a\) [2509.03141]. The model predicts the voxel-wise intensity residual
\[
I_{\Delta_{a,b}} = I_{T_b} - I_{T_a},
\]
and reconstructs the follow-up through
\[
\widehat{I}_{T_b} = I_{T_a} + \widehat{I}_{\Delta_{a,b}}.
\]
Temporal conditioning uses a baseline latent \(z_a = \Phi(I_{T_a})\), the interval \(\Delta_{a,b}\), age at baseline \(A\), and cognitive status \(D\).

BITR is implemented by randomly swapping \((I_{T_a}, I_{T_b})\) with probability \(p = 0.5\). In the forward direction, the model predicts \(I_{T_b}\) from \(I_{T_a}\) using \(\Delta_{a,b} > 0\); in the backward direction, it predicts \(I_{T_a}\) from \(I_{T_b}\) using \(\Delta_{b,a} = -\Delta_{a,b}\). The paper states that predicting past scans has limited clinical applications, but that this regularisation helps the model generate temporally more accurate scans.

The training objective combines a diffusion \(\epsilon\)-prediction term with an age-consistency penalty from a pre-trained Brain-Age Estimator:
\[
L^{DML} = \mathbb{E}_{\epsilon \sim \mathcal{N}(0,1), \bar{I}_{\Delta_{a,b}, t}}
\big[\| G_\theta(\bar{I}_{\Delta_{a,b}, t; z_a, \Delta_{a,b}, A, D}) - \epsilon \|_2^2\big],
\]
\[
\widehat{\Delta}_{a,b} = \Psi(\Phi(\widehat{I}_{T_b})) - \Psi(\Phi(I_{T_a})),
\qquad
L^{BAE} = (\widehat{\Delta}_{a,b} - \Delta_{a,b})^2,
\]
\[
L^{Tot} = L^{DML} + L^{BAE}.
\]
The paper explicitly states that there is no separate \(L_{\text{forward}}\) and \(L_{\text{backward}}\); BITR is a training-time data-direction augmentation. It also states that it does not add a cycle-consistency loss.

The empirical role of backward coherence is pronounced in the ablations. On the internal OASIS-3 test set, full TADM-3D achieved MSE \(0.004 \pm 0.001\) and SSIM \(0.902 \pm 0.014\), whereas the variant w/o BITR achieved MSE \(0.008 \pm 0.003\) and SSIM \(0.874 \pm 0.021\). The paper summarises this as a substantial degradation when BITR is removed: MSE doubles and SSIM drops. On the external NACC test set, TADM-3D achieved MSE \(0.004 \pm 0.002\) and SSIM \(0.902 \pm 0.017\), with best regional MAE on hippocampus, amygdala, thalamus, and CSF, while BrLP was lowest on lateral ventricle.

The model therefore uses backward coherence not as explicit inversion but as temporal bidirectionality. The stated theoretical intuition is that learning both positive and negative residuals reduces drift and biases, improving sensitivity to the sign and magnitude of \(\Delta t\). A related practical distinction is that conditioning is on age difference rather than absolute age, and that a frozen BAE provides a direct temporal scalar constraint rather than a cycle penalty.

## 4. Hidden-state stability in recurrent neural networks

In recurrent models, backward-coherence regularisation is formulated as probabilistic backward consistency of hidden states: a stable hidden state should be reconstructible from its successor [2606.08934]. For an RNN with hidden state \(h_t \in \mathbb{R}^p\) and update \(h_t = f_\theta(h_{t-1}, x_t)\), the backward filtration is
\[
\mathcal{F}^{\mathrm{bwd}}_t = \sigma(h_s : s \ge t).
\]
Backward coherence requires
\[
h_t \approx \mathbb{E}[h_t \mid h_{t+1}],
\]
with a learned backward projector \(g_\phi : \mathbb{R}^p \to \mathbb{R}^p\) in residual form,
\[
g_\phi(h) = h + W_2\,\mathrm{ReLU}(W_1\,h + b_1) + b_2.
\]

The observable backward residual and reverse-martingale drift surrogate are
\[
\delta_t := h_t - g_\phi(h_{t+1}),
\qquad
r_t^\phi := g_\phi(h_{t+1}) - h_{t+1},
\]
with empirical totals
\[
\hat Q_T = \sum_{t=1}^{T-1}\|\delta_t\|,
\qquad
\hat D_T = \sum_{t=1}^{T-1}\|r_t^\phi\|.
\]
The backward-coherence penalty is
\[
L_{\mathrm{bc}}(\theta,\phi)
:= \frac{1}{T-1}\sum_{t=1}^{T-1}\| h_t - g_\phi(h_{t+1}) \|^2,
\]
and the total objective is
\[
\mathcal{L}_{\mathrm{total}} = \mathcal{L}_{\mathrm{task}} + \lambda\,L_{\mathrm{bc}}, \qquad \lambda > 0.
\]

The theoretical analysis proceeds under contraction in the hidden-state argument,
\[
\|f_\theta(h,x)-f_\theta(h',x)\| \le \rho \|h-h'\|,
\qquad \rho \in (0,1),
\]
backward sufficiency,
\[
\mathbb{E}[h_t \mid \mathcal{F}^{\mathrm{bwd}}_{t+1}] = \mathbb{E}[h_t \mid h_{t+1}]
\quad \text{a.s.},
\]
and summable backward drift. Under these assumptions, the hidden-state sequence forms a reverse quasi-martingale and Theorem 3.1 gives almost-sure convergence when \(\sum_t \mathbb{E}[\|\rho_t\|] < \infty\). The paper also gives \(L^1\) convergence under uniform integrability, geometric convergence under exponentially decaying drift defects, change-point tracking bounds, and time-uniform confidence sequences based on increment-sum tubes and a calibrated defect-tail proxy.

Backward coherence also receives a variational interpretation. With
\[
p_\phi(h_t \mid h_{t+1}) := \mathcal{N}(g_\phi(h_{t+1}), \sigma^2 I_p),
\]
minimising \(L_{\mathrm{bc}}\) over \(\phi\) is equivalent to minimising an expected Kullback–Leibler divergence to a Gaussian backward model with the correct conditional mean. The paper further derives the bound
\[
Q_T \le (T-1)\sqrt{\mathbb{E}[L_{\mathrm{bc}}]},
\]
which links the scalar diagnostic \(Q_T\) to the training loss.

Empirically, backward-coherence regularisation reduces the empirical quasi-martingale total \(\hat Q\) by \(43\)–\(58\%\) and reaches stability \(28\)–\(44\%\) earlier than an unregularised RNN. On PhysioNet 2012 ICU data, the Reverse Martingale RNN matched mortality-prediction AUC, \(0.819\) versus \(0.820\) baseline, while reaching stable representations \(13\) hours earlier. On FRED-MD, it reduced one-month-ahead forecast error by about fourfold under concept drift. On UCI Human Activity Recognition, it maintained lower post-transition tracking error with geometric decay. The paper simultaneously emphasises that the guarantees apply under the stated assumptions and that universality is not claimed.

## 5. Magnitude-only proximal regularisation in coherent inverse problems

In coherent inverse problems such as SAR, backward-coherence regularisation has a different technical meaning: regularise only the magnitudes and preserve the instantaneous phase of the current iterate [2410.22161]. The forward model is \(y = A x + \eta\) with \(x \in \mathbb{C}^n\), and the image is written componentwise as
\[
x = r \odot e^{i\theta},
\qquad
r = |x| \in \mathbb{R}^n_{\ge 0},
\qquad
\theta = \arg(x).
\]
The prior information is assumed to reside primarily in \(r\), while phase is often treated as weakly constrained.

For a proper, closed, convex \(H\) on magnitudes, define
\[
G(x) := H(|x|).
\]
If \(\operatorname{prox}_H : \mathbb{R}^n_{\ge 0} \to \mathbb{R}^n_{\ge 0}\), then the main theorem gives the phase-preserving proximal map
\[
\operatorname{prox}_G(z) = e^{i\arg(z)} \odot \operatorname{prox}_H(|z|).
\]
The paper’s interpretation is that the magnitude penalty acts “backward” with respect to phase: phase is updated solely by the data-fidelity step, while the regulariser corrects only magnitudes.

When the sufficient condition fails, the paper introduces the bounded proximal map
\[
\operatorname{prox}_H^+(r) := \arg\min_{x \in \mathbb{R}^n_{\ge 0}} H(x) + \tfrac{1}{2}\|x-r\|_2^2
= \operatorname{prox}_{H + \chi_{\mathbb{R}^n_{\ge 0}}}(r),
\]
and the correct complex proximal becomes
\[
\operatorname{prox}_G(z) = e^{i\arg(z)} \odot \operatorname{prox}_H^+(|z|).
\]
\(\operatorname{prox}_H^+\) is computed through Douglas–Rachford splitting,
\[
x^{k+1} = \operatorname{prox}_H(y^k),\qquad
y^{k+1} = y^k + \operatorname{prox}_F(2x^{k+1}-y^k) - x^{k+1},
\]
with
\[
\operatorname{prox}_F(y) = \operatorname{proj}_{\mathbb{R}^n_{\ge 0}}\!\big((r+y)/2\big).
\]

This framework is instantiated for magnitude sparsity, generalised Tikhonov, TGV, and level-set regularisation. For \(H(r)=\lambda\|r\|_1\), the induced complex proximal is
\[
\operatorname{prox}_G(z)_k = e^{i\theta_k}\max(|z_k|-\lambda,0).
\]
For \(H(r) = (\lambda/2)\|W r\|_2^2\), the direct formula
\[
\operatorname{prox}_G(z) = e^{i\arg(z)} \odot (I+\lambda W^T W)^{-1}|z|
\]
holds under the stated sufficient conditions; otherwise the Douglas–Rachford correction is used.

The paper demonstrates the method on publicly available real SAR data for generalised Tikhonov regularisation applied to multi-channel SAR, and both a simple level set formulation and total generalised variation applied to the standard single-channel case. The stated benefits are that no nonlinear phase fitting is needed, regularisers do not fight the data term by trying to homogenise phases, and interferometric uses are preserved while magnitude structure is improved. The main limitation is equally explicit: in extreme noise or severe model mismatch, the data term may drive phases erratically, and backward-coherence regularisation will not correct phase ambiguities.

## 6. Antecedents, adjacent formulations, and related coherence principles

Several adjacent formulations illuminate the broader landscape in which backward coherence is used. In diffusion MRI, the paper on a coherence enhancing penalty reconstructs orientation distribution functions \(\psi(x,u)\) from HARDI data by solving a linear inverse problem regularised by fiber continuity [1412.1439]. The geometric assumption is
\[
\psi(x+\tau u, u) \approx \psi(x,u),
\]
which yields
\[
u^T \nabla_x \psi(x,u) \approx 0
\]
and the coherence-enhancing penalty
\[
R_{FC}(\psi) = \int_\Omega \int_{S^2} |u^T \nabla_x \psi(x,u)|^2\,du\,dx.
\]
The full constrained objective is
\[
\|T \psi - S^\delta\|_2^2 + \alpha \|\psi\|_2^2 - \beta \langle \psi,\Delta_{S^2}\psi\rangle
+ \gamma \int_\Omega \int_{S^2}|u^T\nabla_x\psi|^2\,du\,dx,
\]
subject to \(\psi \ge 0\). The paper states that this stabilises the backward reconstruction by enforcing low variance along \(u\)-directions across voxels and proves convergence for discrete, noisy data through compact embedding of \(H_{\mathrm{fib}}\) into \(L^2\) and constrained Tikhonov theory with operator approximation.

A different but mathematically related use of backward induction appears in sequential coherence for predictive modelling [1502.06045]. There, predictive densities satisfy
\[
p_t(y) = \int p_{t+1}(y \mid y_{t+1})\,p_t(y_{t+1})\,dy_{t+1},
\]
or equivalently
\[
E[X_{t+1}(y)\mid F_t] = X_t(y),
\qquad X_t(y) \equiv p(y \mid Y_{1:t}),
\]
so \(\{X_t(y)\}_t\) is a martingale. Starting from a terminal predictive \(p_N\), backward induction yields a time-consistent predictive sequence and, in the kernel density example, produces earlier densities as scale mixtures with stochastic bandwidth inflation. The paper describes this as coherent, prior-free uncertainty assessment and derives Azuma–Hoeffding concentration through bounded one-step predictive differences.

Backward coherence also appears in optimisation-theoretic form through backward error analysis of SGD and differentiable games [2307.05789]. For one-step GD, the modified flow is
\[
\dot{\theta}
= -\nabla E(\theta) - \frac{h}{2}\nabla^2 E(\theta)\nabla E(\theta)
= -\nabla\!\Big(E(\theta)+\frac{h}{4}\|\nabla E(\theta)\|^2\Big).
\]
For multiple consecutive SGD steps on exact batches \(X^t,\dots,X^{t+n-1}\), the paper constructs an iteration-dependent modified loss with a full-batch gradient-norm penalty and an inter-batch gradient alignment term depending explicitly on \(\theta_{t-1}\). In two-player differentiable games it similarly obtains per-iteration scalar modified losses containing self-gradient penalties and cross-player interaction terms frozen at the previous iterate. The common device is to exploit the BEA freedom to choose correction fields that depend on the initial point, thereby making the modified vector field integrable.

Taken together, these formulations suggest that backward coherence is best understood as a recurring regularisation principle rather than a single algorithm. The backward object may be an inverse deformation, a reversed temporal pair, a successor hidden state, a retained phase, a terminal predictive distribution, or a previous iterate in a modified flow. What remains stable across the literature is the role of the regulariser: it penalises forward solutions that are incompatible with a specified backward relation, and thereby replaces or supplements conventional smoothness, isotropy, or prior-based constraints.

Source: https://www.emergentmind.com/topics/backward-coherence-regularisation