---
title: 'RestoRect: Latent Flow Restoration'
url: https://www.emergentmind.com/topics/restorect
type: topic
---

# RestoRect: Latent Flow Restoration

RestoRect is a degraded image restoration framework that addresses the trade-off between high-performance but slow models and fast but lower-quality models by recasting feature-level knowledge distillation as a latent rectified-flow problem. It is described as a “Latent Rectified Flow Feature Distillation” method in which a student network learns to synthesize teacher-quality features through learnable trajectories in latent space, while the overall restoration system combines Retinex theory, learnable anisotropic diffusion constraints, trigonometric color space polarization, and a Feature Layer Extraction loss for cross-normalized transformer feature alignment with percentile-based outlier detection [2509.23480]. The reported scope is broad: 15 image restoration datasets, 4 tasks, and 8 metrics, with claimed improvements in training stability, convergence, inference speed, and restoration quality [2509.23480].

## 1. Problem setting and architectural objective

RestoRect is motivated by two limitations stated for existing restoration pipelines: high-performance models are too slow for practical use, and fast models produce poor results. Within this setting, knowledge distillation is treated as necessary but insufficient in its conventional form, because static feature matching cannot capture how modern transformer architectures dynamically generate features [2509.23480].

The central objective is for the student network to learn a small number of ODE integration steps that carry a Gaussian prior to the teacher’s high-quality features in a low-dimensional latent space. At layer \(l\), the teacher and student features are denoted by \(\mathbf{f}_{\mathrm{teach}^l}\in\mathbb{R}^{d_l}\) and \(\mathbf{f}_{\mathrm{stud}^l}\in\mathbb{R}^{d_l}\), where \(d_l\) is the dimensionality after spatial-channel flattening of transformer features [2509.23480].

This design places the method at the interface of restoration, feature distillation, and flow matching. A plausible implication is that the framework is intended not merely to compress a teacher into a smaller student, but to change the geometry of the distillation problem from static alignment to trajectory learning in latent feature space.

## 2. Latent rectified flow formulation

The latent rectified flow component samples
\[
\mathbf{z}\sim\mathcal{N}(0,I_{d_l})
\]
and defines the straight-line interpolation
\[
\mathbf{x}_t=(1-t)\mathbf{z}+t\,\mathbf{f}_{\mathrm{teach}^l},\qquad t\in[0,1].
\tag{1}
\]
Its velocity field is
\[
\mathbf{v}(\mathbf{x}_t,t)=\frac{d\mathbf{x}_t}{dt}=\mathbf{f}_{\mathrm{teach}^l}-\mathbf{z}.
\tag{2}
\]

A residual-MLP predictor \(\epsilon_\theta\) is trained with the flow-matching objective
\[
L_{\mathrm{vel}}
=
\mathbb{E}_{t,\mathbf{z},\mathbf{f}_{\mathrm{teach}^l}}
\Bigl\|
\epsilon_\theta\bigl(\mathbf{x}_t,t,\mathbf{c}\bigr)
-
\mathbf{v}(\mathbf{x}_t,t)
\Bigr\|_2^2,
\tag{3}
\]
where \(\mathbf{c}\) denotes conditioning from the student’s low-quality input [2509.23480].

The predictor \(\epsilon_\theta\) is specified as a small Residual MLP stacking 5 layers, each with LeakyReLU activations. At inference time, the ODE is solved by simple Euler steps,
\[
\mathbf{x}_{t+\Delta t}
=
\mathbf{x}_t+\Delta t\,\epsilon_\theta(\mathbf{x}_t,t,\mathbf{c}),
\qquad
\Delta t=\tfrac{1}{N_{\mathrm{steps}}},
\qquad
N_{\mathrm{steps}}\in\{1,\dots,5\}.
\tag{4}
\]
The reported empirical finding is that \(N_{\mathrm{steps}}=3\)–5 suffices to match teacher-quality features, in contrast to “10+ steps for typical DDIM diffusion” [2509.23480].

A common misunderstanding would be to view this module as a conventional denoising diffusion surrogate. The formulation in fact uses rectified flow to learn straight-line latent transport and explicitly emphasizes a small-step ODE solver rather than a longer DDIM-style sampling chain.

## 3. Feature distillation and the FLEX loss

RestoRect’s feature distillation mechanism is the Feature Layer Extraction loss, abbreviated FLEX. Its stated motivation is that standard \(\ell_2\) or KL losses assume teacher and student feature distributions align in scale and outlier behavior; FLEX instead combines cross-normalization using student statistics, percentile-based outlier masking, and resolution-aware weighting [2509.23480].

For each layer \(l\), student statistics over spatial locations \((h,w)\) are computed as
\[
\mu_{\mathrm{stud}^l}
=
\tfrac{1}{H_lW_l}\sum_{h,w}\mathbf{f}_{\mathrm{stud}^{l,h,w}},
\qquad
\sigma_{\mathrm{stud}^l}
=
\sqrt{
\tfrac{1}{H_lW_l}
\sum_{h,w}
\bigl(\mathbf{f}_{\mathrm{stud}^{l,h,w}}-\mu_{\mathrm{stud}^l}\bigr)^2+\epsilon
}.
\]
Both teacher and student features are then normalized using student statistics:
\[
\mathbf{f}_{\mathrm{teach}^{l,\mathrm{norm}}}
=
\frac{\mathbf{f}_{\mathrm{teach}^l}-\mu_{\mathrm{stud}^l}}{\sigma_{\mathrm{stud}^l}},
\qquad
\mathbf{f}_{\mathrm{stud}^{l,\mathrm{norm}}}
=
\frac{\mathbf{f}_{\mathrm{stud}^l}-\mu_{\mathrm{stud}^l}}{\sigma_{\mathrm{stud}^l}}.
\]

Outlier handling is percentile-based. For channel \(c\) of layer \(l\), if \(\tau_p^{l,c}\) is the \(p\)-th percentile of \(\bigl|\mathbf{f}_{\mathrm{stud}^{l,c,\mathrm{norm}}}\bigr|\), the reliable-location mask is
\[
M_{\mathrm{reliable}^{l,c,h,w}}
=
\mathbf{1}\Bigl\{
\bigl|\mathbf{f}_{\mathrm{stud}^{l,c,\mathrm{norm},h,w}}\bigr|
\le \tau_p^{l,c}
\Bigr\}.
\]

To prevent domination by high-resolution layers, the resolution-aware weight is
\[
w_l^{\mathrm{res}}
=
\max\Biggl(
\Bigl(\frac{H_{\mathrm{base}}W_{\mathrm{base}}}{H_lW_l}\Bigr)^{0.25},
0.1
\Biggr),
\qquad
(H_{\mathrm{base}},W_{\mathrm{base}})=(64,64).
\]

The complete FLEX loss is
\[
L_{\mathrm{FLEX}}
=
\sum_l
w_l^{\mathrm{layer}}
\,w_l^{\mathrm{res}}
\;
\frac{
\sum_{c,h,w}
M_{\mathrm{reliable}^{l,c,h,w}}
\,
\bigl\|
\mathbf{f}_{\mathrm{teach}^{l,c,\mathrm{norm},h,w}}
-
\mathbf{f}_{\mathrm{stud}^{l,c,\mathrm{norm},h,w}}
\bigr\|^2
}{
\sum_{c,h,w}M_{\mathrm{reliable}^{l,c,h,w}}+\epsilon
}.
\]
In practice, \(w_l^{\mathrm{layer}}\) is chosen so that finer layers receive moderate emphasis, and FLEX is applied only in student-training Stage 2 when the signal-to-noise ratio \(t/T<0.4\) [2509.23480].

This suggests that the distillation mechanism is designed to remain stable under architecture mismatch and feature-scale mismatch, especially for transformer-based restoration systems whose internal representations may not be directly comparable layer by layer.

## 4. Physics-based decomposition and auxiliary constraints

RestoRect augments the latent distillation pipeline with physics-informed priors derived from Retinex decomposition. Each image \(I\in\mathbb{R}^{H\times W\times 3}\) is factorized as
\[
I=R\odot L,
\qquad
R\in\mathbb{R}^{H\times W\times 3},
\qquad
L\in\mathbb{R}^{H\times W\times 1}.
\]
Two separate U-Net decomposers, \(\mathcal{D}_\ell\) and \(\mathcal{D}_h\), are used for low- and normal-light pre-conditioning [2509.23480].

The anisotropic diffusion component uses the Perona–Malik operator
\[
\mathcal{A}(X)
=
\nabla\!\cdot\!\bigl(c(|\nabla X|)\,\nabla X\bigr),
\qquad
c(|\nabla X|)
=
\exp\Bigl(-\frac{|\nabla X|^2}{s^2}\Bigr),
\]
where \(s\in[0.01,1.0]\) is learnable. The texture and luminance losses are
\[
L_{\mathrm{tex}}
=
\bigl\|
\mathcal{A}(I_{\mathrm{input}})
-
\mathcal{A}(R_{\mathrm{pred}})
\bigr\|_1,
\]
\[
L_{\mathrm{lum}}
=
\sum_{i,j}
\exp\bigl(-|\nabla L_{i,j}|\bigr)
\Bigl(
|\nabla_xL_{i,j}|^2+|\nabla_yL_{i,j}|^2
\Bigr).
\]

For color modeling, the method introduces a polarized HVI color space intended to avoid HSV’s red-hue discontinuity. It defines
\[
I_{\max}=\max\{R,G,B\},
\qquad
C_k
=
k\sin\!\Bigl(\tfrac{\pi}{2}I_{\max}\Bigr)+\epsilon,
\]
\[
H_{\mathrm{polar}}
=
C_k\,S\,\cos\!\Bigl(\tfrac{\pi}{3}H\Bigr),
\qquad
V_{\mathrm{polar}}
=
C_k\,S\,\sin\!\Bigl(\tfrac{\pi}{3}H\Bigr),
\qquad
I_{\mathrm{polar}}=I_{\max},
\]
with \(k\in[0.1,5.0]\) learnable. The corresponding color loss is
\[
L_{\mathrm{col}}
=
\bigl\|H_{\mathrm{polar}}^{\mathrm{pred}}-H_{\mathrm{polar}}^{\mathrm{gt}}\bigr\|_1
+
\bigl\|V_{\mathrm{polar}}^{\mathrm{pred}}-V_{\mathrm{polar}}^{\mathrm{gt}}\bigr\|_1
+
\bigl\|I_{\mathrm{polar}}^{\mathrm{pred}}-I_{\mathrm{polar}}^{\mathrm{gt}}\bigr\|_1.
\]

The teacher’s total loss is
\[
L_{\mathrm{teach}}
=
L_{\mathrm{rec}}
+
L_{\mathrm{vgg}}
+
L_{\mathrm{sty}}
+
\lambda_{\mathrm{tex}}L_{\mathrm{tex}}
+
\lambda_{\mathrm{col}}L_{\mathrm{col}}
+
\lambda_{\mathrm{lum}}L_{\mathrm{lum}},
\]
where
\[
L_{\mathrm{rec}}=\|I_{\mathrm{pred}}-I_{\mathrm{gt}}\|_1,
\]
\[
L_{\mathrm{vgg}}
=
\sum_l\lambda_l\|\phi_l(I_{\mathrm{pred}})-\phi_l(I_{\mathrm{gt}})\|_2^2,
\]
\[
L_{\mathrm{sty}}
=
\sum_l\|G_l(\phi_l(I_{\mathrm{pred}}))-G_l(\phi_l(I_{\mathrm{gt}}))\|_F^2,
\]
and
\[
\{\lambda_{\mathrm{tex}},\lambda_{\mathrm{col}},\lambda_{\mathrm{lum}}\}=\{0.05,0.05,0.2\}.
\]
The stated combination of Retinex decomposition, anisotropic diffusion, and polarized color modeling indicates that restoration is not treated as purely data-driven regression, but as a constrained reconstruction problem with explicit texture, illumination, and chromatic structure [2509.23480].

## 5. Training protocol, inference regime, and reported results

The experimental program spans 4 tasks and 15 datasets: low-light restoration on LOL-v1, LOL-v2-real/syn, and SID; underwater restoration on UIEB and LSUI; backlit restoration on BAID; fundus restoration on FIE; unpaired settings on DICM, LIME, MEF, NPE, and VV; and contrast enhancement on SICE-Mix and SICE-Grad. The 8 primary metrics are PSNR\u2191, SSIM\u2191, FID\u2193, BIQI\u2193, UIQM\u2191, BRISQUE\u2193, CLIPQ\u2191, and LPIPS\u2193 [2509.23480].

Training is divided into teacher and student stages. Stage 1 trains the teacher for 15–20 epochs per dataset with early stopping on validation, using Adam with \((\beta_1,\beta_2)=(0.9,0.999)\), learning rate \(2\times 10^{-4}\), and batch size 16. Stage 2 contains two student phases. In Phase 1, the restoration net is frozen, the velocity predictors only are trained, learning rates satisfy \(LR_{(\mathrm{rex})}=LR_{(\mathrm{img})}=2\times10^{-4}\), the duration is 10 epochs, and the loss is
\[
L_{\mathrm{phase1}}
=
L_{\mathrm{vel}}^{\mathrm{rex}}
+
L_{\mathrm{vel}}^{\mathrm{img}}
+
\lambda_{\mathrm{KD}}L_{\mathrm{KD}}
+
\lambda_{\mathrm{traj}}L_{\mathrm{traj}}.
\]
In Phase 2, end-to-end training uses learning rate \(10^{-4}\), 10 epochs, and
\[
L_{\mathrm{phase2}}
=
L_{\mathrm{rec}}
+
\lambda_{\mathrm{FLEX}}L_{\mathrm{FLEX}}
+
\lambda_{\mathrm{vel}}
\bigl(
L_{\mathrm{vel}}^{\mathrm{rex}}
+
L_{\mathrm{vel}}^{\mathrm{img}}
\bigr),
\]
with \(\lambda_{\mathrm{FLEX}}=0.15\) and \(\lambda_{\mathrm{vel}}=0.05\) [2509.23480].

The reported convergence profile is that the teacher typically converges in 500 K iterations, while students reach plateau in approximately 200 K iterations per phase. On an RTX 4090 / H100 GPU, inference time is approximately 156 ms per image for RestoRect with 3 rectified-flow steps and approximately 198 ms with 5 steps, compared with “DDIM (10+ steps): \(\approx 350\) ms+.” The paper states that 3–5 steps suffice to match or exceed 10-step DDIM outputs [2509.23480].

Quantitative results are given task by task. On LOL-v1, RestoRect is reported at PSNR 27.84, SSIM 0.945, FID 38.67, and BIQI 8.35, corresponding to “+2.49 dB vs RetiDiff,” “+0.079,” “–10.47,” and “–9.40,” respectively. On UIEB, the reported values are PSNR 25.88, SSIM 0.950, and UIQM 3.121, with gains “+1.76 dB,” “+0.040,” and “+0.033.” On BAID, the reported values are PSNR 27.67, SSIM 0.965, and FID 15.82, with “+4.48 dB” and “–11.65” for PSNR and FID. On FIE, BIQI is reported as 6.033 and CLIPQ as “0.503 vs 0.557.” For the unpaired datasets DICM through VV, BRISQUE improvements of 5–15 points are reported. For SICE-Mix and SICE-Grad, the values are PSNR 15.04/15.45, SSIM 0.667/0.715, with gains “+1.6/2.0 dB” [2509.23480].

Qualitative evidence is also summarized. The paper reports crisp edge preservation under extreme low-light, faithful color recovery in underwater scenes, and artifact-free backlit photos. In a blind user study with 20 images and 8 participants, RestoRect was preferred over RetiDiff and CIDNet in more than 60% of trials [2509.23480].

## 6. Scope, significance, and terminological ambiguity

Within image restoration, RestoRect denotes the latent rectified flow and feature distillation framework described above [2509.23480]. The name, however, is not unique across arXiv-adjacent usage. In harmonic analysis, “RestoRect” is explicitly used to refer to “the problem of understanding exactly how the Fourier-restriction norms above a rectangular patch of the paraboloid depend on the aspect ratios of the patch,” in the context of rectangular-paraboloid restriction, sharpness by Knapp- and Kakeya-type examples, perturbations, and degenerate hypersurfaces [1911.11600]. It should also be distinguished from “RESTORE,” which is a visual token reduction framework for multimodal LLM inference that rectifies positional and attentional distortions by attention calibration and distinctive anchor selection [2606.01711].

This terminological overlap can obscure the specificity of the restoration framework. In the restoration setting, the defining features are the latent rectified-flow ODE, FLEX-based transformer feature alignment, Retinex-based decomposition, learnable anisotropic diffusion constraints, and trigonometric color space polarization [2509.23480]. A plausible implication is that the term is best interpreted contextually: in computer vision restoration it names a student–teacher distillation architecture, whereas in harmonic analysis it names a restriction-theoretic problem over rectangles.

Source: https://www.emergentmind.com/topics/restorect