SURE: Uncertainty-Aware Residual Encoder

• The paper introduces SURE to refine local stroke details in diffusion-based text super-resolution by modeling uncertainty in the structural latent space.
• SURE separates global semantic priors from local geometric cues, employing uncertainty-aware spatial extraction and residual injection to enhance boundary precision.
• Experimental results show that full SURE delivers improved PSNR, SSIM, LPIPS, ACC, and NED compared to variants without uncertainty modeling.

Structure-guided Uncertainty-aware Residual Encoder (SURE) is a module for diffusion-based text image super-resolution that performs local, stroke-level refinement under a globally rectified text prior. In PRISM, SURE is the second major component, following Flow-Matching Prior Rectification (FMPR): FMPR stabilizes character identity and coarse glyph layout through a recovered prior $\hat{c}$, while SURE refines exact stroke shapes and boundaries in image and latent space (Xu et al., 13 May 2026). It is formulated as a frozen-backbone control branch that takes degraded RGB input $x_l$, latent input $z_l$, and recovered prior $\hat{c}$, predicts an uncertainty-aware structural latent $z_s$, and converts that latent into multi-scale residual feature maps injected into the skip connections of a frozen UNet (Xu et al., 13 May 2026). The design addresses a specific failure mode of text super-resolution under severe degradation: local structural cues extracted from low-quality inputs can be unreliable, and hard conditioning on such cues can lock in incorrect boundaries (Xu et al., 13 May 2026).

1. Origins and conceptual lineage

SURE arises in a setting where global semantic priors and local geometric evidence are intentionally separated. In PRISM, FMPR is responsible for “what character(s)” and coarse global layout, whereas SURE is responsible for “exact stroke shapes and boundaries” where local evidence is reliable (Xu et al., 13 May 2026). This decomposition is motivated by the observation that a global embedding does not uniquely determine local stroke topology, especially for intersections, closures, and dot placement, and that low-resolution edges may be missing, displaced, or contaminated by noise and compression (Xu et al., 13 May 2026).

The conceptual lineage of SURE is closely related to earlier uncertainty-aware structural modeling in segmentation. SSU-Net introduced a spatial and scale uncertainty-aware network for elongated physiological structure segmentation, using epistemic and aleatoric spatial uncertainty maps to guide feature refinement through the gated soft uncertainty-aware (GSUA) module, and using scale-dependent “uncertainty” for hierarchical fusion through the multi-scale uncertainty-aware (MSUA) module (Zhang et al., 2023). The synthesis accompanying SSU-Net explicitly noted that its components can inform the design of a “Structure-guided Uncertainty-aware Residual Encoder (SURE)” for elongated physiological structure segmentation, particularly through residual-style uncertainty-aware gating and multi-scale structure guidance (Zhang et al., 2023). In this sense, the SURE module in PRISM can be read as a concretization of that design direction in a diffusion restoration setting rather than in a medical segmentation backbone.

A potential source of confusion is nomenclature. The repository name of “Uncertainty-Supervised Interpretable and Robust Evidential Segmentation” uses SURE, and the technical overview describes a structure-guided, uncertainty-aware residual-encoder-like idea at the conceptual level; however, that manuscript itself is framed as an uncertainty supervision framework built on evidential deep learning rather than as the same residual encoder module defined in PRISM (Li et al., 21 Sep 2025). This suggests that “SURE” is not yet a uniquely standardized term across subfields, and its most precise meaning depends on the paper context.

2. Architectural composition in PRISM

In PRISM, SURE comprises two cascaded submodules,

$$\text{SURE} = \{\mathcal{F}_\eta, \mathcal{C}_\eta\},$$

where $\mathcal{F}_\eta$ is an uncertainty-aware spatial cue extractor operating in image space, and $\mathcal{C}_\eta$ is a structural residual encoder operating in latent space alongside the UNet (Xu et al., 13 May 2026). The module takes degraded image $x_l$, latent $z_l = \mathcal{E}_{\mathrm{vae}}(x_l)$, and recovered prior $x_l$0 from FMPR, and produces an uncertainty-aware structural latent $x_l$1, a projected structural control feature $x_l$2, an auxiliary boundary map $x_l$3, and a set of residual maps $x_l$4 (Xu et al., 13 May 2026).

The inputs and outputs are explicitly defined. $x_l$5 receives $x_l$6 and outputs mean and log-variance maps $x_l$7 of a latent structural cue distribution, a sampled structural latent $x_l$8, a projected control feature

$x_l$9

and a boundary map

$z_l$0

The residual encoder $z_l$1 then takes $z_l$2, $z_l$3, and $z_l$4 and outputs

$z_l$5

with $z_l$6 according to the UNet’s skip levels (Xu et al., 13 May 2026).

These residuals are injected into the frozen UNet $z_l$7 through skip-addition: $z_l$8 At the feature level, for the $z_l$9-th skip-resolution feature $\hat{c}$0 of the frozen backbone, PRISM applies

$\hat{c}$1

This residual injection pattern is described as ControlNet-style, with $\hat{c}$2 initialized from the diffusion backbone’s encoder and architecturally mirroring the encoder path of the original SD2.1 UNet (Xu et al., 13 May 2026).

The uncertainty-aware spatial cue extractor has a defined internal layout. It uses a stem consisting of a $\hat{c}$3 conv (3→32), GroupNorm, SiLU, and a residual block, followed by 5 downsampling blocks producing feature maps at $\hat{c}$4, $\hat{c}$5, $\hat{c}$6, $\hat{c}$7, and $\hat{c}$8 with channels $\hat{c}$9. A lightweight FPN uses the last three scales in a top-down manner to generate

$z_s$0

From this fused feature, two heads predict $z_s$1 and $z_s$2 (Xu et al., 13 May 2026).

3. Uncertainty-aware spatial cue extraction

The defining feature of SURE is that local structural information is modeled as a distribution rather than as a deterministic edge map. From the fused image-space feature $z_s$3, SURE predicts $z_s$4 and $z_s$5, and samples the structural latent by reparameterization: $z_s$6 This yields a stochastic latent representation of structure that is subsequently projected to control space and decoded into an auxiliary boundary map (Xu et al., 13 May 2026).

In PRISM, the uncertainty is explicitly aleatoric and data-dependent over local structural cues derived from $z_s$7. For clear strokes, $z_s$8 can predict low $z_s$9; for blurred, occluded, or missing strokes, $$\text{SURE} = \{\mathcal{F}_\eta, \mathcal{C}_\eta\},$$0 is high (Xu et al., 13 May 2026). The formulation is given as

$$\text{SURE} = \{\mathcal{F}_\eta, \mathcal{C}_\eta\},$$1

which places the uncertainty directly in the structural latent space rather than on output pixels or on a loss weight (Xu et al., 13 May 2026).

This uncertainty-aware cue extraction is designed to avoid overconfident hallucination. The paper states that naively feeding low-resolution edge maps or structural cues as hard conditioning can “lock in” wrong boundaries; SURE instead models uncertainty over those cues and injects only residual corrections on top of a frozen backbone already conditioned by the rectified prior $$\text{SURE} = \{\mathcal{F}_\eta, \mathcal{C}_\eta\},$$2 (Xu et al., 13 May 2026). A plausible implication is that the module behaves conservatively when local evidence is weak, because structural guidance is no longer treated as a deterministic observation.

The behavior is also linked to visualization. In Fig. 5 of PRISM, the predicted uncertainty map $$\text{SURE} = \{\mathcal{F}_\eta, \mathcal{C}_\eta\},$$3 highlights ambiguous regions, and the predicted boundary map $$\text{SURE} = \{\mathcal{F}_\eta, \mathcal{C}_\eta\},$$4 is sharp where $$\text{SURE} = \{\mathcal{F}_\eta, \mathcal{C}_\eta\},$$5 is low and blurred where $$\text{SURE} = \{\mathcal{F}_\eta, \mathcal{C}_\eta\},$$6 is high (Xu et al., 13 May 2026). This indicates that SURE’s uncertainty is intended not merely as a confidence diagnostic but as a mechanism that shapes the structure-control signal before residual injection.

4. Structure guidance and residual encoding

“Structure-guided” in SURE refers to explicit supervision of boundary and stroke structure. The target structure map is obtained from the clean high-resolution text: $$\text{SURE} = \{\mathcal{F}_\eta, \mathcal{C}_\eta\},$$7 where $$\text{SURE} = \{\mathcal{F}_\eta, \mathcal{C}_\eta\},$$8 is a Sobel operator applied channel-wise (Xu et al., 13 May 2026). The decoded boundary prediction $$\text{SURE} = \{\mathcal{F}_\eta, \mathcal{C}_\eta\},$$9 is trained to match this target through

$\mathcal{F}_\eta$0

This supervision forces the latent structural cue $\mathcal{F}_\eta$1 to encode clean stroke boundaries rather than arbitrary low-resolution texture (Xu et al., 13 May 2026).

The interaction among structure, uncertainty, and residual encoding is sequential. First, SURE extracts a structural distribution from $\mathcal{F}_\eta$2 and samples $\mathcal{F}_\eta$3. Second, it projects $\mathcal{F}_\eta$4 into a latent-resolution control feature $\mathcal{F}_\eta$5. Third, the structural residual encoder computes

$\mathcal{F}_\eta$6

Fourth, the frozen diffusion UNet consumes $\mathcal{F}_\eta$7 and is modulated by $\mathcal{F}_\eta$8 through skip-addition (Xu et al., 13 May 2026). The pipeline is therefore not a generic side branch: it is a boundary-aware residual control path whose influence is restricted to internal features of an otherwise frozen, prior-conditioned backbone.

The residual mechanism is feature-level, not image-level. In PRISM, “residual” refers to residual maps $\mathcal{F}_\eta$9 that are added to internal feature maps of the diffusion UNet: $\mathcal{C}_\eta$0 Because the backbone already receives global semantic guidance through $\mathcal{C}_\eta$1, the paper states that SURE’s residuals mostly affect spatial structure rather than global content (Xu et al., 13 May 2026). This separation is central: FMPR determines global identity and SURE refines local geometry without re-learning the backbone.

The architecture also reflects a broader structural-control pattern seen in uncertainty-guided segmentation. In SSU-Net, GSUA uses uncertainty maps to modulate features through a residual attention form,

$\mathcal{C}_\eta$2

and the associated synthesis suggests embedding such uncertainty-aware residual blocks inside encoder stages (Zhang et al., 2023). SURE in PRISM realizes a different instantiation—via stochastic structural latents and ControlNet-style skip residuals—but the shared idea is that uncertainty should govern where structure-sensitive residual refinement is permitted to act strongly.

5. Training objective and optimization regime

SURE is trained in stage 2 of PRISM, after FMPR and the one-step diffusion backbone have been trained. During this stage, FMPR and $\mathcal{C}_\eta$3 are frozen, and only SURE’s parameters $\mathcal{C}_\eta$4 are optimized (Xu et al., 13 May 2026). The stage-2 objective is

$\mathcal{C}_\eta$5

The hyperparameters are $\mathcal{C}_\eta$6, $\mathcal{C}_\eta$7, and $\mathcal{C}_\eta$8 (Xu et al., 13 May 2026).

The KL term,

$\mathcal{C}_\eta$9

regularizes the structural latent distribution and prevents degenerate behavior such as collapsing uncertainty to zero or inflating it arbitrarily (Xu et al., 13 May 2026). The paper describes this as ensuring that $x_l$0 and $x_l$1 stay bounded and that the distribution remains meaningful. This gives SURE a variational interpretation, with a prior $x_l$2 and a posterior approximation $x_l$3.

The training regime is explicitly staged. FMPR is trained first, and SURE is then optimized relative to a fixed prior path and a fixed backbone (Xu et al., 13 May 2026). PRISM uses Stable Diffusion 2.1-base UNet, fine-tuned via LoRA (rank 16). FMPR is trained for 2 × 100K iterations, SURE for 50K iterations with FMPR and UNet frozen, and the optimizer is AdamW with learning rate $x_l$4 and batch size 8 on 2× RTX A6000 (Xu et al., 13 May 2026). This curriculum is intended to prevent entanglement between global prior learning and local structural refinement.

6. Empirical behavior, ablations, and interpretive value

The PRISM ablation on RealCE-val, $x_l$5 super-resolution, isolates the contribution of SURE. The reported variants are FMPR only, residual branch only, w/o uncertainty (deterministic edges), and full SURE (Xu et al., 13 May 2026).

Variant	PSNR	SSIM
FMPR only	19.8257	0.5968
Residual branch only	19.8671	0.5947
w/o uncertainty (deterministic edges)	19.8768	0.5974
Full SURE (ours)	19.8919	0.6012

Variant	LPIPS ↓	ACC ↑
FMPR only	0.2059	64.61%
Residual branch only	0.2099	64.80%
w/o uncertainty (deterministic edges)	0.2056	64.80%
Full SURE (ours)	0.2043	65.19%

Variant	NED ↑
FMPR only	0.8461
Residual branch only	0.8468
w/o uncertainty (deterministic edges)	0.8477
Full SURE (ours)	0.8521

These results show that a plain residual branch yields only minor improvement, deterministic edge guidance improves more, and full uncertainty-aware structure modeling gives the best metrics, especially in text fidelity measures ACC and NED (Xu et al., 13 May 2026). The paper explicitly interprets this as evidence that uncertainty modeling is not cosmetic.

Qualitative examples reinforce the same pattern. Figure 1 compares low-resolution input, a base model, FMPR-only, and the full model. For the English letter “e”, full SURE restores a more accurate loop and horizontal stroke; for Chinese characters, it yields cleaner closures and stroke intersections with fewer broken or merged strokes than FMPR-only (Xu et al., 13 May 2026). Figure 2 shows that the low-quality-derived boundary map is noisy or misleading, whereas the predicted uncertainty map isolates ambiguous regions and the structural prediction $x_l$6 remains sharp only where uncertainty is low (Xu et al., 13 May 2026).

A broader interpretive significance can be drawn by comparison with SSU-Net. SSU-Net uses uncertainty maps to highlight ambiguous regions in corneal endothelium and retinal vessel segmentation and reports improvements over UNet, BayesianNet, and Lee’s uncertainty method, with the full combination of GSUA and MSUA performing best in ablations (Zhang et al., 2023). This suggests a cross-domain regularity: uncertainty becomes most useful when it is integrated into the feature transformation pathway rather than treated only as a post-hoc confidence measure.

7. Related formulations, misconceptions, and limitations

SURE in PRISM should be distinguished from several neighboring ideas. It is not a standard residual module that deterministically propagates low-quality features; the paper argues that such a branch is prone to propagating artifacts from $x_l$7 (Xu et al., 13 May 2026). It is also not equivalent to generic uncertainty-weighted losses in single-image super-resolution, because the uncertainty is placed in the structural latent space and tied to explicit structure decoding $x_l$8 and residual feature injection (Xu et al., 13 May 2026). The mechanism therefore couples uncertainty with boundary supervision and skip-level control.

It should also be distinguished from segmentation-oriented uncertainty supervision frameworks that use the SURE acronym in code repositories. “Uncertainty-Supervised Interpretable and Robust Evidential Segmentation” builds an evidential deep learning pipeline with gradient-based and noise-based supervision losses to make pixel-wise uncertainty interpretable and robust, using a U-Net backbone and single-pass Dirichlet uncertainty estimation (Li et al., 21 Sep 2025). That framework supervises uncertainty maps directly through structural principles and synthetic noise, whereas PRISM’s SURE models a stochastic structural latent and injects residual features into a frozen diffusion UNet (Li et al., 21 Sep 2025). The shared emphasis is structure-guided uncertainty, but the architectures and training targets differ substantially.

Several limitations are explicit or directly implied in PRISM. The uncertainty modeled in SURE is aleatoric over local structural cues from the degraded input, not a joint epistemic–aleatoric treatment (Xu et al., 13 May 2026). The method depends on a two-stage curriculum in which FMPR and the backbone are frozen during SURE training, which simplifies disentanglement but may constrain mutual adaptation (Xu et al., 13 May 2026). The structural target is based on Sobel edges of the high-resolution image, so the notion of “structure” is boundary-centric (Xu et al., 13 May 2026). A plausible implication is that extensions to other restoration domains would require redefining the structural target and perhaps the uncertainty-bearing latent itself.

Within the longer arc of uncertainty-aware structure modeling, SSU-Net highlights additional open directions. Its synthesis points to explicit residual uncertainty-aware encoder blocks, encoder-level uncertainty maps, and tighter feedback between uncertainty estimation and feature extraction as possible extensions of a SURE-type design (Zhang et al., 2023). In parallel, the evidential segmentation work suggests that uncertainty supervision can be guided by human-interpretable structural principles rather than left unsupervised (Li et al., 21 Sep 2025). Taken together, these works position SURE not as a single fixed block, but as a design pattern: uncertainty is treated as a structural control signal, residual pathways are used to localize its influence, and feature refinement is strongest where local evidence is reliable and weakest where ambiguity persists.