---
title: 'UHDPromer: Neural Prompted Transformer for UHD Restoration'
url: https://www.emergentmind.com/topics/uhdpromer
type: topic
---

# UHDPromer: Neural Prompted Transformer for UHD Restoration

UHDPromer is a neural discrimination-prompted Transformer for ultra-high-definition image restoration and enhancement, introduced for UHD low-light image enhancement, image dehazing, and image deblurring [2603.00853]. Its central premise is that “there implicitly exist neural differences between high-resolution and low-resolution features,” and that explicitly measuring and injecting those differences into low-resolution Transformer computation can improve representation quality while keeping memory and FLOPs tractable for UHD inputs such as $3840\times2160$ [2603.00853]. The resulting design places the computationally heavy Transformer stages in a shuffle-down low-resolution branch, augments them with Neural Discrimination Priors (NDP), and recovers high-resolution output through a super-resolution-guided reconstruction pipeline.

## 1. Problem setting and design objective

UHD image restoration poses a coupled optimization problem in representation quality and computational scalability. The paper identifies the core difficulty as the fact that UHD inputs strain memory and computation, while existing CNN- and Transformer-based methods often depend on heavy parameterization, local windows, or compute-intensive correlation or similarity operations that are difficult to scale to 4K imagery without compromising fine detail [2603.00853].

UHDPromer addresses this setting by separating responsibilities across resolution regimes. Full-resolution processing is retained only in shallow high-resolution feature extraction and reconstruction modules, while global modeling is delegated to a low-resolution Transformer branch produced by shuffle down with factor $s=8$. This design suggests a specific efficiency strategy: preserve spatially precise cues in a lightweight high-resolution path, but perform long-range dependency modeling where the token count is reduced by downshuffling.

The model is evaluated on three UHD tasks: UHD-LL, UHD-Haze, and UHD-Blur, all at resolution $3840\times2160$. The paper considers two evaluation settings: training on general datasets and testing on UHD test sets, and training directly on UHD datasets followed by testing on the corresponding UHD test sets [2603.00853].

## 2. Overall architecture and data flow

The architecture has four named components: HRFR, NDPT, FeaSR, and SRG-Recon [2603.00853]. Given an input image $\mathbf{I}\in\mathbb{R}^{H\times W\times 3}$, a $3\times3$ convolution produces low-level embeddings $\mathbf{X}_0\in\mathbb{R}^{H\times W\times C}$. HRFR then encodes $\mathbf{X}_0$ with three ConvNeXt-v2 blocks to generate hierarchical multi-scale high-resolution features $\{\mathbf{X}_1,\mathbf{X}_2,\mathbf{X}_3\}$, all at the input UHD resolution [2603.00853].

The low-resolution branch is formed by shuffle-down features $\mathbf{X}_{down}\in\mathbb{R}^{(H/s)\times(W/s)\times C}$ with $s=8$. This branch is processed by Neural Discrimination-Prompted Transformers (NDPT), which are guided by NDP signals computed from the discrepancy between high-resolution and low-resolution features. The output of NDPT is passed to Feature Super-Resolution (FeaSR), which generates a super-resolution image $\hat{\mathbf{H}}_{SR}\in\mathbb{R}^{H\times W\times 3}$ [2603.00853].

Final restoration is performed by SR-Guided Reconstruction (SRG-Recon), which uses three ConvNeXt-v2 blocks, two $1\times1$ convolutions, and one $3\times3$ convolution to predict a residual image $\mathbf{S}\in\mathbb{R}^{H\times W\times 3}$. The restored output is produced by residual addition,
$$
\mathbf{\hat{H}}=\mathbf{I}+\mathbf{S}.
$$

The paper sets the principal architectural hyperparameters as $L=15$ NDPT blocks, $8$ attention heads per block, and channel dimension $C=16$ [2603.00853]. This parameterization is explicitly paired with low-resolution Transformer execution to reduce memory and FLOPs, while HRFR and SRG-Recon remain shallow modules at full resolution.

## 3. Neural Discrimination Priors and prompted Transformer blocks

The defining mechanism of UHDPromer is the Neural Discrimination Prior. NDP is introduced to quantify the differences between high-resolution features and low-resolution ones and to use those differences to guide low-resolution learning [2603.00853]. The paper defines
$$
\begin{equation}
\begin{split}
NDP_{i}(x) = 1/\sqrt{e^{\textrm{abs}\left| \mathcal{H}_{i}\left[\mathbf{X}_1, \mathbf{X}_2, \mathbf{X}_3\right]\left(x\right)- \mathbf{Y}_{i}\left(x\right) \right|}}
\end{split}
\end{equation}
$$
where $x$ is a pixel position, $i=1,2,\ldots,L$ indexes the Transformer block, $\mathcal{H}_i[\cdot,\cdot,\cdot]$ is composed of concatenation and a stride convolution with $s\times s$ kernel and $s\times s$ stride, and $\mathbf{Y}_i$ is the low-resolution input feature of the $i$-th Transformer block [2603.00853]. The paper further states that when $NDP_i(x)$ approaches $1$, the feature at position $x$ notably diverges from low-resolution features, indicating greater discriminative potential.

Each NDPT block takes the previous block output and the NDP signal, then applies NDP-guided attention followed by an NDP-guided feed-forward network with residual connections:
$$
\mathbf{X}' = NDPA\big(LN(\mathbf{X}^i), \mathbf{Y}_{\text{NDP}}\big) + \mathbf{X}^i,
$$
$$
\mathbf{X}^{i+1} = NDPN\big(LN(\mathbf{X}'), \mathbf{Y}_{\text{NDP}}\big) + \mathbf{X}'.
$$
Here $LN(\cdot)$ denotes layer normalization [2603.00853].

The Neural Discrimination-Prompted Attention (NDPA) re-formulates attention by incorporating NDP in a continuous attention computation scheme. The paper specifies that $Q$, $K$, and $V$ are generated from the low-resolution feature using $1\times1$ point-wise convolution $W_p(\cdot)$, $3\times3$ depth-wise convolution $W_d(\cdot)$, and split operator $\mathcal{S}(\cdot)$; similarly, $K_{\textrm{NDP}}$ and $V_{\textrm{NDP}}$ are generated from $\mathbf{Y}_{\textrm{NDP}}$ [2603.00853]. The attention operator is
$$
\mathcal{A}(\hat{Q}, \hat{K}, \hat{V}) = \hat{V}\cdot \textrm{Softmax}(\hat{K}\cdot \hat{Q}/\alpha),
$$
where $\alpha$ is a learnable scaling parameter. NDPA first computes cross-attention between NDP features and the low-resolution query, then re-computes attention using the residual low-resolution keys and values. The paper emphasizes that this continuous attention computation scheme is intended to “adequately utilize the NDP by long-range pixel dependency modeling.”

The Neural Discrimination-Prompted Network (NDPN) implements continuous gating guided by NDP inside the feed-forward network [2603.00853]. Its formulation is described through split streams $\mathbf{Z}_1,\mathbf{Z}_2$, concatenation operator $\mathcal{C}[\cdot,\cdot]$, GELU $\sigma(\cdot)$, and element-wise multiplication $\odot$:
$$
\mathbf{Z}_{1}, \mathbf{Z}_{2} = \mathcal{S}\left(W_{d}W_{p} (\mathbf{\hat{X}})\right),
$$
$$
\mathbf{X}_{\textrm{fusion}} = W_{p}\big(\mathcal{C}\left[\mathbf{Z}_{1}, \mathbf{Y}_{\textrm{NDP}}\right]\big),
$$
$$
\mathbf{X}_{\textrm{gate}^{1}} = \mathbf{X}_{\textrm{fusion}} \odot \sigma(\mathbf{Z}_{2}),
$$
$$
\mathbf{X}_{\textrm{NDPN}} = W_{p}\Big(W_{d}\left(\mathbf{X}_{\textrm{gate}^{1}}\right) \odot W_{p}\left(\mathbf{X}_{\textrm{fusion}}\right)\Big).
$$
This design selectively permits the passage of beneficial content through two gated interactions, both driven by the NDP-conditioned fusion term. The paper also states that, following Restormer, an expanding factor $\beta$ is used to enlarge intermediate channel dimensions, although its numerical value is not explicitly provided.

## 4. Super-resolution-guided reconstruction and optimization

A second distinctive element of UHDPromer is its super-resolution-guided reconstruction strategy [2603.00853]. After low-resolution NDPT processing, FeaSR super-resolves the output features and produces $\hat{\mathbf{H}}_{SR}$; SRG-Recon then combines the HRFR outputs with these super-resolved features to predict the final residual image. This means that the final restoration is not based solely on low-resolution Transformer outputs, but on a reconstruction path explicitly guided by a super-resolution branch.

The paper describes training with a two-branch loss using a spatial- and frequency-domain based loss function $\phi(\cdot,\cdot)$ used in prior work, weighted by $\alpha=0.5$ [2603.00853]. It also notes that there is a second term for the super-resolution branch and that omitting this term degrades performance. A plausible implication is that the SR branch is not merely auxiliary at the architectural level; it also contributes directly to optimization pressure during training.

The stated optimization settings are AdamW, initial learning rate $5\mathrm{e}{-4}$, cosine annealing to $1\mathrm{e}{-7}$, and training patch size $512\times512$ [2603.00853]. The paper specifies $s=8$ for shuffle down, with stride-convolution kernel and stride sizes also set to $8$. It does not provide batch size, number of epochs, mixed precision details, or hardware.

## 5. Empirical performance and computational profile

The paper reports results on three UHD datasets with the following statistics [2603.00853]:

| Dataset | Train / Test | Resolution |
|---|---:|---:|
| UHD-LL | 2,000 / 150 | 3840×2160 |
| UHD-Haze | 2,290 / 230 | 3840×2160 |
| UHD-Blur | 1,964 / 300 | 3840×2160 |

Evaluation uses PSNR and SSIM computed with IQA PyTorch Toolbox, with LPIPS also reported [2603.00853]. When some methods cannot process 4K directly, the paper follows UHDFour’s recommendation to resize to the largest manageable input rather than use tiling.

On low-light enhancement, UHDPromer achieves $21.714$ PSNR / $0.7807$ SSIM / $0.4176$ LPIPS in Setting 1 and $27.159$ / $0.9285$ / $0.2118$ in Setting 2 [2603.00853]. On dehazing, it records $16.927$ / $0.8666$ / $0.2034$ in Setting 1 and $22.725$ / $0.9432$ / $0.1134$ in Setting 2. On deblurring, it attains $27.582$ / $0.8263$ / $0.2554$ in Setting 1 and $29.527$ / $0.8584$ / $0.2163$ in Setting 2. The paper characterizes these outcomes as maintaining state-of-the-art performance while maximizing computational efficiency across the three UHD tasks.

The efficiency profile is central to the method’s positioning. UHDPromer has $0.7430$M parameters, $32.56$G FLOPs at $1024\times1024$, and runtime $0.12$ s under the paper’s runtime comparison setting [2603.00853]. The same table reports $0.16$ s for UHDformer, $0.45$ s for DehazeFormer, $1.27$ s for FFTformer, $1.86$ s for Restormer, and $3.40$ s for LMAR.

| Quantity | UHDPromer | Comparison noted in paper |
|---|---:|---|
| Parameters | 0.7430M | smaller than LMAR, UHD, UHDFour, FFTformer, DehazeFormer; larger than UHDformer |
| FLOPs at 1024×1024 | 32.56G | lower than all listed baselines, including UHDformer at 51.63G |
| Runtime at 1024×1024 | 0.12 s | fastest among listed models |

Qualitative comparisons in the paper’s figures show clearer content, more natural or vivid colors, reduced haze, and sharper structures [2603.00853]. These observations are qualitative rather than metric-based, but they are consistent with the reported LPIPS, PSNR, and SSIM gains.

## 6. Ablations, limitations, and relation to UHDformer

The ablation study isolates the contribution of NDP, its points of insertion, and the super-resolution guidance mechanism [2603.00853]. On UHD-LL, the full model achieves $27.159$ PSNR / $0.9285$ SSIM in the main branch. Removing NDP from both NDPA and NDPN reduces performance to $26.811$ / $0.9282$; removing NDP only from NDPA lowers it further to $26.183$ / $0.9252$; removing NDP only from NDPN gives $27.026$ / $0.9283$. Replacing NDP with direct features yields $26.138$ / $0.9272$, and using NDP only before the NDPT block gives $26.161$ / $0.9263$. These results indicate that both NDPA and NDPN benefit from NDP, and that continuous in-block prompting is more effective than one-time preconditioning.

The super-resolution-guided reconstruction ablation shows a similar pattern. Cascaded reconstruction without the SR branch gives $26.375$ PSNR / $0.9043$ SSIM, omitting the second term in the loss yields $27.125$ / $0.9051$, and the full SR-branch design reaches $27.159$ / $0.9285$ [2603.00853]. The large SSIM difference suggests that SR guidance primarily strengthens structural fidelity rather than only pixelwise reconstruction.

Sensitivity analyses further constrain the model’s operating regime. The paper states that performance improves with the number of channels up to $C=16$, beyond which gains saturate; it also notes that with only $2$ channels and $0.0168$M parameters, performance is comparable to UHDFour with $17.5$M parameters [2603.00853]. For the shuffle-down factor, $s=4$ increases receptive field but does not improve SSIM, while $s=8$ provides better SSIM than larger values.

The principal limitation is explicitly stated: UHDPromer is not good at handling general image size compared with general image restoration methods [2603.00853]. The paper attributes this weakness to an architecture tailored to operate mostly in low-resolution space and, more specifically, to $8\times$ downsampled features. This suggests that the model’s efficiency-oriented inductive bias is advantageous for UHD restoration but less suitable for general-resolution benchmarks requiring different receptive-field and scale trade-offs.

In relation to UHDformer, UHDPromer can be read as a successor that replaces correlation matching with neural discrimination prompting [2406.00629]. UHDformer builds an HR-to-LR bridge through Dual-path Correlation Matching Transformation and Adaptive Channel Modulator, whereas UHDPromer measures implicit HR-LR neural differences through NDP and injects them into both attention and feed-forward gating [2406.00629]. The paper presents this shift as a way to avoid heavy correlation computation while retaining strong UHD restoration quality and improving computational efficiency [2603.00853].

Source: https://www.emergentmind.com/topics/uhdpromer