---
title: 'Restormer: Efficient Image Restoration Transformer'
url: https://www.emergentmind.com/topics/restormer
type: topic
---

# Restormer: Efficient Image Restoration Transformer

Restormer, short for **Restoration Transformer**, is an efficient Transformer architecture for **high-resolution image restoration** that was introduced to retain the long-range modeling advantages of Transformers while avoiding the quadratic spatial cost of vanilla self-attention [2111.09881]. It is formulated as a **U-Net-like hierarchical encoder-decoder Transformer** for image-to-image restoration, and its defining architectural move is to replace token-wise spatial attention with **channel-wise transposed attention**, combined with a convolution-enhanced gated feed-forward module. In the original study, the model was evaluated on **deraining, single-image motion deblurring, defocus deblurring, Gaussian denoising, and real image denoising**, and reported state-of-the-art or near-state-of-the-art results across **16 benchmarks** [2111.09881].

## 1. Problem setting and design objective

Restormer was proposed against a specific background in low-level vision. CNN-based restorers were already strong at learning image priors, but their convolutional receptive fields remained limited and their filters were static at inference. Standard Transformers, by contrast, could model long-range dependencies and adapt to content, but their self-attention cost grew quadratically with spatial resolution, making them impractical for many restoration problems involving large images [2111.09881].

The central design objective was therefore not merely to import a vision Transformer into restoration, but to construct a model that preserves **global interaction** while remaining feasible on large restoration inputs. Earlier restoration Transformers often reduced cost by restricting attention to windows or independent patches, which weakens global dependency modeling. Restormer instead keeps attention global in effect, but restructures it so that the complexity is **linear with respect to spatial resolution** [2111.09881].

This design objective also explains why Restormer rapidly became a reference architecture in subsequent work. Later papers repeatedly use it either as a baseline, a teacher model, or a reconstruction backbone, which suggests that the architecture established a durable operating point between restoration quality, scalability, and architectural simplicity [2507.17892].

## 2. Core architecture

Restormer uses a **4-level symmetric encoder-decoder**. Given a degraded image
$$
\mathbf{I}\in \mathbb{R}^{H\times W\times 3},
$$
a convolution first produces shallow features, which are then processed by a hierarchical encoder-decoder with skip connections and a high-resolution refinement stage. The output is predicted residually as
$$
\hat{\mathbf{I}} = \mathbf{I} + \mathbf{R}.
$$
In the default configuration, the number of blocks across levels is **\([4,6,6,8]\)**, the number of channels is **\([48,96,192,384]\)**, the MDTA heads are **\([1,2,4,8]\)**, and the refinement stage has **4 blocks** [2111.09881].

The first defining block is **multi-Dconv head transposed attention (MDTA)**. For a layer-normalized input \(Y\), Restormer forms query, key, and value features by combining \(1\times1\) point-wise convolutions with \(3\times3\) depth-wise convolutions:
$$
Q = W_d^Q W_p^Q Y,\quad K = W_d^K W_p^K Y,\quad V = W_d^V W_p^V Y.
$$
Attention is then computed in channel space rather than over all spatial positions:
$$
\hat{X} = W_p\,\textrm{Attention}(\hat{Q}, \hat{K}, \hat{V}) + X,
$$
with
$$
\textrm{Attention}(\hat{Q}, \hat{K}, \hat{V}) = \hat{V}\cdot \textrm{Softmax}\left(\hat{K}\cdot\hat{Q}/\alpha\right).
$$
Here \(\alpha\) is a learnable scaling parameter. This transposed construction avoids a \((HW)\times(HW)\) attention matrix and is the primary reason the model scales to large images [2111.09881].

The second defining block is the **gated-Dconv feed-forward network (GDFN)**. Instead of a standard Transformer feed-forward layer, Restormer uses a gated, depth-wise-convolutional formulation:
$$
\hat{X} = W^0_p\,\textrm{Gating}(X) + X,
$$
where
$$
\textrm{Gating}(X) = \phi\!\left(W^1_d W^1_p(\textrm{LN}(X))\right) \odot W^2_d W^2_p(\textrm{LN}(X)).
$$
Here \(\phi\) is GELU and \(\odot\) is element-wise multiplication. This introduces both local spatial inductive bias and content-adaptive gating into the feed-forward path [2111.09881].

Architecturally, Restormer is therefore neither a plain ViT nor a CNN with attention add-ons. It is a restoration-specific Transformer in which both the attention block and the feed-forward block are re-engineered around the demands of dense image recovery.

## 3. Training protocol and empirical profile

The original training setup uses **AdamW** with \(\beta_1=0.9\), \(\beta_2=0.999\), weight decay \(10^{-4}\), **\(L_1\) loss**, **300K** iterations, an initial learning rate of \(3\times10^{-4}\), cosine annealing down to \(10^{-6}\), and horizontal and vertical flips as augmentation [2111.09881]. For denoising, a **bias-free** version of Restormer is used. A notable training strategy is **progressive learning**, in which patch size increases as
$$
128^2 \to 160^2 \to 192^2 \to 256^2 \to 320^2 \to 384^2,
$$
while batch size is reduced accordingly; the reported effect is a measurable PSNR gain over fixed-size patch training [2111.09881].

Across restoration tasks, the original paper reports a broad empirical profile. On **image deraining**, Restormer improves the average PSNR over SPAIR by **1.05 dB**, with gains as high as **2.06 dB** on Rain100L. On **single-image motion deblurring**, averaged across GoPro, HIDE, RealBlur-R, and RealBlur-J, it improves over MPRNet by **0.26 dB** and over MIMO-UNet+ by **0.47 dB**; relative to MPRNet, it uses **81% fewer FLOPs**, and relative to IPT, it has **4.4× fewer parameters** and is **29× faster** while delivering better quality [2111.09881].

On **defocus deblurring**, Restormer is reported to improve the combined DPDD result by about **0.6 dB** over IFAN and by **1.01 dB** over Uformer. On **Gaussian denoising**, it achieves the best PSNR on all listed grayscale datasets and noise levels, and on **Urban100 at \(\sigma=50\)** it improves by **0.37 dB** over DRUNet and **0.31 dB** over SwinIR in grayscale denoising, while in color denoising the gain is about **0.41 dB** over DRUNet and **0.2 dB** over SwinIR. On **real image denoising**, it is described as the only method reported to surpass **40 dB PSNR** on both **SIDD** and **DND** [2111.09881].

A plausible implication is that Restormer’s importance lies not in one isolated benchmark result, but in the fact that the same architectural template remained competitive across tasks with different degradation statistics, ranging from rain streak removal to real sensor noise.

## 4. Architectural descendants and comparative developments

A substantial later literature treats Restormer as a strong baseline but also interrogates its specific inductive biases. **DiNAT-IR** is explicitly framed as a response to a weakness in Restormer-style restoration Transformers: while Restormer’s **channel-wise self-attention** is efficient and effective, it may overlook **fine local artifacts** and pixel-level structure. DiNAT-IR therefore keeps the same **four-stage U-Net backbone as Restormer**, replaces the attention primitive with **Dilated Neighborhood Attention (DiNA)**, and adds a **channel-aware module (CAM)** to reintroduce global context. On GoPro and HIDE, it reports **33.80 dB / 0.967 SSIM** and **31.57 dB / 0.945 SSIM**, compared with Restormer’s **32.92 dB / 0.961** and **31.22 dB / 0.942**; on SIDD and Rain100L, however, it is slightly below Restormer in PSNR while remaining close in SSIM [2507.17892].

Other work has pursued more heterogeneous attention designs. **DART** describes Restormer as strong but limited by **channel-complexity growth** and a **single-attention bias**, and proposes a multi-attention design combining LongIR attention, local and global attention, plus feature- and positional-dimension attention. Its reported **DART-B** model has **4.5M parameters**, versus **26.13M** for Restormer, while slightly exceeding Restormer on SIDD (**40.10 dB / 0.961** versus **40.02 dB / 0.960**) and on DND (**40.06 dB / 0.957** versus **40.03 dB / 0.956**) [2404.04617].

A separate line of work argues that Restormer’s architecture is not always the primary bottleneck. **ConStyle v2** keeps the Restormer backbone unchanged and adds a frozen prompt generator that turns it into an **all-in-one image restoration** model. In the reported 27-benchmark average, **Original Restormer** is listed at **6.40 / 0.0506**, while **ConStyle v2 Restormer** reaches **27.60 / 0.8352** [2406.18242]. This suggests that prompt conditioning and training protocol can sometimes dominate architecture redesign.

There are also direct efficiency-oriented modifications. One study on high-resolution motion deblurring reports an **18.4% reduction in parameter count**, about **30% fewer layers**, lower memory footprint, richer augmentation, and a new frequency-domain loss while preserving competitive quality on RealBlur-R, RealBlur-J, and UHDM [2501.18403]. A later edge-efficiency study uses Restormer as the transformer teacher and baseline, then distills its blocks into Mamba-style state-space surrogates; on a **Snapdragon 8 Elite CPU**, the Restormer baseline requires **10119.52 ms** for inference, whereas ENS-discovered hybrids reduce this to **2973 ms** for deblurring, **5816 ms** for deraining, and **8666 ms** for denoising while maintaining competitive restoration quality [2605.02794].

Taken together, these developments do not displace Restormer so much as clarify its position: it is a stable architectural reference point against which locality-aware, data-centric, prompt-guided, and edge-efficient alternatives are defined.

## 5. Domain-specific integrations

Restormer has also been embedded into domain-specific systems in which the degradation model is not a generic blur or noise process but part of a larger sensing pipeline.

| Domain | Restormer’s role | Reported result |
|---|---|---|
| Hyperbolic metalens imaging | Learned post-processing module for severe off-axis aberrations | Wide FOV imaging over **54°**; about **50 ms per image** [2507.21562] |
| Atmospheric turbulence mitigation | Core reconstruction backbone with a NIMA residual branch | **98.53% average accuracy** on text-pattern reconstruction [2210.16847] |
| QR code motion deblurring | Baseline transformed into **EG-Restormer** and routed within **ADNet** | **90.00% DR** at **0.737 s** for ADNet [2510.12098] |
| Universal MRI reconstruction | Learned proximal/reconstruction operator in each SDUM cascade | Backbone ablation: **32.090 PSNR / 0.881 SSIM** for Restormer [2512.17137] |

In the hyperbolic metalens study, Restormer is used as a **direct image-to-image restoration model** for a camera whose limitation is not on-axis focal quality but **strong off-axis aberrations**. The network is trained in a **reference-free** manner using simulated blurred images generated with the **eigenPSF** method, and the reported system enables “high-quality imaging over a wide FOV of **54°**” on experimentally captured scenes under diverse lighting conditions [2507.21562].

In the UG\(^2\)+ atmospheric turbulence mitigation report, Restormer is the main reconstruction backbone and is combined with a **NIMA-based image quality assessment module** through residual fusion. The model processes **20 randomly selected frames** from a 100-frame turbulence sequence, uses **\(128\times128\)** sliding windows with stride **100**, and the final system achieves **1st place on the final leaderboard** with **98.53% average accuracy** on the reconstruction result of the text patterns [2210.16847].

For QR code motion deblurring, the emphasis shifts from perceptual quality to **successful decoding**. The paper introduces **EG-Restormer**, which augments Restormer with an **Edge-Guided Attention Block**, and then combines it with a lightweight path in **ADNet**. With GoPro + QRData fine-tuning, **Restormer** reports **88.67% DR**, while **EG-Restormer** reaches **90.00% DR**; **ADNet** preserves **90.00% DR** while reducing average inference time from **0.910 s** to **0.737 s** [2510.12098].

In MRI, Restormer functions not as a standalone restorer but as the image-domain reconstructor inside an unrolled optimization framework. In **SDUM**, every cascade alternates coil-sensitivity estimation, sampling-aware weighted data consistency, and a **Restormer-based** reconstruction step. Under a controlled backbone ablation at \(T=6\), the Restormer option reports **32.090 PSNR / 0.881 SSIM**, ahead of U-Net, DiT, and TAU under the same setup [2512.17137].

## 6. Limitations, robustness, and continuing debates

A common misconception is that Restormer’s strong benchmark accuracy makes it uniformly reliable at inference. Follow-up work shows that this is not the case. One line of analysis revisits **global information aggregation** and argues that restoration models trained on cropped patches but tested on full-resolution images suffer a train-test inconsistency. Applied to Restormer, the proposed **Test-time Local Converter (TLC)** yields **Restormer-Local**, improving GoPro from **32.92 dB** to **33.57 dB** and HIDE from **31.22 dB** to **31.49 dB** without retraining or fine-tuning [2112.04491].

A stronger critique concerns adversarial robustness. On GoPro deblurring, clean-image performance for Restormer is reported as **31.99 PSNR / 0.9635 SSIM**, but under **CosPGD, 20 iters** it drops to **7.59 / 0.1548**. The same study reports that **FGSM adversarial training** raises Restormer’s attacked performance to **21.58 / 0.7317**, and argues that Restormer has better robust generalization potential than simplified descendants such as Baseline and NAFNet, even though it remains vulnerable without explicit hardening [2307.13856].

Stability in the Lipschitz sense is another limitation. A later denoising study uses Restormer as a strong **unconstrained** baseline and argues that, under mild perturbations such as JPEG compression, Restormer’s output can deviate much more than that of a provably contractive denoiser. In standalone denoising, Restormer is usually best or near-best among unconstrained models, but in **PnP restoration** the proposed contractive denoiser often surpasses it substantially, which indicates that raw restoration accuracy and convergence-certified inverse-problem behavior are distinct criteria [2603.26168].

There is also a practical misconception that architectural novelty is always the dominant source of improvement. A data-centric denoising study starts from the **public Restormer \(\sigma=50\) pretrained model**, keeps the backbone unchanged, expands the training corpus to **143,679 images** from seven public sources, and applies **\(\times 8\)** geometric self-ensemble. Under a unified 100-image validation protocol, the final method improves over the public Restormer baseline by up to **+3.3662 dB PSNR**, while the retained **TLC-style local inference wrapper** is reported to have **negligible quantitative effect** in that setting [2604.11468].

These debates locate Restormer’s historical significance more precisely. It is an influential restoration Transformer and a strong empirical baseline, but later work shows that its efficiency is context-dependent, its global aggregation can be inference-sensitive, its clean-image accuracy does not imply robustness, and substantial gains can also come from data, prompting, or physics-based integration rather than from replacing MDTA or GDFN outright.

Source: https://www.emergentmind.com/topics/restormer