Single Image Super-Resolution: Methods & Challenges

• Single Image Super-Resolution is the process of recovering high-resolution images from a single low-resolution input, addressing an ill-posed inverse problem caused by blur, downsampling, and noise.
• The literature emphasizes efficient neural architectures, explicit priors such as patch similarity and edge structure, and accurate degradation modeling to enhance texture and detail.
• Recent methods integrate iterative optimization, attention mechanisms, and diffusion-based strategies to balance pixel fidelity, perceptual realism, and computational efficiency.

Searching arXiv for recent and foundational SISR papers to ground the article. arxiv.search({"query":"Single Image Super-Resolution survey deep learning arXiv 2025 attention linear selective super-resolution", "max_results": 10}) Single Image Super-Resolution (SISR) is the task of recovering a high-resolution (HR) image from a single low-resolution (LR) observation. In representative formulations, the LR image is modeled either as $y = (x \otimes k)\downarrow_s + n$ or as $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$, so blur, downsampling, and noise jointly remove high-frequency information and make the inverse map non-unique; accordingly, SISR is treated as an ill-posed inverse problem rather than a straightforward resizing operation (Yang et al., 2018, Liu et al., 2022). The modern literature organizes this problem around two coupled themes: the design of efficient neural architectures and the design of effective optimization objectives, with additional lines of work emphasizing internal priors, explicit degradation modeling, and generative uncertainty (Yang et al., 2018).

1. Problem formulation and sources of ambiguity

The ambiguity of SISR is usually stated as a one-to-many mapping from LR to HR: many distinct HR images can correspond to the same LR image after degradation. One practical consequence is that the missing information is concentrated in high-frequency structures—edges, textures, repetitive patterns, and local geometry—which are precisely the structures most affected by low-pass filtering and subsampling. The literature therefore treats SISR as a reconstruction problem in which fidelity to the observation and plausibility of the recovered prior must be balanced rather than maximized independently (Gunasekaran, 2023).

Several papers make the degradation mechanism explicit. The survey formulation $y = (x \otimes k)\downarrow_s + n$ incorporates blur kernel $k$, scale factor $s$, and noise $n$, while optimization-oriented models write the same idea as $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$ or $y = (k * x)\downarrow_s + n$ (Yang et al., 2018, Umer et al., 2019). The aliasing perspective sharpens this point: according to the Nyquist sampling theory, degradation reduces signal frequency and introduces aliasing, so fine textures and repetitive structures can be folded into misleading low-frequency patterns, especially in images with rich high-frequency content such as building edges and dense patterns (Liu et al., 2022). This is why SISR papers repeatedly distinguish numerical fidelity from correct texture recovery.

The application scope reported in the literature is correspondingly broad. SISR is described as relevant to surveillance, medical imaging, general image enhancement, aerial and satellite imaging, compressed image and video enhancement, and remote sensing imagery (Gunasekaran, 2023, Araújo et al., 2022, Lu et al., 1 Feb 2025). This breadth matters because different application domains privilege different operating points: some favor distortion metrics such as PSNR and SSIM, whereas others favor structural preservation, perceptual realism, or robustness to realistic blur and noise.

2. Explicit priors, self-similarity, and non-end-to-end formulations

Deep learning did not eliminate explicit priors from SISR. A persistent line of work models the HR image through patch recurrence, sparse coding, graph regularization, or edge structure. In the self-similarity tradition, WSD-SR formulates SISR as $\mathbf{y}=\mathbf{H}\mathbf{x}$, groups multiple similar patches extracted from the input image, performs collaborative Wiener filtering only in the 1D similarity domain, and alternates this prior with iterative back-projection. The method uses no external training data, outperforms the current non-CNN based methods on the tested datasets, reports gains over 1 dB against A+ on certain datasets, and at $\times 4$ performs similarly to very recent state-of-the-art deep convolutional network based approaches (Cruz et al., 2017).

Other explicit-prior formulations regularize local structure directly. The graph-based GLRDN term penalizes mismatches between neighboring-pixel differences in the prediction and the ground truth through

$$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$0

and in matrix form

$$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$1

so that the final objective becomes $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$2 (Zheng et al., 2022). This reframes “detail” as a relation between adjacent pixels rather than as an attribute of isolated intensities.

A related structural move is to treat SISR as inpainting. Edge-Informed SISR interprets missing HR samples as holes between known LR pixels, first predicts a high-resolution edge map, and then reconstructs the final image conditioned on both the incomplete HR canvas and the predicted edges (Nazeri et al., 2019). This decoupling of structure and texture is also visible in patchwise sparse coding. The quantum-annealing formulation of sparse-coding SISR keeps the classical paired-dictionary pipeline—solve sparse codes in the LR dictionary and transfer them to the HR dictionary—but maps the sparse coefficient search to QUBO and solves it on D-Wave hardware, obtaining PSNR comparable to classical annealing while remaining below SwinIR (Choong et al., 2023). Taken together, these works show that explicit priors did not disappear; they were redistributed across graph, patch, edge, and optimization formalisms.

3. Architectural evolution: from bicubic-input CNNs to dense, attentive, and linear-complexity models

The survey literature takes SRCNN as the baseline deep model and identifies three initial limitations: bicubic interpolation at the input, shallow depth, and only trivial built-in knowledge of the SISR process (Yang et al., 2018). Subsequent architectures respond by moving upsampling into the network, increasing depth, and improving feature reuse. FSRCNN uses a deconvolution layer to reconstruct HR from LR feature maps, ESPCN uses subpixel convolution, VDSR introduces very deep residual learning, DRCN uses recursive parameter sharing and multi-supervision, and EDSR removes batch normalization while increasing width and using residual scaling (Yang et al., 2018). Dense connectivity supplies another trajectory through SRDenseNet, MemNet, and RDN.

The Residual Dense Network line exemplifies this progression. The analyzed RDN architecture is organized into four main blocks—Shallow Feature Extraction, Residual Dense Block, Dense Feature Fusion, and Upsampling—and uses dense connections, local feature fusion, local residual learning, and a contiguous memory mechanism to fully exploit hierarchical features from LR images (Gunasekaran, 2023). In that study, removing local dense connections caused the largest performance drop, with values around 13–14 PSNR and 0.38–0.45 SSIM on some datasets, while the full model reported $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$3 on Set5, $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$4 on Set14, and $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$5 on Urban100 at $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$6 (Gunasekaran, 2023). This supports the now-standard view that feature propagation and reuse are central architectural primitives in SISR.

Attention mechanisms added another explicit bias: not all spatial regions require the same reconstruction behavior. The attention-based DenseRes framework predicts a mask $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$7 over interpolated LR input and forms the final output as

$$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$8

thereby enhancing texture-rich regions while suppressing unnecessary residuals in smooth regions (Liu et al., 2018). The reported gains include an average PSNR improvement of $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$9 dB from the attention module itself and average improvements of about $y = (x \otimes k)\downarrow_s + n$0 dB for SRResNet, $y = (x \otimes k)\downarrow_s + n$1 dB for VDSR, and $y = (x \otimes k)\downarrow_s + n$2 dB for DRCN when the module is used as a plug-in (Liu et al., 2018).

Recent efficient attention work pushes this idea further. SSCAN introduces query-aware window selection with fixed $y = (x \otimes k)\downarrow_s + n$3 windows, top-k routing, and FlashAttention-style efficient computation; it reports top-k $y = (x \otimes k)\downarrow_s + n$4 during training, top-k $y = (x \otimes k)\downarrow_s + n$5 at inference, and up to $y = (x \otimes k)\downarrow_s + n$6 dB PSNR improvement on urban datasets at $y = (x \otimes k)\downarrow_s + n$7 (Kim et al., 9 Apr 2025). OmniRWKVSR replaces quadratic self-attention with RWKV-based spatial and channel mixing, reports $y = (x \otimes k)\downarrow_s + n$8G FLOPs versus $y = (x \otimes k)\downarrow_s + n$9G for MambaIR and $k$0G for SwinIR, and at $k$1 reports an average improvement of $k$2 in PSNR and $k$3 in SSIM over MambaIR (Lu et al., 1 Feb 2025). Alternative representations have also been tested: SRCaps organizes SISR around Residual Dense Capsule Blocks and sub-pixel upsampling, achieving competitive but not leading results relative to RCAN and RDN, which indicates that CNN dominance in SISR has been challenged more by feature-routing design than by wholesale representational replacement (Araújo et al., 2022).

4. Optimization-inspired, iterative, and degradation-aware SISR

A parallel development treats SISR as an inverse problem to be unfolded rather than a direct LR-to-HR regression map. In the unified PPP/RED framework, the observation model is

$k$4

and SISR is solved by alternating a quadratic data-consistency step with a denoising step inside ADMM. In PPP, the denoiser is used as a black box; in RED, the denoiser defines an explicit regularizer $k$5 and the $k$6-update is solved through fixed-point iteration (Brifman et al., 2018). This produces a reconstruction loop that is modular, motion-estimation free in the broader framework, and explicitly tied to the forward model.

Other iterative systems learn the solver itself. HSRNet derives its network from half-quadratic splitting, alternates learned Solver LS and Denoiser modules in a low-resolution-specific feature space, and introduces a Hierarchical Exploration Block for local self-similarity plus Multi-Level Spatial Attention for high-frequency emphasis (Liu et al., 2022). With bicubic downsampling and $k$7 iterations, it reports representative BI $k$8 results of Set14 $k$9, B100 $s$0, Urban100 $s$1, and Manga109 $s$2 (Liu et al., 2022). ISRResCNet likewise embeds a degradation model into an unrolled Majorization-Minimization/proximal scheme with a shared Encoder–Resnet–Decoder block, uses $s$3 iterative steps, and reports $s$4K parameters without feedback and $s$5K with feedback, remaining much smaller than EDSR while approaching it numerically on several benchmarks (Umer et al., 2020).

ISRN makes the iterative interpretation even more explicit by mimicking HQS with learned Solver SR, Down-sampler, Solver LR, and Solver MLE modules, and by replacing batch normalization with Feature Normalization. Under BI $s$6, it reports ISRN $s$7 and ISRN+ $s$8 on Set5, and under more difficult blur-downscale and downscale-noise settings it reports gains over prior methods across datasets (Liu et al., 2020). These formulations are significant because they relax the assumption that SISR must be a single forward pass; instead, they make data fidelity, degradation, and prior enforcement first-class components.

Realistic degradation modeling sharpens this trend. SRWDNet explicitly abandons the assumption that LR inputs are only bicubic downsampled and instead uses

$s$9

with motion blur kernels of size $n$0 to $n$1 and noise levels $n$2 (Umer et al., 2019). Its learnable Wiener deconvolution layer, residual CNN denoiser, and sub-pixel upscaling are trained end-to-end. For a $n$3 image with a $n$4 motion blur kernel, $n$5, and $n$6 upscaling, the reported runtime is $n$7 s, compared with $n$8 s for TNRD and $n$9 s for IRCNN, while remaining somewhat slower than SRMD at $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$0 s (Umer et al., 2019). The broader implication is that bicubic-only benchmarking is methodologically narrower than the real degradations many applications require.

5. Objectives beyond pixel fidelity: perceptual, adversarial, probabilistic, and diffusion-based SISR

The optimization side of SISR begins with pixel-wise regression. The survey treats MSE as the baseline objective,

$$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$1

notes its connection to Gaussian likelihood, and emphasizes its tendency to produce overly smooth outputs (Yang et al., 2018). Many later papers therefore prefer $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$2, perceptual feature losses, adversarial losses, or mixed objectives. The DenseRes attention model explicitly chooses $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$3 because MSE-based training often maximizes PSNR but yields images that “look blurry and implausible” (Liu et al., 2018). The modified U-net with MixGE combines MSE with mean gradient error,

$$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$4

and reports best performance around $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$5, particularly improving edge clarity on SET14, BSD300, and ICDAR2003 (Lu et al., 2019).

Perceptual and adversarial SISR reinterpret the task as realism-constrained reconstruction. The structural-coherence GAN introduces a gradient branch in the generator, a U-Net discriminator that evaluates both global and local authenticity, LPIPS perceptual loss, feature matching, pixel $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$6, and gradient-space losses, and reports the best Perceptual Index on Set5, Set14, BSD100, Urban100, and General100 among SRGAN, SFTGAN, ESRGAN, and USRGAN (Li et al., 2021). Edge-Informed SISR adopts hinge GAN losses plus perceptual and style losses in a two-stage pipeline, first predicting edges and then reconstructing content, which improves SSIM and perceptual sharpness even when PSNR remains below EDSR (Nazeri et al., 2019). This is the canonical distortion–perception trade-off in concrete form.

Probabilistic alternatives address a different limitation: mode collapse or hallucinated artifacts. Conditional IMLE models the conditional distribution as $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$7, matches each target to the nearest generated sample in a perceptual feature space built from raw pixels and VGG-19 activations, and reports $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$8 PSNR and $$\mathbf{I}^{LR}=\mathbf{H}\mathbf{I}^{HR}+\mathbf{n}$$9 SSIM for $y = (k * x)\downarrow_s + n$0 super-resolution, compared with bicubic $y = (k * x)\downarrow_s + n$1 and SRGAN $y = (k * x)\downarrow_s + n$2 in its ImageNet setting (Li et al., 2018). The paper’s explicit critique is that SRGAN can introduce high-frequency noise, hallucinated colors, and shape distortions (Li et al., 2018).

Diffusion-based SISR extends this probabilistic turn. LDDPM combines a conditional encoder, CVAE-style latent conditioning, Glow-based normalized flow, and multimodal adversarial training within a conditional DDPM. On CelebA at $y = (k * x)\downarrow_s + n$3, it reports $y = (k * x)\downarrow_s + n$4 dB PSNR, $y = (k * x)\downarrow_s + n$5 SSIM, $y = (k * x)\downarrow_s + n$6 LPIPS, $y = (k * x)\downarrow_s + n$7 LR-PSNR, and $y = (k * x)\downarrow_s + n$8 DISTS, and reports about $y = (k * x)\downarrow_s + n$9 s sampling time versus about $\mathbf{y}=\mathbf{H}\mathbf{x}$0 s for SRDiff, $\mathbf{y}=\mathbf{H}\mathbf{x}$1 s for SR3, and $\mathbf{y}=\mathbf{H}\mathbf{x}$2 s for LDM (Wang et al., 2022). This suggests that recent generative SISR is not defined only by visual richness, but also by attempts to reduce the sampling burden historically associated with diffusion models.

6. Evaluation practice, persistent trade-offs, and current directions

SISR evaluation is highly standardized but not uniform. Training commonly uses DIV2K or DF2K variants, while testing frequently uses Set5, Set14, BSD100 or B100, Urban100, and Manga109; some works additionally use General100, CelebA, Places2, Celeb-HQ, ICDAR2003, or remote-sensing datasets (Liu et al., 2022, Lu et al., 1 Feb 2025). Distortion-oriented evaluation usually reports PSNR and SSIM, often on the Y channel of YCbCr with border cropping, though some works evaluate on RGB or supplement these measures with MS-SSIM, FLIP, LPIPS, DISTS, Perceptual Index, or human preference studies (Araújo et al., 2022, Li et al., 2021). The persistence of multiple metric families is itself substantive evidence that no single scalar captures the full SISR objective.

A recurrent misconception is that higher PSNR necessarily means better perceived reconstruction. Multiple papers reject this equivalence. The survey states that MSE-based methods often score well while looking smoother, and explicitly formulates a perception–distortion trade-off (Yang et al., 2018). The capsule-based study cautions that PSNR and SSIM can be misleading and supplements them with MS-SSIM and FLIP (Araújo et al., 2022). Edge-informed SISR reports cases where EDSR has the best PSNR but the edge-guided model has sharper contours and much higher SSIM in its tables (Nazeri et al., 2019). The controversy is therefore not whether PSNR matters, but whether it is sufficient.

Another persistent fault line concerns degradation assumptions. Bicubic downsampling remains the dominant benchmark protocol, yet several papers argue that real LR images are blurred, noisy, or otherwise mismatched to bicubic training (Umer et al., 2019). This has motivated degradation-aware networks, plug-and-play reconstructions, internal-example methods, and attention systems that attempt to retrieve only semantically relevant context (Brifman et al., 2018, Kim et al., 9 Apr 2025). A plausible implication is that SISR has bifurcated into at least two partially overlapping regimes: benchmark SISR under synthetic degradations, and deployment-oriented SISR under realistic or unknown degradations.

The current frontier combines efficiency, long-range dependency modeling, and richer priors. SSCAN replaces heuristic region selection with query-aware top-k window routing and linear token-to-token complexity in the selected-window stage (Kim et al., 9 Apr 2025). OmniRWKVSR explores linear-complexity RWKV scanning as an alternative to quadratic self-attention, with particular emphasis on remote sensing (Lu et al., 1 Feb 2025). Diffusion-based systems such as LDDPM seek detail diversity without the instability of GANs, while optimization-inspired systems continue to improve compactness and physical interpretability (Wang et al., 2022, Umer et al., 2020). The survey’s forward-looking challenges remain largely intact: lighter architectures, better methods for large-scale SISR and unknown corruption, better theoretical understanding, and better application-specific evaluation criteria (Yang et al., 2018).