3D Super-Resolution (3DSR) Overview

• Three-dimensional super-resolution (3DSR) is an approach to recover detailed high-resolution 3D structures from low-resolution measurements by retrieving high-frequency information.
• It employs a range of techniques including explicit geometric manipulation, frequency-domain regularization, and deep learning architectures on both volumetric and hybrid 2D-3D representations.
• Applications span from single-view depth sensing and biomedical imaging to video synthesis and physical simulations, achieving significant improvements in metrics like RMSE, PSNR, and SSIM.

Three-dimensional super-resolution (3DSR) refers to the algorithmic enhancement of spatial resolution in three-dimensional data—by recovering or reconstructing high-frequency structural information from low-resolution 3D measurements. 3DSR encompasses a spectrum of modalities, including single-view depth sensing, multi-view scene reconstruction, biomedical volumetric imaging, microscopy, and video. Techniques range from explicit geometric manipulation and frequency-domain regularization to deep learning architectures operating on volumetric, multi-view, or hybrid 2D-3D representations.

1. Problem Definition and Representational Frameworks

The 3DSR task is broadly characterized as the recovery of a high-resolution (HR) 3D signal $x^{HR}$ from low-resolution (LR) observations $x^{LR}$, where the degradation is modeled as

$x^{LR} = D \, H(x^{HR}) + n$

with $D$ a downsampling operator, $H$ a blur or point-spread function, and $n$ noise. Key domain instantiations include:

• Depth Super-Resolution (DSR): Upsampling range, LiDAR, or depth-camera output to finer spatial grids—often single-view and lacking high-res RGB guidance (Mas et al., 11 Nov 2025).
• Volumetric Biomedical Imaging: Enhancing isotropy and fine-grained detail in modalities such as MRI, CT, or SRH microscopy (Pérez-Bueno et al., 2024, Jiang et al., 2024).
• Video and Light-Field Data: Exploiting spatiotemporal or angular correlations to improve the reconstruction of dynamic or multi-view data streams (Kim et al., 2018, Tran et al., 2022).
• 3D Scene or Surface Reconstruction: Predicting HR representations (e.g., 3D Gaussians, meshes, point clouds) from sparse or LR projections (Chen et al., 6 Aug 2025, Feng et al., 27 Feb 2026).

Representational choices directly influence the algorithmic approach, with recent trends emphasizing 2D/3D hybridization (such as PNCC, EPI volumes), explicit Gaussian-based scene models, and orientation-agnostic strategies.

2. Algorithmic Principles and 2D–3D Bridging

A major innovation in modern 3DSR is the reframing of volumetric tasks as structured 2D (or 2.5D) image SR via engineered representations. The "2Dto3D-SR" pipeline exemplifies this: surface geometry is encoded as a Projected Normalized Coordinate Code (PNCC),

$$\text{PNCC}(u,v) = \Bigl( \frac{(u-c_x)\,d(u,v)}{f_x\,s},\,\frac{(v-c_y)\,d(u,v)}{f_y\,s},\,\frac{d(u,v)}{s} \Bigr)$$

allowing standard 2D super-resolution networks to operate directly on geometry encoded as regular images. The resulting HR PNCC is then unnormalized and reprojected to yield HR depth or surface coordinates (Mas et al., 11 Nov 2025).

Analogously, 3D light-field imaging may be structured as 3D EPI (epipolar image) volumes, enabling multi-stage SR pipelines that apply both 2D/3D CNNs and tailored attention modules to jointly refine spatial and angular content (Tran et al., 2022). This general approach leverages the maturation of 2D SR architectures—transformers, pixel-shuffle upsampling, and attention—and adapts them to 3D without incurring the full computational and data complexity of native volumetric models.

3. Deep Network Architectures for 3DSR

Diverse network designs have been proposed for 3DSR, tailored to the specifics of the input/output representation:

• 3D Volumetric CNNs: Standard in medical and scientific imaging, these networks process full or patch-based 3D tensors (e.g., 3DSRnet, TV-based Deep 3D SR) and often incorporate residual and skip connections for improved convergence and capacity (Kim et al., 2018, Pérez-Bueno et al., 2024).
• Hybrid 2D/3D Pipelines: PNCC-based frameworks process structured 2D geometric encodings using high-accuracy (Swin Transformer) or high-efficiency (Vision Mamba) backbones, paired with pixel-wise Charbonnier loss (Mas et al., 11 Nov 2025).
• GAN Frameworks: In volumetric SR for MRI, three-player GANs utilize a generator (3D RRDB), a discriminator, and a dynamically updated feature extractor to balance pixel, perceptual, and adversarial losses—even in extremely data-limited regimes (Wang et al., 2023).
• Diffusion Models and 2D Supervision: MSDSR demonstrates that purely 2D diffusion models (e.g., UNet-based DDPMs), trained on high-res 2D slices alone, can be composed to produce isotropic high-res 3D volumes by enforcing orientation-invariant data distributions (Jiang et al., 2024).
• Feed-forward Mapping to 3D Scene Representations: SR3R introduces direct mapping from multi-view LR images to HR 3D Gaussian Splats, injecting local offset learning and feature refinement to decouple HR generation from rigid dependence on 2D SR priors (Feng et al., 27 Feb 2026).

Super-resolution networks for 3DSR increasingly exploit cross-scale (e.g., pyramidal), cross-attention, and modality-specific features (such as spatial/temporal/normal cues) to enhance robustness and generalization.

4. Regularization, Optimization, and Training Protocols

Regularization is pivotal for mitigating ill-posedness in 3DSR, especially where LR-HR ground truth pairs are limited or absent:

• Total Variation (TV) and Analytical Regularizations: Analytical regularization is effective both in optimization-based (e.g., 3D-FSR closed-form Tikhonov, ADMM solvers) and deep-learning self-supervised setups (Tuador et al., 2020, Pérez-Bueno et al., 2024). TV penalizes local gradient magnitudes and avoids over-smoothing of anatomical or structural edges.
• Self-Supervised Consistency: Self-supervised SR enforces model consistency via a degradation loop—forcing the HR estimate, when downsampled, to reproduce the observed LR input. This eliminates dependence on external HR labels and allows direct deployment on arbitrary datasets (Pérez-Bueno et al., 2024).
• Adversarial and Perceptual Losses: Adversarial (e.g., relativistic RaGAN), pixel-level ($\ell_1$ or Charbonnier), and perceptual (feature-distance via a dedicated network) losses are integrated to balance fidelity and realism, particularly for data domains with complex textures or fine vessel morphology (Wang et al., 2023, Yao et al., 6 Apr 2026).
• Physics-guided or Topological Losses: In scientific domains, prior physical knowledge and topological constraints (e.g., divergence minimization in fluid flow, vessel graph connectivity in microvasculature) are encoded as architectural elements or penalty terms in the training objective (Yasuda et al., 2023, Yao et al., 6 Apr 2026).

Training protocols vary, but patch-based mini-batch sampling is widespread for GPU tractability. Optimization typically employs Adam/AdamW, with learning rate schedules adapted to network scale and dataset size.

5. Quantitative Performance and Benchmarking

Method performance is characterized by both domain-general and domain-specific metrics:

• RMSE/MAE/PSNR/SSIM: For depth, volumetric, and light-field reconstructions, these classic error and similarity measures dominate.
• Domain-Driven Metrics: Functional map correlation (fMRI), Dice/Hausdorff (segmentation), LPIPS and FID/SliceFID (perceptual/volumetric realism), and application-specific topological, density, or geometric error (microvasculature, meteorology) are increasingly adopted (Pérez-Bueno et al., 2024, Jiang et al., 2024, Yao et al., 6 Apr 2026).
• Efficiency/Bandwidth: Inference time and parameter count are explicitly reported to demonstrate real-time or feasible deployment, e.g., SwinT-PNCC at 0.164 s/frame (4×) with 11.7M params (Mas et al., 11 Nov 2025), or feed-forward mapping yielding multi-order-of-magnitude speedup over iterative per-scene optimization (Feng et al., 27 Feb 2026).
• Ablation and Trade-off Analysis: Studies routinely compare backbone variants (e.g., SwinT-PNCC vs. VM-PNCC), effect of representation choice (depth vs. PNCC), and the value of explicit geometric or physical priors.

Representative outcomes show substantial improvements of up to an order of magnitude in RMSE (depth), sharp reduction in microvascular localization error (to 1.8 μm from >25 μm), and 2×–3× improvements in light-field and microscopy axial/lateral resolution, depending on the order of the SOFI/SOFFLFM cumulants or degree of saturation in illumination (Mas et al., 11 Nov 2025, Yao et al., 6 Apr 2026, Tran et al., 2022, Huang et al., 2022).

6. Applications, Limitations, and Extensions

3DSR underpins a wide range of modern imaging and simulation applications:

• Single-View Depth Fusion and Active Imaging: Real-time upsampling of depth sensors without need for RGB guidance (Mas et al., 11 Nov 2025, Ruget et al., 2020).
• Science and Medicine: Enhanced MRI, fMRI, CT, PET, and biological microscopy data, preserving function and structural saliency under severe acquisition constraints (Pérez-Bueno et al., 2024, Jiang et al., 2024, Pendharker et al., 2016).
• Video and Multiview Scene Synthesis: High-fidelity 3DGS, NeRF, and light-field approaches for graphics, robotics, and digital content (Chen et al., 6 Aug 2025, Feng et al., 27 Feb 2026, Tran et al., 2022).
• Physical Simulations: Building- and street-scale, physically faithful upscaling for meteorological forecasting and computational fluid dynamics (Yasuda et al., 2023, Yasuda et al., 2024).
• Vascular and Cellular morphology: Accurate recovery in super-resolution ultrasound and microscopy, with clinical and research implications (Yao et al., 6 Apr 2026, Huang et al., 2022).

Limitations are frequently domain-linked:

• Self-supervised and orientation-agnostic approaches may introduce artifacts due to lack of explicit 3D consistency (Jiang et al., 2024).
• Diffusion and GAN-based approaches can be computationally intensive and memory-bounded (Chen et al., 6 Aug 2025, Wang et al., 2023).
• Resource demands, dataset diversity, and generalization remain open requirements for robust adoption.
• Physics-driven constraints and topological priors may need retuning for transfer between domains or modalities.

7. Future Directions and Open Challenges

Key ongoing and anticipated advances in 3DSR include:

• Explicit 3D Consistency: Development of 3D-aware diffusion, GAN, and transformer architectures that propagate geometric and physical cues end-to-end, eliminating reliance on 2D surrogate supervision (Chen et al., 6 Aug 2025, Feng et al., 27 Feb 2026).
• Self-supervised and Data-efficient Learning: Further minimization of ground-truth dependency, via data-intrinsic invariance, weak labeling, or self-distillation—especially relevant in biomedical and simulation domains (Pérez-Bueno et al., 2024, Jiang et al., 2024).
• Speed and Scalability: Efficient practical deployment for real-time applications, via network pruning, patch and token optimization, and feed-forward volumetric reconstruction (Mas et al., 11 Nov 2025, Feng et al., 27 Feb 2026).
• Physical and Topological Priors: Deeper, theoretically justified incorporation of physical laws, anatomical constraints, or scene geometry (e.g., divergence-free flows, vessel connectivity) for robust and interpretable 3D SR (Yasuda et al., 2023, Yao et al., 6 Apr 2026).
• Cross-modal, Cross-domain Extension: Unification of approaches for extension across medical imaging, scientific simulation, robotics, and content creation, opting either for modular pipelines or fully unified architectures.

A plausible implication is that advances in 3DSR will increasingly bridge the gap between high-fidelity, artifact-free reconstructions and practical requirements of data-limited, real-time, or physically constrained scenarios, solidifying 3DSR as an essential tool in computational imaging and scientific discovery.