CV-HoloSR: Hologram to hologram super-resolution through volume-upsampling three-dimensional scenes

Abstract: Existing hologram super-resolution (HSR) methods primarily focus on angle-of-view expansion. Adapting them for volumetric spatial up-sampling introduces severe quadratic depth distortion, degrading 3D focal accuracy. We propose CV-HoloSR, a complex-valued HSR framework specifically designed to preserve physically consistent linear depth scaling during volume up-sampling. Built upon a Complex-Valued Residual Dense Network (CV-RDN) and optimized with a novel depth-aware perceptual reconstruction loss, our model effectively suppresses over-smoothing to recover sharp, high-frequency interference patterns. To support this, we introduce a comprehensive large-depth-range dataset with resolutions up to 4K. Furthermore, to overcome the inherent depth bias of pre-trained encoders when scaling to massive target volumes, we integrate a parameter-efficient fine-tuning strategy utilizing complex-valued Low-Rank Adaptation (LoRA). Extensive numerical and physical optical experiments demonstrate our method's superiority. CV-HoloSR achieves a 32% improvement in perceptual realism (LPIPS of 0.2001) over state-of-the-art baselines. Additionally, our tailored LoRA strategy requires merely 200 samples, reducing training time by over 75% (from 22.5 to 5.2 hours) while successfully adapting the pre-trained backbone to unseen depth ranges and novel display configurations.

• The paper presents a complex-valued deep neural network (CV-RDN) that enables hologram super-resolution via volumetric upsampling and a depth-aware ASM-LPIPS loss.
• It demonstrates improved physical consistency and true linear depth scaling in 3D reconstructions through a custom dataset and physics-informed loss design.
• The study leverages LoRA-based adaptation for efficient tuning to new depth configurations, reducing adaptation time by 75% without performance loss.

CV-HoloSR: Complex-Valued Hologram Super-Resolution via Volumetric Upsampling

Introduction

The paper "CV-HoloSR: Hologram to hologram super-resolution through volume-upsampling three-dimensional scenes" (2604.10393) addresses the core limitations in current data-driven hologram super-resolution (HSR) methods, particularly their lack of support for volumetric spatial up-sampling with linear depth scaling. Existing methods generally focus on angle-of-view (AoV) expansion or are confined by dataset limitations, making them inadequate for artifacts-free, high-fidelity 3D reconstructions at expanded depth and spatial ranges.

The CV-HoloSR framework leverages a complex-valued deep neural architecture (CV-RDN), a depth-aware perceptual reconstruction loss, and an efficient dataset for training on large depth intervals. Moreover, the paper introduces a Low-Rank Adaptation (LoRA) approach for rapid adaptation to new depth configurations, overcoming the depth bias inherent in standard encoders.

Dataset Design and Volume Upsampling

The ability to generate and evaluate holograms with consistent linear depth scaling necessitated a custom dataset of paired low-resolution (LR) and high-resolution (HR) complex-valued holograms, each generated under a fixed pixel pitch and a physically plausible depth range, up to resolutions of $4096 \times 4096$. Depth intervals are selected according to a practical aspect ratio, typically $1:1:2$ for $(x:y:z)$, to avoid extreme over-smoothing associated with excessive diffraction pattern sharpness or insufficient generalization due to overly compact depth support.

Figure 1: The overview of hologram SR dataset with a pair of LR and HR.

Each training pair is constructed at a fixed hologram plane (zero-point) without mid-point dependencies, a critical design for agnostic inference scenarios. Hologram generation is conducted with an advanced silhouette-masking, layer-based CGH pipeline, capturing amplitude, phase, and occlusion handling at the RGB-D input level. Depth is discretized into 4,096 planes, and the physical aspect ratio maintains a scalable design for SR tasks.

Complex-Valued Residual Dense Network (CV-RDN)

The architecture adopts residual dense blocks implemented natively in the complex domain to model physical wave interactions. Each complex convolution preserves the phase-amplitude coupling essential for volumetric holography. Patches are extracted during training due to the prohibitively large input dimensions, and loss is enforced via complex-domain regression rather than generative adversarial objectives, as the latter would harm strict physical consistency. Upsampling is performed through complex sub-pixel convolutions, with pixel shuffling ensuring channel integrity for separate real and imaginary components.

Figure 2: The overview of the proposed network for hologram super-resolution.

For physically meaningful learning, all data augmentations are highly constrained—cropping is allowed, but flipping and rotation are avoided due to phase decorrelation and physical inconsistencies.

Cropping and Artifacts in Physical Reconstruction

Patch-based training, while practical, introduces severe boundary-induced ringing artifacts when hologram patches are propagated via the angular spectrum method (ASM). These artifacts have symmetry in both SR and HR holograms; thus, the loss across reconstructions cancels out their systematic bias.

Figure 3: Cropping-induced ringing artifacts in ASM reconstruction and the effect of the white-hologram formulation.

The network eschews explicit apodization or windowing, instead leveraging the physics-informed loss design to maintain edge and structural consistency, especially for focused/de-focused regions.

Novel Loss Function: Depth-aware ASM-LPIPS

To mitigate over-smoothing from complex pixel-wise loss and enforce perceptually plausible 3D reconstructions, the framework introduces a depth-aware ASM-LPIPS loss. The key strategy is to propagate both prediction and reference holograms to sampled depth planes and compare reconstructed images using the LPIPS metric, providing strong supervision on amplitude-phase features along the depth axis.

The loss is expressed as: $$\mathcal{L}_{\mathrm{ASM\text{-}LPIPS}} = \frac{1}{N} \sum_{i=1}^{N} \mathrm{LPIPS}(r_{z_i}, \hat{r}_{z_i})$$ where $r_{z_i}$ is the ASM-propagated field at plane $z_i$. Stratified random sampling across the patch-supported depth interval avoids supervision collapse in out-of-focus regions.

Parameter-Efficient Adaptation via LoRA

A fundamental problem in scaling CGH networks to large or varied depth intervals is the depth bias ingrained in pre-trained encoders. The encoder progressively "contracts" depth toward the interval observed during training, as evidenced by the focus-depth tracking across intermediate RDB feature maps.

Figure 4: Analysis of depth bias in the pretrained encoder. Focus depth curves for near and far objects reveal contraction towards a narrow depth interval through the CV-RDN.

To decouple the dependency between network weights and specific training depth distributions, the paper implements LoRA by injecting low-rank adapters only in critical CV-RDN convolution layers and fusion layers. This enables efficient fine-tuning to unseen depth statistics using as few as 200 samples, reducing adaptation runtimes by 75% (from 22.5 to 5.2 hours) without PSNR/SSIM or perceptual metric degradation.

Experimental Results

Extensive evaluation is conducted on synthetic (HologramSR), semi-real (Big Buck Bunny via monocular depth), and real image datasets (RealSR), with paired LR/HR holograms and depth-aligned ground truths. Optical and numerical reconstructions are validated in a $4f$ system with SLM and RGB laser illumination.

Figure 5: Optical system configuration for physical holographic reconstruction.

Quantitative Analysis

CV-HoloSR exceeds state-of-the-art baselines by 32% in LPIPS (0.2001 vs. 0.2926) on HologramSR. It is competitive or superior in SSIM and PSNR. The real performance margin arises in the accurate rendering of in-focus structures and natural volumetric defocus blur.

Figure 6: Qualitative comparison of reconstructed planes in LR, SR (proposed), and calibrated bicubic upsampling.

Figure 7: Visual quality comparison of focused and out-of-focus regions, highlighting superior detail and consistent defocus blur from CV-HoloSR.

A continuous sweep of reconstruction depth confirms that the model achieves true linear depth scaling, unlike naive methods with quadratic depth distortion.

Figure 8: Continuous volumetric reconstruction sweep demonstrates successful DoF expansion across propagation distance.

Ablation studies confirm the perceptual superiority of ASM-LPIPS (lower LPIPS, higher visual sharpness) over $L_1$-based loss for restoring high-frequency detail.

Figure 9: Qualitative and quantitative comparison of different loss function configurations.

Optical Verification

Physical optical reconstructions match closely to simulation, even accounting for SLM quantization and system imperfections. Far-plane degradations are primarily hardware-bound, not algorithm-induced.

Figure 10: Optical and numerical reconstruction results for LR, SR, and HR holograms.

LoRA-based Adaptation

Experiments on upscaling $384^2 \rightarrow 1536^2$ and $512^2 \rightarrow 2048^2$ with LoRA fine-tuning show nearly identical or superior performance to full scratch training, verifying LoRA's efficacy as a lightweight transfer mechanism.

Figure 11: Depth-range adaptation results under LoRA-based fine-tuning. LoRA$1:1:2$0 matches or exceeds scratch-trained performance.

Theoretical and Practical Implications

The proposed CV-HoloSR framework advances holographic neural rendering by enforcing and exploiting the physics of linear depth-volumetric scaling, proper amplitude-phase coupling via complex-valued operations, and efficient network adaptation via LoRA. It establishes foundations for real-time, high-fidelity holographic display pipelines, scalable to arbitrary depth volumes with minimal retraining effort. This is highly relevant for AR/VR, holographic microscopy, and computational displays where precise 3D scene reproduction is necessary.

Practical deployment in SLM-driven systems is supported by physical validation, and the complex-valued approach mitigates previous representational inefficiencies observed in real-valued networks for holography.

Limitations and Open Challenges

The computational overhead of current CVNNs and the complexity of complex-valued operations pose challenges for real-time inference in large-scale deployments. While LoRA enables efficient adaptation, true zero-shot depth generalization is unresolved. Future research directions include network quantization, optimized CVNN kernels, and physics-informed architectures capable of learning general volumetric scaling rules in a fundamentally data-agnostic manner.

Conclusion

The CV-HoloSR framework introduces a rigorous and effective solution for volumetric hologram super-resolution, overcoming critical limitations in depth scaling and physical consistency. By integrating complex-valued deep networks with perceptual, depth-aware supervision and parameter-efficient adaptation, this method sets a new standard for scalable, high-fidelity holographic scene upsampling and generalization to variable optical configurations.