NanoVSR: Towards Real-Time Video Super-Resolution on Edge Devices

Abstract: Recent Video Super-Resolution (VSR) methods rely heavily on transformers and explicit optical flow, creating computational overhead and custom operations that hinder deployment on hardware accelerators like TensorRT. To address this, we introduce NanoVSR, a scalable, fully convolutional architecture designed for resource-constrained edge devices. Using structural reparameterization, NanoVSR collapses into standard convolutions during inference, ensuring seamless hardware compatibility and negligible runtime overhead. Furthermore, despite lacking explicit motion compensation, it maintains competitive restoration quality by implicitly learning spatio-temporal alignments through progressive training. Evaluated on the REDS4 benchmark, NanoVSR demonstrates an exceptional balance between accuracy and computational efficiency, significantly improving the trade-off for compact architectures. Our NanoVSR-644k baseline yields 28.64 dB PSNR while delivering 27.2 FPS on the NVIDIA Jetson Orin NX 16GB (25W), offering massive speed gains over heavier models. The scaled NanoVSR-1.7M variant reaches 29.15 dB with a throughput of 19.58 FPS, providing superior, edge-optimized upscaling. Code is available at https://github.com/filippawlicki/nanovsr.

• The paper introduces NanoVSR, a convolutional bidirectional recurrent model that achieves real-time video super-resolution on edge devices without using explicit motion compensation.
• It employs structural reparameterization to collapse multi-branch convolution operations into a single 3×3 convolution, thereby boosting throughput and reducing memory use.
• Empirical results show that NanoVSR achieves competitive PSNR and FPS on platforms like NVIDIA Jetson Orin NX, demonstrating an optimal trade-off between fidelity and computational cost.

NanoVSR: Towards Real-Time Video Super-Resolution on Edge Devices

Motivation and Challenges in Edge-Friendly VSR

Video Super-Resolution (VSR) aims to reconstruct high-resolution (HR) frames from low-resolution (LR) counterparts by leveraging the temporal redundancy in consecutive frames. While state-of-the-art approaches—e.g., EDVR, BasicVSR, and RVRT—exploit optical flow or heavy self-/cross-attention modules to align and aggregate spatio-temporal features, such methods are resource-prohibitive for edge deployment, suffering from high latency, significant memory bandwidth consumption, and custom CUDA dependencies. NanoVSR directly addresses this inefficiency, targeting real-time VSR under strict compute and power constraints, as found in platforms such as NVIDIA Jetson Orin NX.

Architectural Design

NanoVSR is a purely convolutional, bidirectional recurrent architecture that completely eschews explicit motion compensation and exotic attention mechanisms. The entire model is composed of operations natively supported in deployment compilers (e.g., ONNX, TensorRT), facilitating seamless translation into edge executables. As illustrated in the architectural schematic, LR input frames are mapped into a latent space and processed bidirectionally through a stack of reparameterizable blocks. Critically, NanoVSR leverages element-wise addition for recurrent propagation instead of channel concatenation, minimizing both compute and memory bandwidth requirements.

Figure 1: Detailed schematic of the NanoVSR architecture. LR input frames are embedded into a latent space and passed through a bidirectional recurrent structure built with reparameterizable blocks.

Efficiency via Structural Reparameterization

A central innovation underlying NanoVSR's efficiency is the use of structural reparameterization. During training, feature extraction and temporal propagation blocks consist of multi-branch topologies—parallel $3 \times 3$ and $1 \times 1$ convolutions alongside identity, each followed by BN. At deployment, these branches (and their associated BN statistics) are mathematically collapsed into a single $3 \times 3$ convolution. Consequently, the inference path is a contiguous stack of standard convolutions, optimized for high-throughput on accelerator hardware.

Figure 2: Diagram of the structural reparameterization process. Multi-branch train-time blocks collapse into a single $3 \times 3$ convolution for inference.

Progressive Bidirectional Recurrence and Reconstruction

The architecture propagates features in both temporal directions, updating forward and backward hidden states through element-wise addition with the input. Features from both directions are fused and upsampled via cascaded sub-pixel convolutions (PixelShuffle), with a global residual connection using learned bilinear upsampling for base projection. This design choice optimally trades off restoration quality and throughput.

Optimization Strategy

NanoVSR employs a two-stage curriculum:

1. Pre-training on short (7-frame) Vimeo-90K sequences to promote stable spatial/short-term temporal feature learning.
2. Fine-tuning on longer (30-frame) REDS sub-sequences to foster robust long-term recurrence without explicit motion estimation.

The model is optimized with the Charbonnier loss via Adam, leveraging AMP (BFloat16) and aggressive gradient clipping. This training pipeline is critical to achieving convergence and robust propagation in the absence of explicit alignment.

Empirical Results

NanoVSR demonstrates a substantial efficiency-quality trade-off shift relative to classical and SOTA methods across all benchmarks (REDS4, Vid4, Vimeo90K-T). The NanoVSR-644k baseline delivers 28.64 dB PSNR at 27.2 FPS on NVIDIA Jetson Orin NX (25W). Scaling to NanoVSR-1.7M yields 29.15 dB (REDS4) at 19.6 FPS—maintaining real-time capability on edge hardware. Notably, transformer solutions like RVRT exhibit only marginal PSNR gains for a 10–20x compute cost, confirming the architectural optimization of NanoVSR.

Qualitative visual comparison on REDS4 highlights competitive restoration quality and textural fidelity versus much heavier methods.

Figure 3: Qualitative comparison on REDS4. NanoVSR recovers sharp details closely matching heavier transformer-based architectures.

Complementary evaluation on text-rich, heavily deformed Vid4 sequences underscores the robustness of NanoVSR to challenging content.

Figure 4: Qualitative comparison on Vid4. NanoVSR maintains competitive visual fidelity in high-motion, text-bearing video regions.

Model Scaling and Ablation

Scaling NanoVSR’s parameter count yields significant gains in the sub-1M regime, but returns diminish above 5M parameters, indicating adequate representational capacity for the architecture's design. The ablation study establishes that structural reparameterization profoundly improves practical throughput without loss of fidelity. Removing curriculum pre-training or reverting to a single-branch architecture reduces generalization and sharpness. Attaching an explicit SPyNet optical flow estimator increases PSNR but more than doubles runtime and parameter count—defeating the edge deployment goal.

Figure 5: Model scaling analysis on REDS4. Restoration quality (PSNR) saturates beyond moderate capacity, indicating architectural efficiency.

Limitations and Implications

NanoVSR's implicit alignment can underperform in cases of severe non-rigid or occluded motion, occasionally failing to reconstruct high-frequency textures. Additionally, bidirectional recurrence necessitates temporal buffering, potentially introducing minor latency inappropriate for strict online applications. From a societal impact perspective, NanoVSR facilitates sustainable VSR with dramatically lower energy consumption, broader accessibility, and prolonged usability of legacy imaging hardware. However, the probabilistic nature of enhancement mandates cautious application in sensitive domains (e.g., surveillance, forensics).

Conclusion

NanoVSR demonstrates that fully convolutional, structurally reparameterized, bidirectional recurrent architectures can achieve highly efficient, high-quality real-time video super-resolution under edge constraints. By dispensing with explicit optical flow and transformer modules, and adopting direct additive propagation with progressive training, NanoVSR redefines the trade-off between restoration fidelity and hardware cost. This shift enables widespread local deployment of learned VSR, opening further research into sustainable, scalable visual enhancement for embedded and low-power platforms.