Real Image Denoising with Knowledge Distillation for High-Performance Mobile NPUs

Abstract: While deep-learning-based image restoration has achieved unprecedented fidelity, deployment on mobile Neural Processing Units (NPUs) remains bottlenecked by operator incompatibility and memory-access overhead. We propose an NPU-aware hardware-algorithm co-design approach for real-world image denoising on mobile NPUs. Our approach employs a high-capacity teacher to supervise a lightweight student network specifically designed to leverage the tiled-memory architectures of modern mobile SoCs. By prioritizing NPU-native primitives -- standard 3x3 convolutions, ReLU activations, and nearest-neighbor upsampling -- and employing a progressive context expansion strategy (up to 1024x1024 crops), the model achieves 37.66 dB PSNR / 0.9278 SSIM on the validation benchmark and 37.58 dB PSNR / 0.9098 SSIM on the held-out test benchmark at full resolution (2432x3200) in the Mobile AI 2026 challenge. Following the official challenge rules, the inference runtime is measured under a standardized Full HD (1088x1920) protocol, where it runs in 34.0 ms on the MediaTek Dimensity 9500 and 46.1 ms on the Qualcomm Snapdragon 8 Elite NPU. We further reveal an "Inference Inversion" effect, where strict adherence to NPU-compatible operations enables dedicated NPU execution up to 3.88x faster than the integrated mobile GPU. The 1.96M-parameter student recovers 99.8% of the teacher's restoration quality via high-alpha knowledge distillation (alpha = 0.9), achieving a 21.2x parameter reduction while closing the PSNR gap from 1.63 dB to only 0.05 dB. These results establish hardware-aware distillation as an effective strategy for unifying high-fidelity denoising with practical deployment across diverse mobile NPU architectures. The proposed lightweight student model (LiteDenoiseNet) and its training statistics are provided in the NN Dataset, available at https://github.com/ABrain-One/NN-Dataset.

• The paper introduces a hardware-algorithm co-design framework where a high-capacity teacher guides a 1.96M parameter student to achieve nearly identical PSNR performance.
• The methodology leverages standard NPU-native operators and a high-α distillation regime to ensure real-time denoising on mobile NPUs like MediaTek Dimensity 9500 and Snapdragon 8 Elite.
• The technique demonstrates practical deployment benefits by reducing inference times and computational cost while preserving fine image details, challenging conventional reliance on mobile GPUs.

Real Image Denoising with Knowledge Distillation for High-Performance Mobile NPUs

Introduction

This paper addresses the gap between high-fidelity image denoising achieved by deep learning and real-time deployment on resource-constrained mobile Neural Processing Units (NPUs). The authors formulate a hardware–algorithm co-design paradigm whereby a high-capacity teacher network trains a lightweight student architecture. The student is engineered to match the constraints of tiled-memory, operator-restricted mobile NPUs, using only standard $3\times3$ convolutions, ReLU, and nearest-neighbor upsampling. A high-$\alpha$ distillation regime ensures that the student closes the performance gap with the teacher—achieving a $0.08$ dB mean PSNR difference—while remaining deployable in real time on both MediaTek Dimensity 9500 and Snapdragon 8 Elite NPUs.

The approach establishes that hardware-aware distillation into a native operator set can deliver near-teacher-level denoising without fallback to CPU or mobile GPU, a common deployment bottleneck seen in other contemporary solutions.

Background and Related Work

Image denoising has evolved from baseline convolutional residual networks (e.g., DnCNN, FFDNet) targeting synthetic noise to more sophisticated architectures for real-world smartphone images with complex spatially variant noise statistics (CBDNet, RIDNet, MIRNet, HINet, Restormer, NAFNet). While these models deliver strong PSNR/SSIM numbers for desktop evaluation, mobile transferability is hindered by non-NPU-native components such as self-attention, deformable convolutions, or upsampling modules—frequently resulting in CPU fallback or OOM on high-resolution, on-device execution.

Knowledge distillation is an established paradigm in classification and image restoration, but prior works often neglect hardware compatibility as a first-order design constraint. This paper differs by co-designing the student architecture for NPU fast-path compatibility from the outset, ensuring stable execution and maximal throughput across divergent SoC architectures. The work builds on new deployment-oriented mobile denoising benchmarks (e.g., Flepp et al.'s MIDD dataset, Mobile AI Challenge series) that focus on both fidelity and device execution.

Methodology

The architecture comprises two primary networks: a high-capacity U-Net-style teacher and a compact, NPU-centric student (LiteDenoiseNet).

• Teacher: A 41.6M parameter U-Net variant emphasizing high-capacity convolutional representation. It uses standard skip connections, DenoisingBlocks with dense aggregation and PReLU, and transposed convolution-based upsampling.
• Student: A 1.96M parameter model strictly limited to NPU-fast-path operators—standard $3\times3$ convolutions, hardware-native ReLU, local residual links (LiteDenoisingBlock), and nearest-neighbor upsampling. Full transposed convolutions and dynamically shaped intermediate activations are eschewed to avoid memory tiling violations and operator fallbacks.

  Figure 1: The teacher–student pipeline, emphasizing supervised distillation and seamless deployment via TFLite for fallback-free, real-time NPU inference.

Distillation uses a high weight on the perceptual guidance ($\alpha=0.9$), combining MSE on teacher predictions, ground-truth MSE, and L1 loss to ensure textural and structural fidelity. Training leverages a progressive context expansion: initial optimization on small crops yields to fine-tuning with progressively larger $512\times512$ and $1024\times1024$ regions, exposing the student to greater contextual statistics for improved global consistency at deployment scale.

Figure 2: LiteDenoisingBlock—core building unit of the student, implementing an internal bottleneck and residual link with only NPU-native operations.

Figure 3: LiteDenoiseNet, a highly efficient, compact U-Net variant for mobile denoising with strictly hardware-compatible design.

Tensor layout conversion (NCHW→NHWC) is integrated natively during PyTorch-to-TFLite export for unambiguous memory access by the mobile NPU. Rigorous numerical parity validation is performed to ensure fidelity in exported models.

Empirical Results

Benchmarking employs the Mobile AI 2026 challenge dataset: 650 paired images for training, validation on 50 images, and a held-out test set. Ablation illustrates that conventional "lightweight" architectures (~0.34M–0.35M parameters) or those with attention fail to sufficiently model sensor noise or exceed device memory. The teacher, while maximally accurate (37.71 dB PSNR), is not deployable at full resolution.

The critical result: distillation into the 1.96M student model recovers 99.8% of teacher PSNR (37.66 dB vs. 37.71 dB), with a 21.2$\times$ parameter reduction and robust denoising on $8$MP images. This is achieved with

• 34.0 ms (Dimensity 9500) and 46.1 ms (Snapdragon 8 Elite) NPU runtime at Full HD, and,
• 132 ms on a mobile GPU—demonstrating a 3.88$\times$ speedup for the NPU (“Inference Inversion” effect).

Qualitative analysis further reveals that LiteDenoiseNet preserves fine textures and structures, avoiding the oversmoothing typical in aggressively compact architectures.

Figure 4: Sample validation output: noisy input, lightweight student output, and ground truth; LiteDenoiseNet reconstructs fine detail while suppressing visible noise.

Per-image PSNR analysis indicates the residual gap between teacher and student clusters tightly around zero. In rare cases, the student outperforms the teacher due to overfitting mitigation in the distilled training objective.

Figure 5: Distribution of per-image PSNR gap between teacher and student across the validation set.

Implications for Mobile Vision Deployment

The methodology and results shift the paradigm for mobile imaging and low-level vision system design:

• Hardware co-design is necessary—not optional—to bridge the fidelity–efficiency gap for on-device imaging tasks.
• NPUs can outperform mobile GPUs on compatible architectures, contradicting the status-quo assumption of always preferring mobile GPU inference for restoration workloads.
• Distillation must target a strictly compatible operator set; post-hoc conversion or model slimming without regard for fallback behavior is insufficient.

The techniques generalize to other ill-posed restoration problems—deblurring, super-resolution, raw denoising—pending expansion of the operator set and context exposure strategies for next-generation mobile AI hardware.

Limitations and Future Outlook

The study's main constraint is reliance on mobile benchmark runtimes provided by challenge organizers; comprehensive, uncontrolled real-world latency for other platforms is unaddressed. The operator set is intentionally conservative for cross-platform security but may restrict deployment on more heterogeneous, capable future NPUs.

Scaling LiteDenoiseNet to richer operator sets (group/depthwise convolution, fast attention) and extending progressive context exposure (wider crops, transformer-guided global context) are potential avenues for even higher quality at comparable or lower computational budgets. As NPU platforms evolve, co-optimization frameworks incorporating automatic operator compatibility reasoning and architecture search (potentially LLM-guided) should further blur the gap between research prototypes and practical deployment.

Conclusion

This work demonstrates that knowledge distillation into an NPU-native, compact architecture facilitates competitive real-image denoising on flagship mobile SoCs, matching teacher-level restoration at dramatically lower complexity and power cost. The evidence—quantitative, qualitative, and runtime—directly supports hardware-aware algorithmic co-design as the foundation for deployable mobile vision. Such frameworks are likely to become central to multi-platform low-level vision and are applicable to a wide range of resource-constrained AI deployments where fidelity and latency are jointly paramount.