2000 character limit reached
Efficient Hybrid Zoom using Camera Fusion on Mobile Phones (2401.01461v1)
Published 2 Jan 2024 in cs.CV
Abstract: DSLR cameras can achieve multiple zoom levels via shifting lens distances or swapping lens types. However, these techniques are not possible on smartphone devices due to space constraints. Most smartphone manufacturers adopt a hybrid zoom system: commonly a Wide (W) camera at a low zoom level and a Telephoto (T) camera at a high zoom level. To simulate zoom levels between W and T, these systems crop and digitally upsample images from W, leading to significant detail loss. In this paper, we propose an efficient system for hybrid zoom super-resolution on mobile devices, which captures a synchronous pair of W and T shots and leverages machine learning models to align and transfer details from T to W. We further develop an adaptive blending method that accounts for depth-of-field mismatches, scene occlusion, flow uncertainty, and alignment errors. To minimize the domain gap, we design a dual-phone camera rig to capture real-world inputs and ground-truths for supervised training. Our method generates a 12-megapixel image in 500ms on a mobile platform and compares favorably against state-of-the-art methods under extensive evaluation on real-world scenarios.
- Symmetrical dense optical flow estimation with occlusions detection. IJCV 75 (2007), 371–385.
- Wireless software synchronization of multiple distributed cameras. In ICCP. IEEE, Tokyo, Japan, 1–9.
- GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution. In CVPR. IEEE, Virtual/Online, 14245–14254.
- Ferenc Huszar Jose Caballero Andrew Cunningham Alejandro Acosta Andrew Aitken Alykhan Tejani Johannes Totz Zehan Wang Wenzhe Shi Christian Ledig, Lucas Theis. 2017. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. In CVPR.
- Xiaodong Cun and Chi-Man Pun. 2020. Defocus blur detection via depth distillation. In ECCV.
- Learning a deep convolutional network for image super-resolution. In ECCV.
- Jochen Gast and Stefan Roth. 2018. Lightweight probabilistic deep networks. In ICCV.
- Image processing using multi-code GAN prior. In CVPR.
- Burst photography for high dynamic range and low-light imaging on mobile cameras. ACM TOG (2016).
- GCFSR: a Generative and Controllable Face Super Resolution Method Without Facial and GAN Priors. In CVPR.
- HonorMagic 2023. Honor Magic4 Ultimate Camera test. https://www.dxomark.com/honor-magic4-ultimate-camera-test-retested/. Accessed: 2023-03-07.
- Task decoupled framework for reference-based super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
- Robust Reference-based Super-Resolution via C2-Matching. In CVPR.
- Perceptual losses for real-time style transfer and super-resolution. In ECCV.
- Accurate image super-resolution using very deep convolutional networks. In CVPR.
- Deep Laplacian pyramid networks for fast and accurate super-resolution. In CVPR.
- Face deblurring using dual camera fusion on mobile phones. ACM TOG (2022).
- Reference-based video super-resolution using multi-camera video triplets. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
- Deep defocus map estimation using domain adaptation. In CVPR.
- MASA-SR: Matching acceleration and spatial adaptation for reference-based image super-resolution. In CVPR.
- The contextual loss for image transformation with non-aligned data. In ECCV.
- Pulse: Self-supervised photo upsampling via latent space exploration of generative models. In CVPR.
- Deconvolution and Checkerboard Artifacts. Distill (2016). http://distill.pub/2016/deconv-checkerboard/
- Attention-based multi-reference learning for image super-resolution. In CVPR.
- Film: Frame interpolation for large motion. In ECCV.
- Color transfer between images. IEEE Computer graphics and applications 21, 5 (2001), 34–41.
- RAISR: rapid and accurate image super resolution. IEEE TCI (2016).
- U-net: Convolutional networks for biomedical image segmentation. In MICCAI.
- Edward Rosten and Tom Drummond. 2006. Machine learning for high-speed corner detection. In ECCV.
- Robust reference-based super-resolution with similarity-aware deformable convolution. In CVPR.
- Disentangling Architecture and Training for Optical Flow. In ECCV.
- AutoFlow: Learning a Better Training Set for Optical Flow. In CVPR.
- Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume. In CVPR.
- Libin Sun and James Hays. 2012. Super-resolution from internet-scale scene matching. In ICCP.
- Richard Szeliski. 2022. Computer vision: algorithms and applications. Springer Nature.
- Defusionnet: Defocus blur detection via recurrently fusing and refining multi-scale deep features. In CVPR.
- Zachary Teed and Jia Deng. 2020. Raft: Recurrent all-pairs field transforms for optical flow. In ECCV.
- Robert Triggs. 2023. All the new HUAWEI P40 camera technology explained. https://www.androidauthority.com/huawei-p40-camera-explained-1097350/. Accessed: 2023-03-07.
- Multi-view image fusion. In CVPR.
- Dual-camera super-resolution with aligned attention modules. In CVPR.
- ESRGAN: Enhanced super-resolution generative adversarial networks. In ECCV.
- Event-specific image importance. In CVPR.
- Component divide-and-conquer for real-world image super-resolution. In ECCV.
- Handheld multi-frame super-resolution. ACM TOG (2019).
- Coarse-to-Fine Embedded PatchMatch and Multi-Scale Dynamic Aggregation for Reference-based Super-Resolution. In AAAI.
- Feature representation matters: End-to-end learning for reference-based image super-resolution. In ECCV.
- Defocus map estimation and deblurring from a single dual-pixel image. In ICCV.
- Zero-Shot Dual-Lens Super-Resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
- Learning texture transformer network for image super-resolution. In CVPR.
- Designing a practical degradation model for deep blind image super-resolution. In ICCV.
- Zoom to learn, learn to zoom. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
- Efficient Long-Range Attention Network for Image Super-resolution. In ECCV.
- Image super-resolution using very deep residual channel attention networks. In ECCV.
- Self-Supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations. In ECCV.
- Image super-resolution by neural texture transfer. In CVPR.
- Enhancing diversity of defocus blur detectors via cross-ensemble network. In CVPR.
- Crossnet: An end-to-end reference-based super resolution network using cross-scale warping. In ECCV.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.