Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Dynamic Implicit Image Function for Efficient Arbitrary-Scale Image Representation (2306.12321v2)

Published 21 Jun 2023 in cs.CV

Abstract: Recent years have witnessed the remarkable success of implicit neural representation methods. The recent work Local Implicit Image Function (LIIF) has achieved satisfactory performance for continuous image representation, where pixel values are inferred from a neural network in a continuous spatial domain. However, the computational cost of such implicit arbitrary-scale super-resolution (SR) methods increases rapidly as the scale factor increases, which makes arbitrary-scale SR time-consuming. In this paper, we propose Dynamic Implicit Image Function (DIIF), which is a fast and efficient method to represent images with arbitrary resolution. Instead of taking an image coordinate and the nearest 2D deep features as inputs to predict its pixel value, we propose a coordinate grouping and slicing strategy, which enables the neural network to perform decoding from coordinate slices to pixel value slices. We further propose a Coarse-to-Fine Multilayer Perceptron (C2F-MLP) to perform decoding with dynamic coordinate slicing, where the number of coordinates in each slice varies as the scale factor varies. With dynamic coordinate slicing, DIIF significantly reduces the computational cost when encountering arbitrary-scale SR. Experimental results demonstrate that DIIF can be integrated with implicit arbitrary-scale SR methods and achieves SOTA SR performance with significantly superior computational efficiency, thereby opening a path for real-time arbitrary-scale image representation. Our code can be found at https://github.com/HeZongyao/DIIF.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (32)
  1. Ntire 2017 challenge on single image super-resolution: Dataset and study. In CVPRW, pages 126–135, 2017.
  2. Sal: Sign agnostic learning of shapes from raw data. In CVPR, pages 2565–2574, 2020.
  3. Deep local shapes: Learning local sdf priors for detailed 3d reconstruction. In ECCV, pages 608–625. Springer, 2020.
  4. Pre-trained image processing transformer. In CVPR, pages 12299–12310, 2021.
  5. Learning continuous image representation with local implicit image function. In CVPR, pages 8628–8638, 2021.
  6. Learning implicit fields for generative shape modeling. In CVPR, pages 5939–5948, 2019.
  7. Learning a deep convolutional network for image super-resolution. In ECCV, pages 184–199, 2014.
  8. Implicit geometric regularization for learning shapes. arXiv preprint arXiv:2002.10099, 2020.
  9. Baking neural radiance fields for real-time view synthesis. In ICCV, pages 5875–5884, October 2021.
  10. Meta-sr: A magnification-arbitrary network for super-resolution. In CVPR, pages 1575–1584, 2019.
  11. Single image super-resolution from transformed self-exemplars. In CVPR, pages 5197–5206, 2015.
  12. Local implicit grid representations for 3d scenes. In CVPR, pages 6001–6010, 2020.
  13. Adam: A method for stochastic optimization. In ICLR, 2015.
  14. Hypernetwork functional image representation. In ICANN, pages 496–510. Springer, 2019.
  15. Deep laplacian pyramid networks for fast and accurate super-resolution. In CVPR, pages 624–632, 2017.
  16. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR, pages 4681–4690, 2017.
  17. Local texture estimator for implicit representation function. In CVPR, pages 1929–1938, 2022.
  18. Swinir: Image restoration using swin transformer. In ICCV, pages 1833–1844, 2021.
  19. Enhanced deep residual networks for single image super-resolution. In CVPRW, pages 136–144, 2017.
  20. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In ICCV, pages 416–423, 2001.
  21. Implicit surface representations as layers in neural networks. In ICCV, pages 4743–4752, 2019.
  22. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, pages 405–421. Springer, 2020.
  23. Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision. In CVPR, pages 3504–3515, 2020.
  24. Deepsdf: Learning continuous signed distance functions for shape representation. In CVPR, pages 165–174, 2019.
  25. Convolutional occupancy networks. In ECCV, pages 523–540. Springer, 2020.
  26. Implicit neural representations with periodic activation functions. NIPS, 33:7462–7473, 2020.
  27. Scene representation networks: Continuous 3d-structure-aware neural scene representations. NIPS, 32, 2019.
  28. Memnet: A persistent memory network for image restoration. In ICCV, pages 4539–4547, 2017.
  29. Nex: Real-time view synthesis with neural basis expansion. In CVPR, pages 8534–8543, June 2021.
  30. On single image scale-up using sparse-representations. In ICCS, pages 711–730, 2010.
  31. Image super-resolution using very deep residual channel attention networks. In ECCV, pages 286–301, 2018.
  32. Residual dense network for image super-resolution. In CVPR, pages 2472–2481, 2018.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now