Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Neural Field Convolutions by Repeated Differentiation (2304.01834v4)

Published 4 Apr 2023 in cs.CV and cs.GR

Abstract: Neural fields are evolving towards a general-purpose continuous representation for visual computing. Yet, despite their numerous appealing properties, they are hardly amenable to signal processing. As a remedy, we present a method to perform general continuous convolutions with general continuous signals such as neural fields. Observing that piecewise polynomial kernels reduce to a sparse set of Dirac deltas after repeated differentiation, we leverage convolution identities and train a repeated integral field to efficiently execute large-scale convolutions. We demonstrate our approach on a variety of data modalities and spatially-varying kernels.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. The Theory of Splines and Their Applications. Vol. 38. Elsevier.
  2. Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. ICCV (2021).
  3. Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields. CVPR (2022).
  4. E Oran Brigham. 1988. The fast Fourier transform and its applications. Prentice-Hall, Inc.
  5. Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graph. 26, 3 (2007), 1–9.
  6. Franklin C Crow. 1984. Summed-area tables for texture mapping. In SIGGRAPH. 207–212.
  7. Learning Signal-Agnostic Manifolds of Neural Fields. In NeurIPS.
  8. Coin: Compression with implicit neural representations. In ICLR (Neural Compression Workshop).
  9. From data to functa: Your data point is a function and you can treat it like one, Vol. 162. PMLR, 5694–5725.
  10. Generative Models as Distributions of Functions, Vol. 151. PMLR, 2989–3015.
  11. Hyperdiffusion: Generating implicit neural fields with weight-space diffusion. In ICCV.
  12. Convolution pyramids. ACM Trans. Graph. 30, 6 (2011), 1–8.
  13. Multiplicative filter networks. In ICLR.
  14. Alain Fournier and Eugene Fiume. 1988. Constant-time filtering with space-variant kernels. ACM Trans. Graph. 22, 4 (1988), 229–238.
  15. The design and use of steerable filters. IEEE TPAMI 13, 9 (1991), 891–906.
  16. Paul S Heckbert. 1986. Filtering by repeated integration. ACM Trans. Graph. 20, 4 (1986), 315–321.
  17. Monte carlo convolution for learning on non-uniformly sampled point clouds. ACM Trans. Graph. 37, 6 (2018), 1–12.
  18. Sape: Spatially-adaptive progressive encoding for neural optimization. NeurIPS 34 (2021), 8820–8832.
  19. Exact-NeRF: An Exploration of a Precise Volumetric Parameterization for Neural Radiance Fields. CVPR (2023).
  20. Gradient-domain path tracing. ACM Trans. Graph. 34, 4 (2015), 1–13.
  21. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.
  22. Neural Point Catacaustics for Novel-View Synthesis of Reflections. ACM Trans. Graph. 41, 6 (2022), 1–15.
  23. Noise2Noise: Learning Image Restoration without Clean Data. In ICML. 2965–2974.
  24. Laplacian Kernel Splatting for Efficient Depth-of-field and Motion Blur Synthesis or Reconstruction. ACM Trans. Graph. 37, 4 (2018), 1–11.
  25. Tony Lindeberg. 2013. Scale-space theory in computer vision. Vol. 256. Springer Science & Business Media.
  26. Autoint: Automatic integration for fast neural volume rendering. In CVPR. 14556–14565.
  27. Bacon: Band-limited coordinate networks for multiscale scene representation. In CVPR. 16252–16262.
  28. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV. 405–421.
  29. Efficient Spatially Adaptive Convolution and Correlation. Technical Report 2006.13188. arXiv preprint.
  30. Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. ACM Trans. Graph. 41, 4 (2022), 1–15.
  31. Harald Niederreiter. 1992. Low-discrepancy point sets obtained by digital constructions over finite fields. Czechoslovak Mathematical Journal 42, 1 (1992), 143–166.
  32. Deepsdf: Learning continuous signed distance functions for shape representation. In CVPR. 165–174.
  33. Nerfies: Deformable neural radiance fields. In ICCV. 5865–5874.
  34. Automatic differentiation in pytorch. (2017).
  35. Kenneth Perlin. 1984. Personal communication with Paul Heckbert, mentioned in Heckbert [1986].
  36. Searching for activation functions. arXiv preprint arXiv:1710.05941 (2017).
  37. Nicholas Sharp and Alec Jacobson. 2022. Spelunking the Deep: Guaranteed Queries on General Neural Implicit Surfaces via Range Analysis. ACM Trans. Graph. 41, 4 (2022), 1–16.
  38. From discrete to continuous convolution layers. arXiv preprint arXiv:2006.11120 (2020).
  39. Boxlets: a fast convolution algorithm for signal processing and neural networks. NeurIPS 11 (1998).
  40. Analysis of sample correlations for Monte Carlo rendering. In Comp. Graph. Forum, Vol. 38. Wiley Online Library, 473–491.
  41. Implicit neural representations with periodic activation functions. NeurIPS 33 (2020), 7462–7473.
  42. Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations. In NeurIPS.
  43. Ilya Meerovich Sobol. 1967. On the distribution of points in a cube and the approximate evaluation of integrals. Zhurnal Vychislitel’noi Matematiki i Matematicheskoi Fiziki 7, 4 (1967), 784–802.
  44. Kenneth O Stanley. 2007. Compositional pattern producing networks: A novel abstraction of development. Genetic programming and evolvable machines 8 (2007), 131–162.
  45. Fourier features let networks learn high frequency functions in low dimensional domains. NeurIPS 33 (2020), 7537–7547.
  46. Advances in neural rendering. Comp. Graph. Forum 41, 2 (2022), 703–735.
  47. Carlo Tomasi and Roberto Manduchi. 1998. Bilateral filtering for gray and color images. In ICCV. 839–846.
  48. Non-rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video. In ICCV. 12959–12970.
  49. CUF: Continuous Upsampling Filters. In CVPR.
  50. Differentiable signed distance function rendering. ACM Trans. Graph. 41, 4 (2022), 1–18.
  51. Paul Viola and Michael Jones. 2001. Rapid object detection using a boosted cascade of simple features. In CVPR, Vol. 1. I–I.
  52. Deep parametric continuous convolutional neural networks. In CVPR. 2589–2597.
  53. NeRFocus: Neural Radiance Field for 3D Synthetic Defocus. arXiv preprint arXiv:2203.05189 (2022).
  54. Lance Williams. 1983. Pyramidal parametrics. In SIGGRAPH, Vol. 17. 1–11.
  55. Andrew P Witkin. 1987. Scale-space filtering. In Readings in Computer Vision. 329–332.
  56. Neural fields in visual computing and beyond. Comp. Graph. Forum 41, 2 (2022), 641–676.
  57. Signal Processing for Implicit Neural Representations. In NeurIPS.
  58. Geometry processing with neural fields. NeurIPS 34 (2021), 22483–22497.
  59. Polynomial neural fields for subband decomposition and manipulation. NeuRIPS 35 (2022), 4401–4415.
  60. NeRF-editing: geometry editing of neural radiance fields. In ICCV. 18353–18364.
  61. Xian-Da Zhang. 2022. Modern signal processing. In Modern Signal Processing. De Gruyter.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets