Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Discretization Invariant Networks for Learning Maps between Neural Fields (2206.01178v4)

Published 2 Jun 2022 in cs.LG, cs.CV, and cs.NE

Abstract: With the emergence of powerful representations of continuous data in the form of neural fields, there is a need for discretization invariant learning: an approach for learning maps between functions on continuous domains without being sensitive to how the function is sampled. We present a new framework for understanding and designing discretization invariant neural networks (DI-Nets), which generalizes many discrete networks such as convolutional neural networks as well as continuous networks such as neural operators. Our analysis establishes upper bounds on the deviation in model outputs under different finite discretizations, and highlights the central role of point set discrepancy in characterizing such bounds. This insight leads to the design of a family of neural networks driven by numerical integration via quasi-Monte Carlo sampling with discretizations of low discrepancy. We prove by construction that DI-Nets universally approximate a large class of maps between integrable function spaces, and show that discretization invariance also describes backpropagation through such models. Applied to neural fields, convolutional DI-Nets can learn to classify and segment visual data under various discretizations, and sometimes generalize to new types of discretizations at test time. Code: https://github.com/clintonjwang/DI-net.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (74)
  1. Low-discrepancy point sets for non-uniform measures, 2013. URL https://arxiv.org/abs/1308.5049.
  2. Plenoxels: Radiance fields without neural networks, 2021.
  3. Zip-nerf: Anti-aliased grid-based neural radiance fields. ICCV, 2023.
  4. Are neural operators really neural operators? frame theory meets operator learning, 2023.
  5. Model reduction and neural networks for parametric pdes. arXiv preprint arXiv:2005.03180, 2020.
  6. Alexandre Boulch. Generalizing discrete convolutions for unstructured point clouds. CoRR, abs/1904.02375, 2019. URL http://arxiv.org/abs/1904.02375.
  7. On the koksma–hlawka inequality. Journal of Complexity, 29(2):158–172, 2013. ISSN 0885-064X. doi: https://doi.org/10.1016/j.jco.2012.10.003. URL https://www.sciencedirect.com/science/article/pii/S0885064X12000854.
  8. Russel E Caflisch. Monte carlo and quasi-monte carlo methods. Acta numerica, 7:1–49, 1998.
  9. pi-gan: Periodic implicit generative adversarial networks for 3d-aware image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  5799–5809, 2021.
  10. Approximations of continuous functionals by neural networks with application to dynamic systems. IEEE Transactions on Neural networks, 4(6):910–918, 1993.
  11. Approximation capability in c(r /sup n/) by multilayer feedforward networks and related problems. IEEE Transactions on Neural Networks, 6(1):25–30, 1995. doi: 10.1109/72.363453.
  12. Learning rotation-invariant and fisher discriminative convolutional neural networks for object detection. IEEE Transactions on Image Processing, 28(1):265–278, 2018.
  13. Streamable neural fields. arXiv preprint arXiv:2207.09663, 2022.
  14. Group equivariant convolutional networks. In International conference on machine learning, pp. 2990–2999. PMLR, 2016a.
  15. Steerable cnns. arXiv preprint arXiv:1612.08498, 2016b.
  16. The cityscapes dataset for semantic urban scene understanding. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  17. Mednerf: Medical neural radiance fields for reconstructing 3d-aware ct-projections from a single x-ray, 2022. URL https://arxiv.org/abs/2202.01020.
  18. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09, 2009.
  19. Generative models as distributions of functions. CoRR, abs/2102.04776, 2021. URL https://arxiv.org/abs/2102.04776.
  20. From data to functa: Your data point is a function and you should treat it like one. arXiv preprint arXiv:2201.12204, 2022.
  21. Robustness (python library), 2019. URL https://github.com/MadryLab/robustness.
  22. Splinecnn: Fast geometric deep learning with continuous b-spline kernels. CoRR, abs/1711.08920, 2017. URL http://arxiv.org/abs/1711.08920.
  23. Objectfolder: A dataset of objects with implicit visual, auditory, and tactile representations. In CoRL, 2021.
  24. Objectfolder 2.0: A multisensory object dataset for sim2real transfer, 2022. URL https://arxiv.org/abs/2204.02389.
  25. Kurt Hornik. Approximation capabilities of multilayer feedforward networks. Neural networks, 4(2):251–257, 1991.
  26. Convolutional neural networks on non-uniform geometrical signals using euclidean spectral transformation, 2019. URL https://arxiv.org/abs/1901.02070.
  27. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023. URL https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/.
  28. Learning operators with coupled attention, 2022.
  29. On universal approximation and error bounds for fourier neural operators. Journal of Machine Learning Research, 22:Art–No, 2021a.
  30. Neural operator: Learning maps between function spaces. arXiv preprint arXiv:2108.08481, 2021b.
  31. Panoptic neural fields: A semantic object-aware neural scene representation, 2022. URL https://arxiv.org/abs/2205.04334.
  32. Error estimates for DeepONets: a deep learning framework in infinite dimensions. Transactions of Mathematics and Its Applications, 6(1), 03 2022. ISSN 2398-4945. doi: 10.1093/imatrm/tnac001. URL https://doi.org/10.1093/imatrm/tnac001. tnac001.
  33. Hyperdeeponet: learning operator with complex target function space using the limited resources via hypernetwork. In The Eleventh International Conference on Learning Representations, 2022.
  34. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020a.
  35. Neural operator: Graph kernel network for partial differential equations, 2020b.
  36. A convnet for the 2020s. CoRR, abs/2201.03545, 2022. URL https://arxiv.org/abs/2201.03545.
  37. Fixing weight decay regularization in adam. CoRR, abs/1711.05101, 2017. URL http://arxiv.org/abs/1711.05101.
  38. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218–229, mar 2021a. doi: 10.1038/s42256-021-00302-5. URL https://doi.org/10.1038%2Fs42256-021-00302-5.
  39. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218–229, 2021b.
  40. The expressive power of neural networks: A view from the width. Advances in neural information processing systems, 30, 2017.
  41. On the benefits of invariance in neural networks. CoRR, abs/2005.00178, 2020. URL https://arxiv.org/abs/2005.00178.
  42. Acorn: Adaptive coordinate networks for neural scene representation. arXiv preprint arXiv:2105.02788, 2021.
  43. Modulated periodic activations for generalizable local functional representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  14214–14223, 2021.
  44. Occupancy networks: Learning 3d reconstruction in function space. CoRR, abs/1812.03828, 2018. URL http://arxiv.org/abs/1812.03828.
  45. Nerf: Representing scenes as neural radiance fields for view synthesis. CoRR, abs/2003.08934, 2020. URL https://arxiv.org/abs/2003.08934.
  46. TorchKbNufft: A high-level, hardware-agnostic non-uniform fast Fourier transform. In ISMRM Workshop on Data Sampling & Image Reconstruction, 2020.
  47. Instant neural graphics primitives with a multiresolution hash encoding. arXiv:2201.05989, January 2022.
  48. Autorf: Learning 3d object radiance fields from single view observations, 2022. URL https://arxiv.org/abs/2204.03593.
  49. Occupancy flow: 4d reconstruction by learning particle dynamics. In Proceedings of the IEEE/CVF international conference on computer vision, pp.  5379–5389, 2019.
  50. Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020.
  51. Deepsdf: Learning continuous signed distance functions for shape representation. CoRR, abs/1901.05103, 2019. URL http://arxiv.org/abs/1901.05103.
  52. Variable-input deep operator networks, 2022.
  53. Pointnet++: Deep hierarchical feature learning on point sets in a metric space, 2017. URL https://arxiv.org/abs/1706.02413.
  54. Flexconv: Continuous kernel convolutions with differentiable kernel sizes. CoRR, abs/2110.08059, 2021a. URL https://arxiv.org/abs/2110.08059.
  55. Ckconv: Continuous kernel convolution for sequential data. CoRR, abs/2102.02611, 2021b. URL https://arxiv.org/abs/2102.02611.
  56. Nomad: Nonlinear manifold decoders for operator learning, 2022.
  57. Diffusionnet: Discretization agnostic learning on surfaces. ACM Transactions on Graphics (TOG), 41(3):1–16, 2022.
  58. Scene representation networks: Continuous 3d-structure-aware neural scene representations. CoRR, abs/1906.01618, 2019. URL http://arxiv.org/abs/1906.01618.
  59. Metasdf: Meta-learning signed distance functions. CoRR, abs/2006.09662, 2020a. URL https://arxiv.org/abs/2006.09662.
  60. Implicit neural representations with periodic activation functions. CoRR, abs/2006.09661, 2020b. URL https://arxiv.org/abs/2006.09661.
  61. Light field networks: Neural scene representations with single-evaluation rendering. CoRR, abs/2106.02634, 2021. URL https://arxiv.org/abs/2106.02634.
  62. Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction, 2021. URL https://arxiv.org/abs/2111.11215.
  63. Variable bitrate neural fields. In ACM SIGGRAPH 2022 Conference Proceedings, pp.  1–9, 2022.
  64. Efficientnet: Rethinking model scaling for convolutional neural networks. CoRR, abs/1905.11946, 2019. URL http://arxiv.org/abs/1905.11946.
  65. Learned initializations for optimizing coordinate-based neural representations. CoRR, abs/2012.02189, 2020a. URL https://arxiv.org/abs/2012.02189.
  66. Fourier features let networks learn high frequency functions in low dimensional domains. CoRR, abs/2006.10739, 2020b. URL https://arxiv.org/abs/2006.10739.
  67. Lagrangian fluid simulation with continuous convolutions. In International Conference on Learning Representations, 2019.
  68. Nesf: Neural semantic fields for generalizable semantic segmentation of 3d scenes, 2021.
  69. Deep parametric continuous convolutional neural networks. CoRR, abs/2101.06742, 2021. URL https://arxiv.org/abs/2101.06742.
  70. Signal processing for implicit neural representations, 2022. URL https://arxiv.org/abs/2210.08772.
  71. Dmitry Yarotsky. Universal approximations of invariant maps by neural networks. Constructive Approximation, 55(1):407–474, 2022.
  72. PlenOctrees for real-time rendering of neural radiance fields. In ICCV, 2021.
  73. Deep sets. CoRR, abs/1703.06114, 2017. URL http://arxiv.org/abs/1703.06114.
  74. A functional perspective on learning symmetric functions with neural networks. In International Conference on Machine Learning, pp. 13023–13032. PMLR, 2021.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - clintonjwang/DI-net (6 stars)