Learning Spatially Collaged Fourier Bases for Implicit Neural Representation (2312.17018v1)
Abstract: Existing approaches to Implicit Neural Representation (INR) can be interpreted as a global scene representation via a linear combination of Fourier bases of different frequencies. However, such universal basis functions can limit the representation capability in local regions where a specific component is unnecessary, resulting in unpleasant artifacts. To this end, we introduce a learnable spatial mask that effectively dispatches distinct Fourier bases into respective regions. This translates into collaging Fourier patches, thus enabling an accurate representation of complex signals. Comprehensive experiments demonstrate the superior reconstruction quality of the proposed approach over existing baselines across various INR tasks, including image fitting, video representation, and 3D shape representation. Our method outperforms all other baselines, improving the image fitting PSNR by over 3dB and 3D reconstruction to 98.81 IoU and 0.0011 Chamfer Distance.
- Learning implicit fields for generative shape modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5939–5948.
- trimesh.
- Multiplicative Fourier Level of Detail. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1808–1817.
- Eastman Kodak Company. 1999. Kodak lossless true color image suite. http://r0k.us/graphics/kodak/. [Accessed 14-08-2023].
- Multiplicative filter networks. In International Conference on Learning Representations.
- Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31.
- Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
- Bacon: Band-limited coordinate networks for multiscale scene representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16252–16262.
- Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4460–4470.
- Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1): 99–106.
- Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 165–174.
- PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, 8024–8035. Curran Associates, Inc.
- Wire: Wavelet implicit neural representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18507–18516.
- Residual multiplicative filter networks for multiscale reconstruction. Advances in Neural Information Processing Systems, 35: 8550–8563.
- Implicit neural representations with periodic activation functions. Advances in neural information processing systems, 33: 7462–7473.
- Scene representation networks: Continuous 3d-structure-aware neural scene representations. Advances in Neural Information Processing Systems, 32.
- Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems, 33: 7537–7547.
- Attention is all you need. Advances in neural information processing systems, 30.
- Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4): 600–612.
- A structured dictionary perspective on implicit neural representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19228–19238.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.