Gaussian RBFNet: Gaussian Radial Basis Functions for Fast and Accurate Representation and Reconstruction of Neural Fields (2503.06762v1)

Abstract: Neural fields such as DeepSDF and Neural Radiance Fields have recently revolutionized novel-view synthesis and 3D reconstruction from RGB images and videos. However, achieving high-quality representation, reconstruction, and rendering requires deep neural networks, which are slow to train and evaluate. Although several acceleration techniques have been proposed, they often trade off speed for memory. Gaussian splatting-based methods, on the other hand, accelerate the rendering time but remain costly in terms of training speed and memory needed to store the parameters of a large number of Gaussians. In this paper, we introduce a novel neural representation that is fast, both at training and inference times, and lightweight. Our key observation is that the neurons used in traditional MLPs perform simple computations (a dot product followed by ReLU activation) and thus one needs to use either wide and deep MLPs or high-resolution and high-dimensional feature grids to parameterize complex nonlinear functions. We show in this paper that by replacing traditional neurons with Radial Basis Function (RBF) kernels, one can achieve highly accurate representation of 2D (RGB images), 3D (geometry), and 5D (radiance fields) signals with just a single layer of such neurons. The representation is highly parallelizable, operates on low-resolution feature grids, and is compact and memory-efficient. We demonstrate that the proposed novel representation can be trained for 3D geometry representation in less than 15 seconds and for novel view synthesis in less than 15 mins. At runtime, it can synthesize novel views at more than 60 fps without sacrificing quality.

• The paper introduces Gaussian RBFNet, which replaces traditional MLPs with a single layer of Gaussian Radial Basis Functions for faster, more efficient neural field representation and reconstruction.
• Gaussian RBFNet achieves significant performance improvements, training 3D geometry in under 15 seconds and enabling real-time rendering above 60 fps, while maintaining competitive accuracy across tasks.
• This method offers a scalable approach for representing neural fields, with implications for applications in virtual reality, computer graphics, and advanced vision systems requiring rapid processing.

Gaussian RBFNet: Gaussian Radial Basis Functions for Fast and Accurate Representation and Reconstruction of Neural Fields

The paper presents "Gaussian RBFNet," an innovative approach employing Gaussian Radial Basis Function (RBF) networks for efficient representation and reconstruction of neural fields, particularly in the domains of 2D images, 3D geometry, and 5D radiance fields. This paper addresses the well-documented computational and memory challenges associated with state-of-the-art models like DeepSDF and Neural Radiance Fields (NeRF) by proposing a compact, fast-training solution with minimal trade-offs in accuracy and computational efficiency.

Methodological Advancements

The core contribution of the research lies in replacing traditional multi-layer perceptrons (MLPs) used in neural fields with a single-layer network of Radial Basis Functions with Gaussian kernels. Gaussian RBFNet capitalizes on the simplicity of RBF computations to handle complex nonlinear functions, thus simplifying the overall architecture and enhancing computational efficiency without sacrificing quality. Through this approach, the authors demonstrate significant improvements in training times and inference speeds.

• Input Encoding and Feature Decoding: The paper utilizes low-resolution feature grids, employing Gaussian RBFs to decode these into high-quality signal outputs. This framework bridges the gap between discrete local features and continuous mappings critical for neural fields.
• Performance Metrics: The authors highlight the ability of the Gaussian RBFNet to handle multiple-dimensional signals with reduced computational resources, demonstrating the ability to train 3D geometry representations in less than 15 seconds, with real-time synthesis rendering rates exceeding 60 fps.

Experimental Validation

The experimental setup evaluates various structural complexities to validate the efficacy of Gaussian RBFNet across diverse scenarios. The results underscore a marked reduction in training time (outperforming InstantNGP by a margin) while maintaining competitive levels of accuracy for representation tasks critical in both image and geometry processing.

• 3D Geometry Representation: Experiments conducted on canonical datasets reveal the robustness of Gaussian RBFNet in efficiently encoding signed distance functions (SDFs) with fewer RBF kernels than traditional methods. Notably, it achieves the best results in Chamfer Distance and Normal Consistency metrics compared to comparable methods like InstantNGP and FactorFields.
• Radiance Field Representation: For radiance fields, a pivotal component for novel view synthesis, Gaussian RBFNet closely matches InstantNGP's efficiency while surpassing it in more nuanced quality assessments, showing that the elegant handling of radiance fields can achieve high fidelity with less computational overhead.

Implications and Future Prospects

This paper's findings suggest a feasible path to more scalable neural field representations, spotlighting Gaussian RBFs as efficient feature decoders in the neural rendering landscape. The implications extend to applications in virtual reality, computer graphics, and advanced vision systems relying on rapid rendering and processing of complex visual data.

Looking forward, potential developments may explore integrating this approach with domain-specific refinements or adaptations to dynamic and unbounded scene representations. The methodology's extension to handle temporal changes or view-dependent effects effectively could also yield substantial advancements in real-time, multidimensional representations and facilitate broader adoption across computational imaging tasks. Overall, Gaussian RBFNet offers a promising step toward more efficient, scalable architectural designs in the broader scope of AI-driven visual computing technologies.