Free-Range Gaussians: Non-Grid-Aligned Generative 3D Gaussian Reconstruction

Abstract: We present Free-Range Gaussians, a multi-view reconstruction method that predicts non-pixel, non-voxel-aligned 3D Gaussians from as few as four images. This is done through flow matching over Gaussian parameters. Our generative formulation of reconstruction allows the model to be supervised with non-grid-aligned 3D data, and enables it to synthesize plausible content in unobserved regions. Thus, it improves on prior methods that produce highly redundant grid-aligned Gaussians, and suffer from holes or blurry conditional means in unobserved regions. To handle the number of Gaussians needed for high-quality results, we introduce a hierarchical patching scheme to group spatially related Gaussians into joint transformer tokens, halving the sequence length while preserving structure. We further propose a timestep-weighted rendering loss during training, and photometric gradient guidance and classifier-free guidance at inference to improve fidelity. Experiments on Objaverse and Google Scanned Objects show consistent improvements over pixel and voxel-aligned methods while using significantly fewer Gaussians, with large gains when input views leave parts of the object unobserved.

• The paper introduces a unified generative framework that uses flow matching on non-grid-aligned 3D Gaussian parameters to achieve high-fidelity reconstruction with significantly fewer Gaussians.
• It employs a hierarchical patchification strategy with level-of-detail slicing to reduce sequence lengths and enable computationally efficient transformer-based denoising.
• Empirical evaluations show that the method outperforms traditional grid-aligned approaches under both full and partial observation scenarios, delivering sharper and more complete reconstructions.

Free-Range Gaussians: Non-Grid-Aligned Generative 3D Gaussian Reconstruction

Introduction

"Free-Range Gaussians: Non-Grid-Aligned Generative 3D Gaussian Reconstruction" (2604.04874) presents a unified generative framework for reconstructing 3D objects from sparse views using non-grid-aligned 3D Gaussians. The approach leverages flow matching on Gaussian parameters, enabling both high input fidelity and the synthesis of plausible content in unobserved regions. By decoupling the output representation from pixel and voxel grids, the method can utilize significantly fewer Gaussians than grid-aligned approaches without sacrificing reconstruction quality. Several architectural innovations are introduced, particularly a hierarchical patchification strategy and reconstruction-guided flow matching, making the model computationally efficient and scalable to high-fidelity reconstructions.

Figure 1: Non-grid-aligned generative reconstruction from four sparse views delivers a complete 3D Gaussian via flow matching, achieving high PSNR and low FID with $\sim$10$\times$ fewer Gaussians.

Non-Grid-Aligned Generative Reconstruction

Conventional 3D Gaussian Splatting (3DGS) feed-forward models typically anchor predicted Gaussians to pixels or voxels in one or more input views, resulting in dense, redundant, and grid-aligned Gaussian sets. Such methods struggle with partial observations, yielding holes and blurry reconstructions in unobserved regions due to conditional mean regression. By formulating reconstruction as a generative process through flow matching, the presented method learns to sample from the conditional distribution $P(\mathcal{G}|\mathcal{I})$ of 3D Gaussian parameters given sparse observations. This enables the model to synthesize sharp and perceptually realistic completions in unobserved areas, disentangling geometry from both image and voxel grids.

Grid-free supervision is enabled by interpolating between ground-truth Gaussian parameters and noise during training, thus bypassing permutation invariance or computationally expensive set-matching losses. The generative sampling process begins with noise and employs a DiT-based transformer to iteratively denoise towards Gaussian sets consistent with the input views. During test-time, the system can place Gaussians anywhere in space, optimizing spatial economy and coverage.

Figure 3: Pipeline overview—including hierarchical token patchification, flow-matching conditioning, and guided denoising—for efficient non-grid-aligned 3D Gaussian generation.

Hierarchical Patchification with Level-of-Detail

A fundamental challenge in generative 3D Gaussian modeling is scalability: typical high-fidelity object reconstructions require on the order of 50,000+ Gaussians, for which quadratic complexity self-attention is prohibitive. The paper introduces a hierarchical binary tree over the Gaussian parameters, from which arbitrary level-of-detail (LoD) slices can be extracted for coarse-to-fine training and token grouping. At any tree level, pairs of sibling nodes (spatially proximate) are concatenated into single transformer tokens, halving the effective sequence length with minimal information loss.

Figure 2: Hierarchical patchification with binary trees over Gaussian sets provides LoD slicing and patch tokenization, reducing sequence lengths while preserving local information.

This patchification both reduces computational burden and introduces a spatial locality prior into the transformer’s attention mechanism. The design ensures that the transformer is agnostic to grid structures and sequence length, enabling flexible adaptation to different token budgets and Gaussian counts at inference or during fine-tuning. The combination of LoD curriculum (progressive training from 2K to 8K Gaussian sets) and patchification is empirically shown to be critical for scaling generative 3D Gaussian models.

Reconstruction-Guided Flow Matching

To address the signal-to-noise trade-off in generative supervision, especially with coarse LoD slices and the inherent stochasticity of diffusion-based approaches, the method introduces three critical guidance mechanisms:

• Timestep-Weighted Multi-View Photometric Loss: The photometric (L1) loss is applied to rendered predictions with a monotonically increasing timestep-dependent weight, sharpening supervision at late stages of the denoising process.
• Photometric Gradient Guidance: Gradients of the rendering loss with respect to each Gaussian parameter are computed, concatenated to the model input at every denoising step, and used for direct guidance during integration.
• Classifier-Free Guidance: During inference, the model blends conditional and unconditional predictions, amplifying the effect of the observed input image features and improving alignment with the posed views.

Ablation studies demonstrate that all three guidance strategies contribute significantly to quantitative and qualitative performance, with flow matching, multiview photometric loss, and LoD training/patchification yielding the largest single ablation drops.

Figure 6: Ablations demonstrate that generative modeling, timestep-weighted photometric loss, and hierarchical curriculum are the principal contributors to performance and detail.

Experimental Evaluation

Quantitative results on both Objaverse and Google Scanned Objects (GSO) validate the method's effectiveness under both full and partial observation conditions. When input views provide comprehensive coverage, Free-Range Gaussians matches fidelity and perceptual quality of state-of-the-art grid-aligned approaches, but with a significant reduction in total Gaussian count. Under partial observation, when large portions of the object are unobserved, the method outperforms all baselines across PSNR, DINO, DreamSim, and FID metrics—producing sharper and more plausible completions and reducing grid-based redundancy by a factor of 5$\times$--10$\times$.

Figure 4: Qualitative comparisons on the Objaverse dataset highlight hole-filling and consistency advantages over pixel- and voxel-aligned baselines.

Figure 7: Qualitative comparisons on the GSO dataset further corroborate the method’s superior input fidelity and global completeness over pixel-aligned, generative, and volumetric approaches.

Theoretical and Practical Implications

The proposed generative framework shifts the paradigm of sparse-view 3D reconstruction from regression-based and grid-anchored supervision toward direct modeling of conditional distributions in unstructured parameter spaces. This unlocks several theoretical advantages: grid invariance, reduced overfitting to viewpoint arrangements, and improved handling of geometric ambiguities. Practically, the method’s compact representations—and its ability to hallucinate unseen structure consistent with input views—are particularly beneficial for scenarios with limited images, long-tailed object categories, on-device deployment, and interactive editing applications.

Limitations center around the Gaussian token budget and the iterative (multi-step) sampling process. Scaling to fine geometric detail will likely require integration of sparse or linear complexity attention mechanisms, or post-hoc upsampling stages leveraging optimization or super-resolution modules. Inference speed, dictated by the denoising trajectory, could be addressed with progressive distillation or rectified flow straightening.

Conclusion

Free-Range Gaussians establishes non-grid-aligned flow matching as a scalable and effective foundation for sparse-view 3D object reconstruction. By leveraging a hierarchical patching scheme, guided generative supervision, and transformer-based denoising, the approach yields compact, complete, and high-fidelity 3D reconstructions from as few as four input images. Performance gains are most pronounced in the challenging partial-observation regime and persist with an order-of-magnitude Gaussian budget reduction. The generality of the formulation and its computational efficiency suggest broad impact for 3D generative modeling and future research on scalable, grid-free geometric representations (2604.04874).