Gaussians on Fire: High-Frequency Reconstruction of Flames (2511.22459v1)

Abstract: We propose a method to reconstruct dynamic fire in 3D from a limited set of camera views with a Gaussian-based spatiotemporal representation. Capturing and reconstructing fire and its dynamics is highly challenging due to its volatile nature, transparent quality, and multitude of high-frequency features. Despite these challenges, we aim to reconstruct fire from only three views, which consequently requires solving for under-constrained geometry. We solve this by separating the static background from the dynamic fire region by combining dense multi-view stereo images with monocular depth priors. The fire is initialized as a 3D flow field, obtained by fusing per-view dense optical flow projections. To capture the high frequency features of fire, each 3D Gaussian encodes a lifetime and linear velocity to match the dense optical flow. To ensure sub-frame temporal alignment across cameras we employ a custom hardware synchronization pattern -- allowing us to reconstruct fire with affordable commodity hardware. Our quantitative and qualitative validations across numerous reconstruction experiments demonstrate robust performance for diverse and challenging real fire scenarios.

• The paper introduces an explicit Gaussian-based framework that segments static and dynamic flame regions for improved high-frequency 3D reconstructions.
• It employs dense optical flow with stereo and monocular depth cues to optimize dynamic flame modeling, achieving superior PSNR, SSIM, and RMSE performance over baseline methods.
• The method integrates precise temporal synchronization and rolling-shutter correction to mitigate geometric errors and capture fast, partially transparent flames accurately.

High-Frequency 3D Reconstruction of Real Flames via Spatiotemporal Gaussian Splatting

Introduction and Problem Statement

The paper "Gaussians on Fire: High-Frequency Reconstruction of Flames" (2511.22459) presents a methodology for reconstructing dynamic fire phenomena in 3D from sparse multiview video, using an explicit Gaussian-based space-time representation. Accurate 3D capture of fire is highly challenging due to the non-rigid nature, high temporal and spatial frequency, partially transparent and emissive appearance, and rapid motion of flames. Previous approaches leveraging dynamic NeRFs or tomographic methods often require dense view coverage, physics-based simulation priors, or are prohibitively slow for real scenes with only a few commodity cameras.

The authors propose separating the scene into static (background) and dynamic (fire) components, fusing depth cues for the static scene and leveraging dense optical flow to initialize dynamic 3D Gaussian primitives. Their system operates on monocular and stereo modalities, enables temporal coherence through sub-frame camera synchronization, and supports rendering of novel dynamic views of fire using as few as three relatively affordable, unsynchronized cameras.

Pipeline and Representational Framework

The method begins with synchronized video captured from three viewpoints. Each view is spatially registered and photometrically calibrated via COLMAP. Stereo depth is extracted with PatchMatchStereo; monocular depth priors are linearly aligned and used for regularization. A minimum intensity projection is computed over time sequences to remove fire from the static background, enhancing scene and camera calibration.

Figure 1: Overview of the reconstruction pipeline integrating calibrated multi-view video, stereo+monocular depth fusion for static backgrounds, and fused optical flow-based motion fields for initializing dynamic Gaussians.

Dynamic fire regions are identified and segmented. For each pixel in each camera, dense multi-frame optical flow (MEMFOF) estimates per-pixel motion, which is fused via 3D back-projection into a volumetric flow field. This field provides spatially and temporally coherent estimates of flame motion used to initialize the velocities and lifetimes of FreeTimeGS-based dynamic 3D Gaussians, encoding both radiance and dynamic trajectories. Each Gaussian primitive thus carries explicit position, color, temporal lifetime, and velocity.

Optimizing the static scene employs vanilla 3D Gaussian Splatting (3DGS) with strong stereo + monocular depth cues and a high-depth loss weighting to mitigate overfitting given limited view constraints. Dynamic Gaussians are initialized using voxelized flow and optimized using photometric losses and smoothness priors conditioned on velocity fields.

Figure 2: Intermediate modalities for scene reconstruction, including input frame, monocular and stereo depth predictions, dynamic mask, 2D optical flow, and final voxelized 3D flow field for dynamic initialization.

Background Extraction and Scene Calibration

When static frames are unavailable, minimum intensity projection (MIP) across the temporal dimension removes most of the dynamic fire, yielding a background suitable for traditional structure-from-motion pipelines and depth fusion.

Figure 3: Removal of dynamic fire via temporal minimum intensity projection for background scene recovery, shown for both small and large flames.

Volumetric Dynamic Flow and FreeTimeGS Parametrization

Dense 2D flows from synchronized cameras are projected into a 3D grid. Each grid cell estimates a 3-vector flow via optimization with Tikhonov regularization, incorporating geometric projection constraints. The velocity field $\vec{F}(\mathbf{x})$ directly initializes the FreeTimeGS primitives, which define, at each time $t$, the Gaussian center as $\mathbf{x}(t) = \mathbf{x}_0 + (t - t_\mu)\vec{v}$, with temporal opacity modulation for smooth blending.

Monocular depth predictions ($D_{\text{mono}}$) are aligned by least-squares to stereo estimates ($D_{\text{stereo}}$), and serve as a regularization term during static/dynamic optimization. The background Gaussians do not update during dynamic optimization to prevent temporal overfitting of the static regions.

Precise Camera Synchronization and Rolling-Shutter Modeling

Coherent multi-view 4D reconstruction of high-speed dynamic phenomena such as fire requires hardware-level temporal alignment. Given the unsynchronized nature of commodity action cameras, the authors develop a microcontroller-driven LED-based pattern for sub-frame and frame-level synchronization, detected visually using known LED switching and rolling-shutter artifacts, attaining microsecond accuracy.

The authors further analyze rolling-shutter distortion, which, if not accounted for, can lead to spatiotemporal inconsistencies, especially in dynamic flames. By modeling per-pixel exposure delay based on known sensor line readout times, dynamic Gaussians are attributed with temporal corrections during rendering, mitigating unwanted distortion artifacts.

Figure 4: Rolling-shutter effects illustrated in global versus rolling frame rendering, emphasizing temporal warping in high-speed motion.

Figure 5: Three-camera physical capture setup and custom LED synchronization system designed for frame- and sub-frame visual sync.

Dataset and Experimental Evaluation

The introduced dataset comprises 17 real fire scenes recorded at 400 Hz from three GoPro Hero 13 Black cameras in fixed multi-view arrangements, with varying fuel types (propane, wood, paper, gasoline, cardboard) and background conditions.

Figure 6: Multi-view and depth reconstruction for a scene with dynamic flames, confirming geometric consistency across wide appearance changes.

Qualitative renderings (Fig. 7) highlight the spatiotemporal fidelity across fuel types and temporal progression of flames.

Figure 7: Rendered test frames showing dynamic reconstructions of various fuel scenarios, demonstrating consistent geometry and motion.

Quantitatively, the method outperforms baseline dynamic Gaussian splatting approaches (4DGS [Wu_2024_CVPR] and GS4D [yang2023gs4d]), specifically in flame regions:

• PSNR (flame): 27.46 vs. 18.24 (Wu et al.) and 25.51 (Yang et al.)
• SSIM (flame): 0.8469 vs. 0.5218 and 0.8019
• RMSE (depth): 0.0412 vs. 0.2655 and 0.2749

The method yields both higher local visual metrics inside dynamic regions and markedly lower geometric error, indicating improved depth and structure plausibility in the reconstructed flame.

Figure 8: Qualitative comparison between this method and two leading 4D Gaussian Splatting baselines, exposing superior reconstruction of transparent and fine fire details, along with more plausible depth.

Ablation analysis shows that flow-based initialization, timing correction, and separated static/dynamic optimization are all critical for high-fidelity dynamic scene reconstruction; naive joint optimization and random flow initialization significantly degrade depth plausibility and PSNR/SSIM.

Discussion, Limitations, and Theoretical Implications

This work provides compelling evidence that explicit dynamic Gaussian splatting, with fused 3D flow and velocity-based parametrization (FreeTimeGS), can overcome the inherent ambiguity and under-constrained geometry of sparse multi-view dynamic capture. The system enables physically plausible, temporally coherent, high-resolution reconstructions with affordable hardware and minimal controlled capture requirements.

However, dependence on dense optical flow limits performance in regions of low texture or high emissive variance, which can induce inaccuracies in motion field initialization. The current assumption of linear per-Gaussian velocity does not fully capture turbulence, leading to potential temporal blurring. Additionally, explicit separation of static/dynamic optimization presumes limited background-fire interaction, possibly limiting extensibility to large-scale, smoke-rich, or occlusion-heavy scenes.

Practically, the pipeline supports applications in graphics, robotics (e.g., dynamic navigation near fire), and safety simulations, where photorealistic spatiotemporal 3D models of fire are valuable.

Theoretically, this approach advances understanding of dynamic scene representations, demonstrating the utility of physically interpretable flow and explicit temporally parametrized Gaussians over black-box MLP-based NeRF architectures for certain under-constrained, highly dynamic phenomena.

Future Directions

The paper identifies integration of physics-based priors for combustion, learning-based volumetric flow estimation, and further temporal modeling flexibility as directions to address remaining limits. Extension to monocular dynamic reconstruction, and joint static/dynamic optimization under complex environmental interactions, present ongoing challenges.

As the literature migrates towards unified, high-resolution, and semantically controllable 4D perception systems, the explicit, data-efficient, and interpretable properties demonstrated in this work offer a promising route for advancing both empirical performance and theoretical transparency.

Conclusion

"Gaussians on Fire" establishes a substantially improved method for high-frequency, multiview 3D reconstruction of dynamic flames using a flow-initialized, velocity-parametric dynamic splatting framework. The system achieves superior geometric and photometric accuracy over prior dynamic Gaussian approaches, using only three commodity, visually synchronized cameras. The implications extend towards more robust, data-efficient dynamic scene capture and reconstruction, stimulating advancement in explicit 4D modeling paradigms and their integration with learned and physics-guided priors.