FieryGS: Fire Synthesis in 3DGS Pipeline

• FieryGS is a physically-based framework that unifies scene reconstruction, multimodal material reasoning, combustion simulation, and rendering within a 3DGS pipeline.
• The system employs grid-based volumetric combustion and charring models, using PDE techniques for simulating flames, smoke, and fire propagation.
• Experimental results demonstrate FieryGS delivers superior physical plausibility and temporal coherence compared to traditional CFD and static 3D editing workflows.

Searching arXiv for the specified paper and closely related context. FieryGS is a physically-based framework for synthesizing photorealistic, physically plausible combustion effects in reconstructed real scenes by integrating combustion simulation and rendering within a 3D Gaussian Splatting pipeline. It is presented in "FieryGS: In-the-Wild Fire Synthesis with Physics-Integrated Gaussian Splatting" (Shen et al., 30 Apr 2026). The method targets the gap between traditional CFD or graphics/VFX workflows, which depend on handcrafted geometry, exhaustive material annotation, scene discretization, and expert parameter tuning, and standard NeRF/3DGS approaches, which reconstruct static appearance efficiently but do not infer combustion-relevant materials or couple to a physics-based fire simulator. FieryGS unifies scene reconstruction, multimodal material reasoning, volumetric combustion simulation, and a renderer that jointly handles flames, smoke, and 3D Gaussian Splatting, with support for flame propagation, smoke dispersion, surface carbonization, and user control over ignition, airflow, and fire intensity.

1. Problem setting and system definition

FieryGS addresses the problem of synthesizing photorealistic, physically plausible fire effects directly inside reconstructed real scenes. The motivating claim is that traditional CFD and graphics pipelines can produce realistic fire effects, but rely on handcrafted geometry, expert-tuned parameters, and labor-intensive workflows, which limits scalability to in-the-wild scenes. The same account identifies a pronounced sim-to-real gap in such pipelines, while large video models can overlay fire but lack physical consistency and precise control. Standard 3D Gaussian Splatting methods provide high-fidelity real-world scene reconstruction, yet do not include physical grounding for combustion (Shen et al., 30 Apr 2026).

The framework is defined by three tightly coupled components: multimodal large-language-model-based physical material reasoning, efficient volumetric combustion simulation, and a unified renderer for fire and 3DGS. In the formulation given for the system, inputs consist of multi-view images of a real scene and a reconstructed 3DGS with appearance, normals, and depth obtained via PGSR together with semantic and material cues inferred by an MLLM. Intermediate representations include 3D Gaussians with centers, covariances, opacities, view-dependent color represented by spherical harmonics, and learned features $f_g$ per splat; a voxel occupancy grid distinguishing air from solid and annotating combustible labels; and volumetric fields comprising divergence-free fluid velocity $u$, reaction coordinate $Y$ in air, material temperature $T_m$ in solids, and char mass $M_c$ in solids. The outputs are rendered frames with fire emission, smoke absorption, 3DGS appearance with charring, and fire illumination, with optional generative refinement video.

The paper characterizes its main contribution as the first integration of physically-based combustion simulation and rendering inside a 3DGS pipeline for real scenes, combined with automatic material inference via MLLM and coherent coupling to scene geometry. A further contribution is a unified volumetric renderer that composites simulated fire and smoke with 3DGS while incorporating charring and illumination. A practical implication is that the system is intended to avoid expert-only workflows by automatically initializing combustible regions and material properties from scene reconstruction and material reasoning.

2. Pipeline architecture and data flow

The FieryGS pipeline is organized as a sequence from reconstruction to segmentation, material reasoning, simulation, rendering, and user interaction (Shen et al., 30 Apr 2026). Scene reconstruction begins with PGSR to obtain anisotropic Gaussians together with normals and depth. These quantities serve not only the 3DGS representation itself but also later visibility analysis and illumination.

For 3D Gaussian segmentation, each Gaussian is assigned a trainable feature vector $f_g \in \mathbb{R}^D$. Features are rendered to 2D through 3DGS alpha blending, and SAM is applied across views to segment 2D frames. The features are then trained with contrastive learning so that pixels within the same SAM mask share embeddings. The resulting Gaussian features are clustered into 3D regions with HDBSCAN, yielding instance-level segments. The reported fixed hyperparameters across scenes are minimum cluster size 10, epsilon 0.01, and hierarchical clustering at affinity feature scales 0.9, 0.5, and 0.1.

Material reasoning is then performed segment by segment. For each 3D segment, the system chooses the viewpoint with highest visibility by counting unoccluded Gaussians via rendered depth. The selected region is rasterized to 2D and provided to GPT-4o with three visual prompts: a full-scene rendering, the same rendering with the target region highlighted by bounding box and mask overlay, and an isolated zoomed-in render of the segment. The text prompt asks for step-by-step reasoning, including region captioning, material selection from a predefined library, and inference of combustion attributes such as burnability, thermal diffusivity, smoke color, and possibly ignition threshold. Predicted attributes are assigned back to all Gaussians in the segment.

An occupancy grid binds reconstruction to simulation. A voxel is marked occupied if it is overlapped by one or more Gaussians with opacity above a threshold, and combustible if any overlapped Gaussian is predicted burnable. Unoccupied voxels are treated as air and occupied voxels as solid. This occupancy grid defines the simulation domain and supports the coupling between gaseous fire dynamics and solid charring.

The stated workflow culminates in rendering. Fire emission is rendered spectrally using blackbody radiation, smoke is rendered as absorption under participating-media assumptions, charring is integrated into 3DGS colors, and Phong illumination is produced from hot voxels. The paper also includes an optional enhancement pass based on Wan2.1, applied after simulation and rendering to enrich high-frequency effects such as flicker and illumination detail.

3. Multimodal material reasoning and scene initialization

Material reasoning in FieryGS is explicitly zero-shot and MLLM-based. Its purpose is to infer combustion-relevant physical properties directly from 3DGS reconstructions rather than requiring manual annotation. The segmentation stage provides the 3D parts on which reasoning operates, and the viewpoint-selection step ensures that each segment is queried from the camera with maximum visibility (Shen et al., 30 Apr 2026).

The prompt design combines visual and textual conditioning. The visual inputs establish both global context and local identity of a segment, while the text prompt asks the model to caption the region, choose a material from a predefined library, and infer combustion attributes. The attributes named in the description include burnability, thermal diffusivity $\beta$, smoke color, and possibly ignition threshold $T_{\text{ign}}$. These predictions are projected back to 3D by directly assigning them to all Gaussians in the segment.

The reported cost and accuracy figures are specific. The system averages 84 GPT-4o calls per scene, with a range from 9 to 209 calls, and an average cost of about \$0.55 per scene under ChatGPT-4o-Latest pricing with approximately 1,282 input tokens and 18 output tokens per call. The average material reasoning accuracy across six scenes is 89.31%.

These measurements situate the reasoning module as a practical scene-initialization mechanism rather than a manually curated preprocessing stage. A plausible implication is that the framework treats semantic segmentation and physical property attribution as part of the end-to-end authoring pipeline, but gradients do not flow through the physics modules; the simulation and rendering are used forwardly.

4. Volumetric combustion and charring model

The combustion simulation is an incompressible, grid-based volumetric model operating on a $256 \times 256 \times 256$ grid with all fields stored at cell centers (Shen et al., 30 Apr 2026). In air, the governing equations are given as

$$\partial u/\partial t + u \cdot \nabla u = -(1/\rho)\nabla p + f,\qquad \nabla \cdot u = 0;$$

$u$0

Here $u$1 is divergence-free velocity, $u$2 is density, $u$3 is pressure, $u$4 is the reaction coordinate, and $u$5 is the reaction rate. The interpretation stated in the paper is that $u$6 denotes burning and $u$7 unburnt. Initialization sets $u$8 in all voxels and $u$9 only at user-specified ignition voxels that are combustible.

The choice of incompressible flow is motivated as being more efficient than compressible alternatives. External forces include buoyancy and vorticity confinement, with buoyancy specified as

$Y$0

and vorticity confinement following Nguyen et al. 2002. Rather than solving a full thermal PDE for the gaseous phase, local temperature $Y$1 is approximated as a quadratic function of $Y$2.

Time integration follows operator splitting in the Stable Fluids style: semi-Lagrangian advection, external-force addition, reaction, and pressure projection. The explicit update skeleton is

$Y$3

followed by

$Y$4

and then solution of

$Y$5

by Gauss-Seidel iteration with the velocity projection

$Y$6

Boundary conditions are open at the simulation box. Obstacles use an open boundary if velocity points outward and a no-through Neumann condition when velocity points inward, allowing fluid to exit obstacles while preventing flow into them.

The solid model governs heat transfer and charring. Material temperature evolves according to

$Y$7

where $Y$8 is thermal diffusivity, $Y$9 is a radiative cooling coefficient, and $T_m$0 approximates internal heat generation by clamping $T_m$1 to $T_m$2 once $T_m$3 is exceeded. Char accumulation is defined by

$T_m$4

where $T_m$5 is relative char mass, $T_m$6 is charring rate, and $T_m$7 if $T_m$8 and $T_m$9 otherwise. Each 3D Gaussian inherits $M_c$0 from its containing voxel. Thermal diffusion, described as a Poisson operator, uses smaller sub-steps inside $M_c$1 for stability.

The model supports inter-object fire propagation when adjacent combustible solids heat each other through thermal diffusion. The paper specifically notes that inter-object propagation is demonstrated when solids contact. This suggests that the simplified solid-phase model is central not only to carbonization but also to spatially extended ignition behavior.

5. Unified rendering of flames, smoke, illumination, and carbonization

FieryGS combines volumetric fire and smoke rendering with conventional 3DGS rendering in a single composition model (Shen et al., 30 Apr 2026). The 3DGS preliminaries are stated in terms of anisotropic Gaussians parameterized by center, covariance, opacity, and view-dependent color via spherical harmonics. During rendering, a Gaussian is projected into screen space with covariance

$M_c$2

where $M_c$3 is the camera extrinsic matrix and $M_c$4 is the Jacobian of the projection, and pixel colors are obtained by alpha blending over depth-sorted Gaussians.

The rendering assumptions are explicit: fire is treated as a blackbody radiator with negligible scattering, smoke is low-albedo, 3DGS is rendered as an opaque background, charring affects 3DGS colors, and fire illumination is modeled by Phong shading from hot voxels. The final pixel radiance is written exactly as

$M_c$5

The accompanying explanation states that $M_c$6 and $M_c$7 are accumulated along the ray before reaching the 3DGS, $M_c$8 is transmittance, $M_c$9 is 3DGS radiance with charring, and $f_g \in \mathbb{R}^D$0 is fire illumination. Scattering terms are omitted.

Fire emission is based on Planck’s blackbody radiation law:

$f_g \in \mathbb{R}^D$1

reproduced as written in the paper. The spectral-to-RGB pipeline converts spectral radiance to CIE XYZ via tristimulus curves, then to LMS using $f_g \in \mathbb{R}^D$2, applies chromatic adaptation based on maximum fire temperature, converts back to RGB, and applies gamma correction, following Nguyen et al. 2002 and Pegoraro & Parker 2006. The absorption coefficient $f_g \in \mathbb{R}^D$3 is fixed positive when $f_g \in \mathbb{R}^D$4 and zero otherwise. Sampling is coarse-to-fine along each ray, using 128 uniform samples and then 1024 importance samples based on $f_g \in \mathbb{R}^D$5.

Smoke is rendered when the reaction coordinate satisfies $f_g \in \mathbb{R}^D$6. Its color is determined by the inferred burning material, with the examples wood $f_g \in \mathbb{R}^D$7 white and plastic $f_g \in \mathbb{R}^D$8 black. Smoke is composed in the same volume-rendering pass as fire.

Charring modifies the appearance of the 3DGS background. For charred Gaussians, the paper specifies that color is dimmed if $f_g \in \mathbb{R}^D$9 by a scaling factor written exactly as

$\beta$0

where $\beta$1 controls the final dimming at $\beta$2. The intended behavior is progressive darkening as char mass increases.

Illumination from fire onto scene surfaces is approximated by treating sufficiently hot voxels as volumetric light sources. For each 3D Gaussian, only diffuse and specular terms are computed:

$\beta$3

Here $\beta$4 is the spectral radiance emitted by voxel $\beta$5, $\beta$6 is the surface normal from 3DGS, $\beta$7 is the light direction, $\beta$8 is the view direction, $\beta$9 is the reflection direction, and $T_{\text{ign}}$0, $T_{\text{ign}}$1, and $T_{\text{ign}}$2 are diffuse coefficient, specular coefficient, and specular exponent. The accumulated spectral radiance is converted to RGB through the same spectral pipeline as fire rendering, and ACES tone mapping is applied to bring RGB values into $T_{\text{ign}}$3.

An optional refinement stage uses the Wan2.1 diffusion video model with classifier-free guidance, inspired by SDEdit and PhysGen. The simulated video is encoded into latent space, perturbed with noise, and denoised conditioned on the first frame to add high-frequency detail, flicker, and richer illumination. The paper also notes that this optional stage may alter background content or break 3D consistency.

6. Controllability, implementation, and quantitative evaluation

FieryGS exposes physically meaningful controls at the simulation level (Shen et al., 30 Apr 2026). Ignition is controlled by setting $T_{\text{ign}}$4 at chosen ignition voxels, provided they are combustible according to the material reasoning stage. Fire intensity is controlled by increasing the buoyancy coefficient $T_{\text{ign}}$5 to lift flames and decreasing the reaction rate $T_{\text{ign}}$6 to extend flame visibility. Airflow is controlled by adding an external wind force to steer flame and smoke direction. Material and carbonization behavior can be altered by editing thermal diffusivity $T_{\text{ign}}$7 and charring rate $T_{\text{ign}}$8. The paper states that all combustion parameters are accessible for fine-tuning. In the Garden scene, different ignition locations and variants with intensified fire or added rightward wind produce the expected semantically meaningful changes.

Implementation details are also specified. Simulation is implemented in Taichi, while rendering of fire, smoke, 3DGS composition, and the spectral pipeline is implemented from scratch in PyTorch. The reconstruction stage uses PGSR, and the pipeline depends on SAM for 2D segmentation and GPT-4o for multimodal material reasoning. Optional generative refinement depends on Wan2.1.

The experimental evaluation uses six real-world scenes: Firewood, Kitchen, Chair, and Stool captured with iPhone, together with Garden from MipNeRF360 and Playground from Tanks & Temples. Baselines are AutoVFX, described as Blender physics with LLM scripts and dynamic edits in 3DGS; Runway-V2V, described as commercial video-to-video synthesis; and Instruct-GS2GS, described as text-driven static 3DGS editing.

The reported quantitative results are:

Method	Aesthetic Quality	Imaging Quality	DINO Structure
AutoVFX	0.488	0.603	1.04
Runway-V2V	0.605	0.701	0.68
Instruct-GS2GS	0.451	0.394	0.66
Ours	0.624	0.702	0.38

The metrics are VBench Aesthetic Quality and Imaging Quality, where higher is better, and DINO Structure Score, where lower is better.

User studies were conducted on Amazon Mechanical Turk using A/B pairwise comparison with 31 pairs. For perceptual realism, FieryGS was preferred over AutoVFX by 88.9% on images and 77.8% on video, over Runway-V2V by 79.4% and 66.5%, and over Instruct-GS2GS by 85.5% and 63.0%. For physical plausibility, preferences were 86.6% and 85.5% against AutoVFX, 85.3% and 79.0% against Runway-V2V, and 83.2% and 84.5% against Instruct-GS2GS.

Efficiency is reported for an NVIDIA RTX 4090D. Average per-frame runtime is 2.37 s, broken down into 1.52 s for simulation, 0.012 s for GS render, and 0.84 s for fire and smoke render. Peak GPU memory is below 10 GB. AutoVFX is reported to require 4 to 10 minutes per frame, while Runway-V2V is closed-source with an official generation time of about 30 s for 10 s of video, and Instruct-GS2GS is fast but limited to static edits.

7. Limitations, failure modes, and research significance

The paper states several limitations explicitly (Shen et al., 30 Apr 2026). For efficiency, FieryGS does not simulate mass loss or thermal degradation such as shrinkage and disintegration. The flame model is simplified, and large-scale conflagrations such as forests or buildings are not handled. Non-uniform 3DGS point distribution can introduce artifacts in volumetric simulation and rendering. Material reasoning errors may occur in tiny, distant, or occluded regions or due to reconstruction artifacts, and such errors can lead to incorrect combustion behavior. The optional generative refinement can alter background content and lacks strong temporal or 3D consistency, with failures illustrated by altered reappearance of occluded textures.

The qualitative comparison to baselines clarifies the scope of these limitations. Runway-V2V is described as producing appealing flames but altering scene geometry and appearance, lacking ignition and spread dynamics, and not using material-aware smoke color. AutoVFX often yields implausible flames in complex scenes. Instruct-GS2GS provides static, coarse edits. FieryGS is instead characterized as temporally coherent, scene-preserving, and able to capture ignition, spread, smoke dispersion, illumination, and carbonization.

In a broader methodological sense, FieryGS defines an overview pipeline in which reconstruction, semantic and physical attribution, PDE-based simulation, and physically-based rendering are integrated around a 3DGS representation. This suggests a direction in which 3D scene representations are not merely passive carriers of appearance, but substrates for physically grounded dynamic editing. Within the specific scope established by the paper, the framework’s significance lies in replacing manual scene modeling and expert tuning with automatic initialization from scene reconstruction and material reasoning while retaining explicit control over combustion parameters.