Real-time Neural Six-way Lightmaps
This presentation explores a breakthrough in interactive volumetric rendering that enables real-time smoke, fog, and explosions in video games and VR. The research introduces a neural architecture that predicts six-way lightmaps from screen-space guiding maps, achieving photorealistic volumetric effects in under 4 milliseconds per frame while supporting dynamic cameras, arbitrary lighting, and interactive obstacles—capabilities impossible with traditional precomputed lightmaps.Script
Rendering realistic smoke in video games requires solving an equation so expensive that a single frame can take minutes. Yet gamers expect 60 frames per second. This paper cracks that paradox with a neural approach that delivers photorealistic volumetric effects in under 4 milliseconds.
The standard workaround precomputes lightmaps from six directions, trading flexibility for speed. But those lightmaps are locked in place. Move the camera, change the light, add an obstacle, and the illusion shatters.
The authors replace rigid precomputation with a neural network that predicts lightmaps on the fly.
The pipeline starts with a real-time fluid simulator outputting smoke density. A fast ray march produces a guiding map encoding the smoke's rough appearance. That guiding map feeds a U-Net with specialized channel adapters that disentangle the six directional lightmaps, predicting how light scatters from every angle in a single forward pass.
The method achieves remarkable numerical accuracy. Peak signal-to-noise ratios climb above 40, and mean squared error drops as low as 0.00009, even across densities far from the training distribution. This stability means smoke that's twice as thick or half as dense still renders with photorealistic fidelity, no retraining required.
The payoff is profound. Designers can now move the camera freely, relight scenes interactively, and introduce obstacles that cast volumetric shadows, all at 60 frames per second. The technique integrates directly into Unreal Engine, making it production-ready for next-generation games and virtual reality.
This work transforms six-way lightmaps from a static compromise into a dynamic rendering primitive. Photorealistic smoke is no longer a luxury reserved for pre-rendered cutscenes; it's now a real-time canvas for interactive storytelling. Visit EmergentMind.com to explore this research further and create your own video presentations.
