Color-Encoded Illumination for High-Speed Volumetric Scene Reconstruction

Abstract: The task of capturing and rendering 3D dynamic scenes from 2D images has become increasingly popular in recent years. However, most conventional cameras are bandwidth-limited to 30-60 FPS, restricting these methods to static or slowly evolving scenes. While overcoming bandwidth limitations is difficult for general scenes, recent years have seen a flurry of computational imaging methods that yield high-speed videos using conventional cameras for specific applications (e.g., motion capture and particle image velocimetry). However, most of these methods require modifications to a camera's optics or the addition of mechanically moving components, limiting them to a single-view high-speed capture. Consequently, these methods cannot be readily used to capture a 3D representation of rapid scene motion. In this paper, we propose a novel method to capture and reconstruct a volumetric representation of a high-speed scene using only unaugmented low-speed cameras. Instead of modifying the hardware or optics of each individual camera, we encode high-speed scene dynamics by illuminating the scene with a rapid, sequential color-coded sequence. This results in simultaneous multi-view capture of the scene, where high-speed temporal information is encoded in the spatial intensity and color variations of the captured images. To construct a high-speed volumetric representation of the dynamic scene, we develop a novel dynamic Gaussian Splatting-based approach that decodes the temporal information from the images. We evaluate our approach on simulated scenes and real-world experiments using a multi-camera imaging setup, showing first-of-a-kind high-speed volumetric scene reconstructions.

• The paper introduces a novel method using rapid RGB strobing to encode temporal information within a single exposure.
• It adapts dynamic 3D Gaussian Splatting to reconstruct high-speed volumetric scenes up to 600 Hz using commodity cameras.
• The approach demonstrates practical trade-offs related to lighting, object reflectance, and motion complexity while achieving robust multi-view reconstruction.

High-Speed Volumetric Scene Reconstruction via Color-Encoded Illumination

Introduction and Motivation

The paper "Color-Encoded Illumination for High-Speed Volumetric Scene Reconstruction" (2604.26920) introduces a novel imaging paradigm to enable high-speed 3D reconstruction of dynamic scenes using only commodity low-frame-rate cameras. The fundamental advance is the introduction of rapid color strobing to temporally encode scene motion within a single exposure, circumventing the bandwidth limitations inherent to typical camera hardware and eliminating the need for specialized optics or sensor augmentation.

The approach targets applications where precise 3D motion capture is paramount (e.g., particle image velocimetry, ballistics tracking, and object tracking), and the appearance of objects is secondary to their dynamics. Instead of relying on specialized cameras, the method deploys rapid, programmable color strobing and leverages multi-view geometry. Temporal information is encoded in color mixtures and subsequently decoded using an extension of 3D Gaussian Splatting optimized for dynamic scenes.

Figure 1: High-speed volumetric scene encoding and reconstruction: (a) sequential color strobing for rapid motion; (b) blurred frame under constant illumination; (c) strobed frame containing color mixtures; (d) dynamic volumetric reconstruction yielding novel high-speed views at 600 Hz.

Methodological Framework

Color-Encoded Exposure and Image Formation

The image formation model introduces sequential color strobing via three RGB LED channels. Each strobe interval encodes a distinct temporal slot by modulating the LED intensities $(\alpha_n, \beta_n, \gamma_n)$, resulting in linear mixtures of object colors in the camera's RGB space. The temporal sequence is constructed such that each camera frame records a combination of object positions, each tagged by a unique color. The encoding exploits the linearity of color mixing, yielding elliptical manifolds in camera RGB space.

Figure 2: Strobing protocol in $\alpha,\beta,\gamma$ space and observed color ellipses in camera RGB space, demonstrating the color encoding's linearity.

Pulse-width modulation is employed for precise color generation, attenuating the LEDs' influence during each strobe interval. The method assumes object quasi-staticity during each strobe, an assumption validated empirically.

Dynamic Scene Reconstruction

Extension to Dynamic 3D Gaussian Splatting

The reconstruction step adapts the Gaussian-Flow dynamic 3DGS method [lin2024gaussian], wherein the 3D scene comprises time-evolving Gaussians optimized via differentiable rendering. The temporal evolution of Gaussians is modeled using simple deformation bases, and the loss function is amended to minimize discrepancies between the rendered strobe mixtures and captured color images.

Temporal slots (interframes) are reconstructed by decomposing color mixtures using the known strobing schedule and multi-view geometry. Additional total variation regularization in inverse depth promotes geometric consistency.

Calibration Pipeline

Camera spectral responses are equalized via color checker calibration, and backgrounds are statically subtracted to isolate moving objects. COLMAP is used for view calibration and point cloud initialization. All data processing is performed in a multi-view synchronized context.

Figure 3: Experimental prototype setup—three RGB LED channels, high-speed PWM control, and synchronized multi-camera capture.

Experimental Results and Performance Analysis

Real-World Evaluation

Experiments encompass rapid object motion scenarios: spinning disks, flying chess pieces, and Nerf darts. The system achieves temporal upsampling from 60 Hz to 600 Hz, reconstructing volumetric representations via dynamic Gaussian Splatting. Compared to conventional illumination, strobed images contain discernible color mixtures, enabling high-speed recovery with minimal artifacts. The method supports both existing and novel view synthesis at ultra-high frame rates.

Figure 4: Simulation analysis showing reconstruction quality dependence on interframe count, ambient light contamination, albedo variation, camera number, and motion complexity.

Simulation-Based Limitations

Simulation studies reveal the primary bottlenecks:

• Interframe Overlap: Increasing the number of interframes reduces separation in RGB color space, amplifying noise susceptibility and leading to reconstruction artifacts beyond roughly 22 upsampling slots.
• Ambient Light Sensitivity: Ambient illumination decreases color contrast, degrading reconstruction fidelity linearly.
• Albedo Dependence: Broad-spectrum (off-white) object reflectances are preferred; narrow-band albedos are detrimental.
• Multi-View Constraint: Quality improves with increased camera count; six views suffice for robust synthesis.
• Motion Complexity: Highly non-smooth or complex motion can induce vanishing gradients and dropped interframes due to the ill-posedness of mixture decomposition.

  Figure 1: Volumetric reconstruction pipeline illustration, depicting color-coded strobing and multi-view capture enabling high-speed scene synthesis.

Discussion, Limitations, and Implications

The framework is optimized for scenarios with single-color objects and dark backgrounds. The uniform albedo constraint, although restrictive, aligns with common high-speed motion analysis domains. The upsampling limit is dictated by the color-space discrimination achievable with current LED technology and camera spectral response. SNR degradation with distance and ambient light is a generic active illumination issue, not specific to the proposed method.

Crucially, the methodology eschews the need for camera augmentation, facilitating straightforward multi-view scaling. The multi-view optimization constrains the decomposition ill-posedness, implicitly leveraging epipolar geometry.

Future work may address heterogeneous albedos, light-squared falloff compensation, joint motion-appearance recovery, and integration of machine-learned priors for robustness in diverse environments. Extensions to hyperspectral strobing and deployment on faster cameras for further temporal upscaling are also apparent directions.

Conclusion

This paper establishes a principled framework for high-speed volumetric scene reconstruction using color-encoded illumination and unaugmented cameras. Strong empirical results demonstrate temporal upsampling by a factor of 10, validated both qualitatively and quantitatively. The theoretical and practical implications are significant for computational imaging, motion analysis, and dynamic scene synthesis. Future developments will likely extend to broader classes of objects and enable scalable, robust capture in demanding real-world conditions.