Imaging Hidden Objects with Consumer LiDAR via Motion Induced Sampling

Abstract: LiDARs are being increasingly deployed for consumer imaging in handheld, wearable, and robotic applications. These sensors can capture the time-of-flight of light at picosecond resolution, which in principle, enables them to capture information about objects hidden from their field of view. While such non-line-of-sight (NLOS) imaging capabilities have been shown on research-grade LiDARs, they are challenging to achieve on consumer devices due to poor signal quality resulting from low laser power, low spatial resolution, and object and camera motion. Inspired by burst photography and synthetic aperture radar, we propose a multi-frame fusion strategy to overcome these challenges and demonstrate NLOS imaging on consumer LiDAR. We first introduce the motion-induced aperture sampling model to unify the effects of object shape, object motion, and camera motion under a single measurement model. Using this model, we demonstrate several NLOS capabilities on a smartphone-grade LiDAR: (1) 3D reconstruction, (2) single and multi-object tracking, and (3) camera localization using hidden objects. Previously, NLOS imaging capabilities were largely restricted to bulky and expensive research-grade hardware that requires extensive setup and calibration. Our results represent a shift towards plug-and-play NLOS imaging, where anyone can image hidden objects with off-the-shelf hardware ($<100) and no additional setup. We believe that democratization of such capabilities will advance consumer applications of NLOS imaging.

• The paper proposes the motion-induced aperture sampling model that extends the light-cone transform to enable effective non-line-of-sight imaging.
• It leverages burst fusion and particle filtering to mitigate low SNR and motion blur, achieving robust 3D reconstruction, multi-object tracking, and camera localization.
• Empirical results demonstrate real-time performance on sub-$100 mobile LiDAR systems, outperforming conventional backprojection methods in challenging scenarios.

Imaging Hidden Objects with Consumer LiDAR via Motion Induced Sampling

Overview and Motivation

Consumer-grade LiDAR devices are increasingly integrated into smartphones, AR headsets, and robotics platforms. While these sensors are designed for direct line-of-sight (LOS) depth imaging, their picosecond time-of-flight (ToF) sensitivity inherently enables the possibility of non-line-of-sight (NLOS) imaging. Prior NLOS imaging demonstrations have relied on high-performance, research-grade hardware with significant optical power, spatial resolution, and extensive calibration. This work proposes and validates a framework that enables NLOS imaging—3D reconstruction, multi-object tracking, and camera localization—using commodity ($<\$100$) mobile LiDAR sensors, overcoming dominant constraints such as low signal-to-noise ratio (SNR), spatial sampling limitations, and challenges posed by joint object and camera motion.

Figure 1: Consumer-grade LiDAR enables NLOS imaging by interpreting relay surfaces as virtual mirrors; multi-frame fusion addresses the challenges of consumer hardware for 3D reconstruction, tracking, and localization.

Motion-Induced Aperture Sampling Model

The core technical contribution is the motion-induced aperture sampling (MAS) model, which extends the light-cone transform (LCT) formalism to unify the effects of object geometry, object dynamics, and camera pose. The MAS model enables robust multi-frame fusion under low-SNR and motion-blurred conditions. Key innovation lies in the decomposition of space-time impulse responses of the hidden scene into a canonical, time-invariant STIR modulated by temporally-varying shifts induced by rigid-body object translation and camera motion.

The LCT maps spatial and temporal measurement coordinates such that scene depth transforms as $v_z=z^2$ and time as $v_\tau=(c\tau/2)^2$, aligning the scene and measurement spaces for efficient convolutional modeling. Under MAS, the measured signal at a pixel is a function of spatial sampling, canonical STIR, and translation vectors that encode object motion and camera pose.

Figure 2: MAS builds on LCT to align scene and measurement spaces, capturing depth and time transformations to jointly model object and camera motions.

Applications: 3D Reconstruction, Object Tracking, and Camera Localization

Burst Fusion for 3D Reconstruction

Due to the limited spatial resolution ($\sim$100 pixels) and low SNR of consumer LiDARs, single-frame NLOS inference is infeasible. By exploiting natural handheld motion (inspired by burst photography and synthetic aperture approaches), the MAS model facilitates fusion of multiple frames, increasing both virtual aperture size and temporal/spatial sampling density. When the scene is static and camera poses are known, multi-frame fusion achieves robust 3D reconstruction, leveraging non-uniform sampling paradigms.

Particle Filtering for Single and Multi-Object Tracking

The tracking problem is framed as inference of object position(s) under known shape(s) and camera pose, highly non-convex due to ambiguities in the NLOS regime. MAS parameterizes the measurement model to enable sequential Bayesian inference via particle filtering. Each particle represents a candidate object (or joint object state) hypothesis, iteratively propagated according to motion priors, evaluated against stagewise measurements via MAS, and resampled according to likelihoods.

Figure 3: Particle filtering enables probabilistic sequential estimation of hidden object states, leveraging motion prior and measurement likelihoods under the MAS model.

Multi-object tracking extends the state space and renders data likelihoods as normalized superpositions of individual object measurements. Posterior distributions are analyzed via clustering to resolve multimodality, allowing robust tracking even when objects are occluded or ambiguous.

Figure 4: Multi-object NLOS tracking supports tracking both static and dynamic objects, including complex tasks such as hand tracking; probabilistic output reflects spatial uncertainty due to signal ambiguity.

Camera Localization via NLOS Landmarks

When LOS surfaces lack depth or texture features (e.g., planar walls), standard visual odometry fails. MAS utilizes signals reflected from static hidden objects to solve for the unknown (x, y) translation of the camera. By rendering hypothetical measurements for candidate camera poses and performing particle filtering, the system achieves localization in settings with minimal texture or geometric cues.

Extending MAS to Diffuse Objects

While MAS is derived assuming retroreflective hidden objects (to maximize SNR in the confocal configuration), empirical results demonstrate its competence on diffuse objects, albeit with degraded performance due to $r^4$ attenuation and additional non-confocal contributions. Real-time 3D reconstruction and tracking are achieved on challenging diffuse targets.

Figure 5: MAS empirically supports NLOS imaging of diffuse objects; performance is limited but still enables real-time shape recovery and tracking.

Numerical Results and Claims

The paper presents the first demonstration of real-time NLOS imaging tasks on consumer ($<\$100$), uncalibrated mobile LiDAR systems, validating robust multi-frame fusion, single/multi-object tracking, and camera localization. The particle filtering implementation supports uncertainty estimation and resolves regions of ambiguity inherent to NLOS measurements. Results are quantitatively benchmarked against reference trajectories and LOS-only methods; empirical tracking and localization accuracies are significantly improved over naive approaches. MAS outperforms conventional backprojection, particularly in low-SNR and dynamic contexts.

Implications and Future Directions

Practically, plug-and-play NLOS imaging democratizes access to robust hidden object inference for robotics, AR, UI/UX, and safety-critical navigation. Theoretically, MAS and burst fusion techniques extend NLOS imaging to real-world, unconstrained, handheld scenarios. Handling of multiple moving objects, uncertain poses, and diffuse reflectance broadens application scope.

Future research could pursue joint estimation of object shape, object motion, and camera pose (full NLOS SLAM), model non-rigid and rotational dynamics via deformation fields (e.g., neural radiance fields), and leverage learned score functions for robust particle likelihood evaluation. Signal enhancement via learned or physically-inspired priors and integration with minimalist vision paradigms are promising directions.

Conclusion

This paper establishes a rigorous and tractable methodology for non-line-of-sight imaging using commodity LiDAR sensors, driven by the MAS model and validated on mobile devices. The approach achieves reliable 3D reconstruction, single/multi-object tracking, and camera localization in challenging consumer sensor regimes. The results motivate further research in democratized NLOS imaging, joint estimation frameworks, and application to unconstrained environments (2605.17865).