Motion-Induced Aperture Sampling Model
- Motion-Induced Aperture Sampling (MAS) is a measurement model for NLOS imaging that jointly encodes hidden scene shape, rigid object motion, and camera sampling in one forward model.
- The model employs a light-cone transformed domain to represent the hidden-scene impulse response as a 3D convolution, facilitating burst fusion from low-power LiDAR measurements.
- MAS enables efficient 3D reconstruction, object tracking, and camera localization on consumer hardware by decoupling geometry and dynamics, as validated by real-time experiments.
Searching arXiv for the exact topic and nearby related work to ground the article in current preprints. arxiv_search(query="\"Motion Induced Sampling\" LiDAR", max_results=10) arxiv_search(query="\"Motion-Induced Aperture Sampling\" OR \"Imaging Hidden Objects with Consumer LiDAR via Motion Induced Sampling\"", max_results=10) Motion-Induced Aperture Sampling (MAS) is a measurement model for non-line-of-sight (NLOS) imaging with consumer LiDAR that unifies hidden-object shape, object motion, and camera motion in a single forward model. Introduced by Somasundaram et al. for smartphone-grade LiDAR, the model is formulated in the light-cone–transformed domain so that the hidden-scene impulse response becomes a 3D convolution, and it treats camera motion as a mechanism that sweeps a larger virtual aperture across the relay wall. Within that formulation, burst measurements from low-power, low-resolution, moving sensors can be fused for 3D reconstruction, single- and multi-object tracking, and camera localization using hidden objects (Somasundaram et al., 18 May 2026).
1. Conceptual scope and terminological context
The MAS model addresses a specific NLOS imaging setting: a consumer LiDAR observes a visible relay wall while a hidden scene lies outside direct line of sight. The central claim of the model is that object shape, rigid object translation, and camera pose variation can be expressed jointly in one sampling equation, rather than treated as separate nuisance factors. In this construction, the hidden scene is represented by a canonical space–time impulse response (STIR), object motion shifts that STIR, and camera motion changes which wall locations are sampled at each frame. As the user moves the LiDAR, the camera extrinsics change and the virtual aperture on the wall sweeps out a larger region (Somasundaram et al., 18 May 2026).
A common source of confusion is the word aperture. In MAS, it refers to motion-induced sampling support on the relay wall in an NLOS imaging pipeline. This is distinct from the classical aperture problem in motion perception, where local velocity measurements of elongated contours are ambiguous and are resolved through motion-based predictive coding (Perrinet et al., 2012). It is also distinct from motion-driven synthetic-aperture Fourier ptychography, where target rotation translates sampled patches of an object’s Fourier spectrum under fixed optics (Chan et al., 4 Jul 2025). These neighboring usages are conceptually related in that motion increases inferential leverage, but the MAS model is specifically a confocal LiDAR measurement model for hidden-scene imaging (Somasundaram et al., 18 May 2026).
2. Light-cone formulation of the canonical hidden-scene response
The MAS formulation is expressed in the light-cone–transformed domain, denoted by , where the hidden-scene impulse response becomes a 3D convolution. The canonical STIR is written as for a hidden object with volumetric albedo . Under the light-cone transform, depth is mapped to , and time is mapped to . The forward model is
with
Here, 0 is the remapping 1 of the object’s 3D albedo, 2 is the 3D point-spread function in the light-cone domain, and 3 denotes convolution over 4 (Somasundaram et al., 18 May 2026).
This representation is important because it isolates a time-independent canonical response. Once 5 has been defined, subsequent frames need not recompute hidden-scene shape. Instead, later variation is introduced by shifting or sampling that canonical response. This suggests a decomposition in which geometry and dynamics are separated at the level of the forward model, with shape encoded once and motion handled through translation and sampling operators.
3. Rigid object motion and camera-induced sampling
Rigid object motion enters MAS as a translation of the canonical STIR. If the object undergoes a rigid translation 6 at frame 7, then by shift invariance of convolution,
8
The model explicitly states that no re-evaluation of shape is required—only a coordinate shift (Somasundaram et al., 18 May 2026).
Camera sampling is then defined in confocal form. For a consumer LiDAR pixel at image coordinate 9 and time bin 0, with intrinsics 1 and frame-dependent extrinsics 2, the continuous measurement is
3
The Dirac term enforces that only the wall point 4 projecting to pixel 5 contributes; 6 carries the shifted STIR, thereby encoding object shape and object motion; and camera motion enters through 7 (Somasundaram et al., 18 May 2026).
This decomposition clarifies the model’s central unification. Canonical shape is contained in 8, rigid motion is a shift of that same tensor, and camera motion changes the wall sampling pattern. A plausible implication is that MAS can be read as a sampling-theoretic account of NLOS burst imaging: motion does not merely corrupt measurements, but changes which portions of a latent hidden-scene response are observed.
4. Multi-frame fusion and extended-aperture reconstruction
Because each LiDAR frame is noisy, low-resolution, and spatially cropped, MAS uses multi-frame fusion to estimate a higher-quality, extended-aperture STIR. The pipeline has three steps (Somasundaram et al., 18 May 2026).
First, each frame is warped into a common world grid using the known camera pose. For each pixel 9, the pose 0 determines the world-wall coordinate 1, and the warp is written as a linear operator 2:
3
Second, the warped frames are fused by weighted accumulation. Using per-frame weights 4, for example proportional to the integrated signal strength, the fused STIR estimate is
5
Third, once 6 is sufficiently dense on 7 and has good signal-to-noise ratio, reconstruction of 8 proceeds via non-uniform backprojection or an LCT-inverse solver. The description includes a generic filtered-backprojection expression,
9
where 0 is a known filter kernel (Somasundaram et al., 18 May 2026).
For static scenes, the reconstruction problem is solved by discrete filtered backprojection on the fused 1. No iterative solver is required if sampling density is high enough. This places MAS close to burst-fusion imaging methods, but with the additional constraint that the fused quantity is not an ordinary image stack: it is a wall-parameterized, time-resolved hidden-scene impulse response.
5. Sequential inference for tracking and localization
MAS is not limited to static reconstruction. The same forward model supports sequential Bayesian estimation when the canonical hidden-scene response is known but motion or camera pose is unknown (Somasundaram et al., 18 May 2026).
For single-object 3D tracking, the unknown state is the translation 2. Let the photon-histogram measurement vector at frame 3 be 4, and let 5 denote the rendering of the canonical 6 shifted by 7 and sampled according to the MAS measurement equation. The posterior is
8
Inference is implemented with a particle filter. Propagation samples
9
and the particle weight is
0
Particles are then resampled by importance sampling and normalized to equal weights (Somasundaram et al., 18 May 2026).
For multi-object tracking with 1 known shapes, the state becomes 2, and the rendered measurement is the superposition of 3 shifted STIRs. The same particle-filter steps apply, after which the particles are clustered, for example by 4-means, to extract object positions (Somasundaram et al., 18 May 2026).
For camera localization with a known static hidden shape, the unknown becomes the 2D camera translation 5 parallel to the wall. The line-of-sight point cloud is first used to fix camera height and orientation, leaving only planar translation unknown. The NLOS signal from the known hidden object is then rendered under each hypothesized 6 using the same measurement model, and a particle filter estimates camera position in real time (Somasundaram et al., 18 May 2026).
6. Assumptions, limitations, and reported empirical performance
The MAS model is derived under a specific set of assumptions. The relay wall is planar at 7 and infinitely Lambertian. The hidden object undergoes rigid translation with no rotations. Reflectance is approximated as retroreflective in confocal geometry; diffuse objects still work empirically but with lower signal-to-noise ratio. Noise is modeled as dominated by Poisson photon counts and sensor jitter, while timing quantization and dark counts are neglected in the core model. The formulation excludes interreflections and occlusions beyond the first virtual bounce. Camera intrinsics 8 and tracking of 9 via onboard IMU/SLAM are assumed known. Maximum translation per frame is assumed small enough to avoid aliasing. The paper states that these simplifications preclude arbitrary rotations, non-planar relay surfaces, and strong secondary bounces (Somasundaram et al., 18 May 2026).
These assumptions address several likely misconceptions. MAS does not claim unrestricted hidden-scene motion modeling; its motion component is rigid translation. It does not require purely static hidden scenes; instead, it treats motion as a shift of a canonical response. It is also not restricted to retroreflective targets in an absolute sense, since diffuse objects still work qualitatively, but the reported behavior includes lower SNR and larger ambiguity lobes (Somasundaram et al., 18 May 2026).
The reported empirical validation is summarized below.
| Task | Setup | Reported result |
|---|---|---|
| Static hidden-object reconstruction | 0 s handheld scans with smartphone LiDAR | Quality matches research-grade phasor-field systems up to 1–2 cm resolution |
| Single-object 3D tracking | 3 Hz | Mean error 4 cm in a 5 m6 volume |
| Multi-object tracking | Two retroreflective targets, and also human hand | Run-time 7 ms/frame on a GPU |
| Camera localization | Planar walls with no visual texture | Drift reduced from 8 cm to 9 cm over 0 m translation |
The hardware is described as an off-the-shelf smartphone LiDAR with approximately 1 SPAD pixels, a 2 Hz frame rate, and an eye-safe laser (Somasundaram et al., 18 May 2026). The paper further states that these experiments enable the first demonstrations of NLOS imaging, tracking, and localization on sub-$100 consumer LiDAR hardware with real-time on-device performance. This suggests that the principal significance of MAS lies less in a new inverse solver than in a measurement unification that makes burst fusion, tracking, and localization tractable on commodity time-of-flight hardware.
7. Relation to adjacent motion-based sampling models
MAS belongs to a broader class of models that use motion to resolve incomplete measurements, but its technical role is distinct. In the visual-motion literature, "Motion-based prediction is sufficient to solve the aperture problem" formulates a joint posterior over position and velocity, updated by Bayes–Markov filtering and implemented by Sequential Monte Carlo. There, temporally coherent features accumulate probability mass and incoherent local features are explained away, yielding a progressive solution of the classical aperture problem without explicit end-stopped filters (Perrinet et al., 2012). The connection to MAS is methodological rather than domain-specific: both models use motion-conditioned propagation and particle-based inference, but the latent quantities and sensors differ.
In Fourier ptychography, "Inverse Synthetic Aperture Fourier Ptychography" uses unknown target rotation to induce a spatially linear phase ramp that translates sampled patches of the object spectrum. Target motion thereby replaces illumination-angle changes to generate synthetic-aperture diversity, and a learned module estimates the induced 3-space coordinates from dual-plane measurements (Chan et al., 4 Jul 2025). Relative to that framework, MAS does not operate in the object Fourier domain and does not rely on phase retrieval; instead, it samples a hidden-scene impulse response through a relay wall under confocal LiDAR geometry.
Taken together, these neighboring formulations indicate that motion-induced sampling is not a single algorithmic template but a recurring principle. In MAS, that principle takes the specific form of a light-cone-domain sampling model for NLOS imaging, in which hidden-scene shape, rigid translation, and camera pose are expressed as separable but coupled components of one forward operator (Somasundaram et al., 18 May 2026).