Papers
Topics
Authors
Recent
Search
2000 character limit reached

Motion-Induced Aperture Sampling Model

Updated 4 July 2026
  • Motion-Induced Aperture Sampling (MAS) is a measurement model for NLOS imaging that jointly encodes hidden scene shape, rigid object motion, and camera sampling in one forward model.
  • The model employs a light-cone transformed domain to represent the hidden-scene impulse response as a 3D convolution, facilitating burst fusion from low-power LiDAR measurements.
  • MAS enables efficient 3D reconstruction, object tracking, and camera localization on consumer hardware by decoupling geometry and dynamics, as validated by real-time experiments.

Searching arXiv for the exact topic and nearby related work to ground the article in current preprints. arxiv_search(query="\"Motion Induced Sampling\" LiDAR", max_results=10) arxiv_search(query="\"Motion-Induced Aperture Sampling\" OR \"Imaging Hidden Objects with Consumer LiDAR via Motion Induced Sampling\"", max_results=10) Motion-Induced Aperture Sampling (MAS) is a measurement model for non-line-of-sight (NLOS) imaging with consumer LiDAR that unifies hidden-object shape, object motion, and camera motion in a single forward model. Introduced by Somasundaram et al. for smartphone-grade LiDAR, the model is formulated in the light-cone–transformed domain so that the hidden-scene impulse response becomes a 3D convolution, and it treats camera motion as a mechanism that sweeps a larger virtual aperture across the relay wall. Within that formulation, burst measurements from low-power, low-resolution, moving sensors can be fused for 3D reconstruction, single- and multi-object tracking, and camera localization using hidden objects (Somasundaram et al., 18 May 2026).

1. Conceptual scope and terminological context

The MAS model addresses a specific NLOS imaging setting: a consumer LiDAR observes a visible relay wall while a hidden scene lies outside direct line of sight. The central claim of the model is that object shape, rigid object translation, and camera pose variation can be expressed jointly in one sampling equation, rather than treated as separate nuisance factors. In this construction, the hidden scene is represented by a canonical space–time impulse response (STIR), object motion shifts that STIR, and camera motion changes which wall locations are sampled at each frame. As the user moves the LiDAR, the camera extrinsics Pt\mathbf P_t change and the virtual aperture on the wall sweeps out a larger region (Somasundaram et al., 18 May 2026).

A common source of confusion is the word aperture. In MAS, it refers to motion-induced sampling support on the relay wall in an NLOS imaging pipeline. This is distinct from the classical aperture problem in motion perception, where local velocity measurements of elongated contours are ambiguous and are resolved through motion-based predictive coding (Perrinet et al., 2012). It is also distinct from motion-driven synthetic-aperture Fourier ptychography, where target rotation translates sampled patches of an object’s Fourier spectrum under fixed optics (Chan et al., 4 Jul 2025). These neighboring usages are conceptually related in that motion increases inferential leverage, but the MAS model is specifically a confocal LiDAR measurement model for hidden-scene imaging (Somasundaram et al., 18 May 2026).

2. Light-cone formulation of the canonical hidden-scene response

The MAS formulation is expressed in the light-cone–transformed domain, denoted by R\mathcal R, where the hidden-scene impulse response becomes a 3D convolution. The canonical STIR is written as I(x,y,v)\mathcal I(x,y,v) for a hidden object with volumetric albedo ρ(x,y,z)\rho(x,y,z). Under the light-cone transform, depth zz is mapped to v=z2v=z^2, and time τ\tau is mapped to v=(cτ/2)2v=(c\tau/2)^2. The forward model is

I(x,y,v)=[Rz{ρ}](x,y,v)∗x,y,vh(x,y,v)\boxed{ \mathcal I(x,y,v)=\bigl[\mathcal R_z\{\rho\}\bigr](x,y,v)\ast_{x,y,v} h(x,y,v) }

with

h(x,y,v)=δ ⁣(x2+y2−v).h(x,y,v)=\delta\!\bigl(x^2+y^2-v\bigr).

Here, R\mathcal R0 is the remapping R\mathcal R1 of the object’s 3D albedo, R\mathcal R2 is the 3D point-spread function in the light-cone domain, and R\mathcal R3 denotes convolution over R\mathcal R4 (Somasundaram et al., 18 May 2026).

This representation is important because it isolates a time-independent canonical response. Once R\mathcal R5 has been defined, subsequent frames need not recompute hidden-scene shape. Instead, later variation is introduced by shifting or sampling that canonical response. This suggests a decomposition in which geometry and dynamics are separated at the level of the forward model, with shape encoded once and motion handled through translation and sampling operators.

3. Rigid object motion and camera-induced sampling

Rigid object motion enters MAS as a translation of the canonical STIR. If the object undergoes a rigid translation R\mathcal R6 at frame R\mathcal R7, then by shift invariance of convolution,

R\mathcal R8

The model explicitly states that no re-evaluation of shape is required—only a coordinate shift (Somasundaram et al., 18 May 2026).

Camera sampling is then defined in confocal form. For a consumer LiDAR pixel at image coordinate R\mathcal R9 and time bin I(x,y,v)\mathcal I(x,y,v)0, with intrinsics I(x,y,v)\mathcal I(x,y,v)1 and frame-dependent extrinsics I(x,y,v)\mathcal I(x,y,v)2, the continuous measurement is

I(x,y,v)\mathcal I(x,y,v)3

The Dirac term enforces that only the wall point I(x,y,v)\mathcal I(x,y,v)4 projecting to pixel I(x,y,v)\mathcal I(x,y,v)5 contributes; I(x,y,v)\mathcal I(x,y,v)6 carries the shifted STIR, thereby encoding object shape and object motion; and camera motion enters through I(x,y,v)\mathcal I(x,y,v)7 (Somasundaram et al., 18 May 2026).

This decomposition clarifies the model’s central unification. Canonical shape is contained in I(x,y,v)\mathcal I(x,y,v)8, rigid motion is a shift of that same tensor, and camera motion changes the wall sampling pattern. A plausible implication is that MAS can be read as a sampling-theoretic account of NLOS burst imaging: motion does not merely corrupt measurements, but changes which portions of a latent hidden-scene response are observed.

4. Multi-frame fusion and extended-aperture reconstruction

Because each LiDAR frame is noisy, low-resolution, and spatially cropped, MAS uses multi-frame fusion to estimate a higher-quality, extended-aperture STIR. The pipeline has three steps (Somasundaram et al., 18 May 2026).

First, each frame is warped into a common world grid using the known camera pose. For each pixel I(x,y,v)\mathcal I(x,y,v)9, the pose ρ(x,y,z)\rho(x,y,z)0 determines the world-wall coordinate ρ(x,y,z)\rho(x,y,z)1, and the warp is written as a linear operator ρ(x,y,z)\rho(x,y,z)2:

ρ(x,y,z)\rho(x,y,z)3

Second, the warped frames are fused by weighted accumulation. Using per-frame weights ρ(x,y,z)\rho(x,y,z)4, for example proportional to the integrated signal strength, the fused STIR estimate is

ρ(x,y,z)\rho(x,y,z)5

Third, once ρ(x,y,z)\rho(x,y,z)6 is sufficiently dense on ρ(x,y,z)\rho(x,y,z)7 and has good signal-to-noise ratio, reconstruction of ρ(x,y,z)\rho(x,y,z)8 proceeds via non-uniform backprojection or an LCT-inverse solver. The description includes a generic filtered-backprojection expression,

ρ(x,y,z)\rho(x,y,z)9

where zz0 is a known filter kernel (Somasundaram et al., 18 May 2026).

For static scenes, the reconstruction problem is solved by discrete filtered backprojection on the fused zz1. No iterative solver is required if sampling density is high enough. This places MAS close to burst-fusion imaging methods, but with the additional constraint that the fused quantity is not an ordinary image stack: it is a wall-parameterized, time-resolved hidden-scene impulse response.

5. Sequential inference for tracking and localization

MAS is not limited to static reconstruction. The same forward model supports sequential Bayesian estimation when the canonical hidden-scene response is known but motion or camera pose is unknown (Somasundaram et al., 18 May 2026).

For single-object 3D tracking, the unknown state is the translation zz2. Let the photon-histogram measurement vector at frame zz3 be zz4, and let zz5 denote the rendering of the canonical zz6 shifted by zz7 and sampled according to the MAS measurement equation. The posterior is

zz8

Inference is implemented with a particle filter. Propagation samples

zz9

and the particle weight is

v=z2v=z^20

Particles are then resampled by importance sampling and normalized to equal weights (Somasundaram et al., 18 May 2026).

For multi-object tracking with v=z2v=z^21 known shapes, the state becomes v=z2v=z^22, and the rendered measurement is the superposition of v=z2v=z^23 shifted STIRs. The same particle-filter steps apply, after which the particles are clustered, for example by v=z2v=z^24-means, to extract object positions (Somasundaram et al., 18 May 2026).

For camera localization with a known static hidden shape, the unknown becomes the 2D camera translation v=z2v=z^25 parallel to the wall. The line-of-sight point cloud is first used to fix camera height and orientation, leaving only planar translation unknown. The NLOS signal from the known hidden object is then rendered under each hypothesized v=z2v=z^26 using the same measurement model, and a particle filter estimates camera position in real time (Somasundaram et al., 18 May 2026).

6. Assumptions, limitations, and reported empirical performance

The MAS model is derived under a specific set of assumptions. The relay wall is planar at v=z2v=z^27 and infinitely Lambertian. The hidden object undergoes rigid translation with no rotations. Reflectance is approximated as retroreflective in confocal geometry; diffuse objects still work empirically but with lower signal-to-noise ratio. Noise is modeled as dominated by Poisson photon counts and sensor jitter, while timing quantization and dark counts are neglected in the core model. The formulation excludes interreflections and occlusions beyond the first virtual bounce. Camera intrinsics v=z2v=z^28 and tracking of v=z2v=z^29 via onboard IMU/SLAM are assumed known. Maximum translation per frame is assumed small enough to avoid aliasing. The paper states that these simplifications preclude arbitrary rotations, non-planar relay surfaces, and strong secondary bounces (Somasundaram et al., 18 May 2026).

These assumptions address several likely misconceptions. MAS does not claim unrestricted hidden-scene motion modeling; its motion component is rigid translation. It does not require purely static hidden scenes; instead, it treats motion as a shift of a canonical response. It is also not restricted to retroreflective targets in an absolute sense, since diffuse objects still work qualitatively, but the reported behavior includes lower SNR and larger ambiguity lobes (Somasundaram et al., 18 May 2026).

The reported empirical validation is summarized below.

Task Setup Reported result
Static hidden-object reconstruction τ\tau0 s handheld scans with smartphone LiDAR Quality matches research-grade phasor-field systems up to τ\tau1–τ\tau2 cm resolution
Single-object 3D tracking τ\tau3 Hz Mean error τ\tau4 cm in a τ\tau5 mτ\tau6 volume
Multi-object tracking Two retroreflective targets, and also human hand Run-time τ\tau7 ms/frame on a GPU
Camera localization Planar walls with no visual texture Drift reduced from τ\tau8 cm to τ\tau9 cm over v=(cτ/2)2v=(c\tau/2)^20 m translation

The hardware is described as an off-the-shelf smartphone LiDAR with approximately v=(cτ/2)2v=(c\tau/2)^21 SPAD pixels, a v=(cτ/2)2v=(c\tau/2)^22 Hz frame rate, and an eye-safe laser (Somasundaram et al., 18 May 2026). The paper further states that these experiments enable the first demonstrations of NLOS imaging, tracking, and localization on sub-$100 consumer LiDAR hardware with real-time on-device performance. This suggests that the principal significance of MAS lies less in a new inverse solver than in a measurement unification that makes burst fusion, tracking, and localization tractable on commodity time-of-flight hardware.

7. Relation to adjacent motion-based sampling models

MAS belongs to a broader class of models that use motion to resolve incomplete measurements, but its technical role is distinct. In the visual-motion literature, "Motion-based prediction is sufficient to solve the aperture problem" formulates a joint posterior over position and velocity, updated by Bayes–Markov filtering and implemented by Sequential Monte Carlo. There, temporally coherent features accumulate probability mass and incoherent local features are explained away, yielding a progressive solution of the classical aperture problem without explicit end-stopped filters (Perrinet et al., 2012). The connection to MAS is methodological rather than domain-specific: both models use motion-conditioned propagation and particle-based inference, but the latent quantities and sensors differ.

In Fourier ptychography, "Inverse Synthetic Aperture Fourier Ptychography" uses unknown target rotation to induce a spatially linear phase ramp that translates sampled patches of the object spectrum. Target motion thereby replaces illumination-angle changes to generate synthetic-aperture diversity, and a learned module estimates the induced v=(cτ/2)2v=(c\tau/2)^23-space coordinates from dual-plane measurements (Chan et al., 4 Jul 2025). Relative to that framework, MAS does not operate in the object Fourier domain and does not rely on phase retrieval; instead, it samples a hidden-scene impulse response through a relay wall under confocal LiDAR geometry.

Taken together, these neighboring formulations indicate that motion-induced sampling is not a single algorithmic template but a recurring principle. In MAS, that principle takes the specific form of a light-cone-domain sampling model for NLOS imaging, in which hidden-scene shape, rigid translation, and camera pose are expressed as separable but coupled components of one forward operator (Somasundaram et al., 18 May 2026).

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Motion-Induced Aperture Sampling Model.