Motion-Induced Aperture Sampling Model

• Motion-Induced Aperture Sampling (MAS) is a measurement model for NLOS imaging that jointly encodes hidden scene shape, rigid object motion, and camera sampling in one forward model.
• The model employs a light-cone transformed domain to represent the hidden-scene impulse response as a 3D convolution, facilitating burst fusion from low-power LiDAR measurements.
• MAS enables efficient 3D reconstruction, object tracking, and camera localization on consumer hardware by decoupling geometry and dynamics, as validated by real-time experiments.

Searching arXiv for the exact topic and nearby related work to ground the article in current preprints. arxiv_search(query="^^^^^^^^4\4 Induced Sampling^^^^^^^^4\4 LiDAR4^^^^^^^^", max_results=^^^^4 LiDAR4\4^^^^) arxiv_search(query="^^^^^^^^4\4 Aperture Sampling^^^^^^^^4\4 OR ^^^^^^^^4\4 Hidden Objects with Consumer^^^^^^^^4 LiDAR4^^^^^^^^ via Motion Induced Sampling^^^^^^^^4\4 max_results=^^^^4 LiDAR4\4^^^^) Motion-Induced Aperture Sampling (MAS) is a measurement model for non-line-of-sight (NLOS) imaging with consumer^^^^^^^^4 LiDAR4^^^^^^^^ that unifies hidden-object shape, object motion, and camera motion in a single forward model. Introduced by Somasundaram et al. for smartphone-grade^^^^^^^^4 LiDAR4^^^^^^^^, the model is formulated in the light-cone–transformed domain so that the hidden-scene impulse response becomes a ^^^^4 OR \4^^^^D convolution, and it treats camera motion as a mechanism that sweeps a larger virtual aperture across the relay wall. Within that formulation, burst measurements from low-power, low-resolution, moving sensors can be fused for ^^^^4 OR \4^^^^D reconstruction, single- and multi-object tracking, and camera localization using hidden objects (&&&^^^^4\4^^^^&&&).

^^^^4 LiDAR4^^^^. Conceptual scope and terminological context

The MAS model addresses a specific NLOS imaging setting: a consumer^^^^^^^^4 LiDAR4^^^^^^^^ observes a visible relay wall while a hidden scene lies outside direct line of sight. The central claim of the model is that object shape, rigid object translation, and camera pose variation can be expressed jointly in one sampling equation, rather than treated as separate nuisance factors. In this construction, the hidden scene is represented by a canonical space–time impulse response (STIR), object motion shifts that STIR, and camera motion changes which wall locations are sampled at each frame. As the user moves the^^^^^^^^4 LiDAR4^^^^^^^^, the camera extrinsics PRESERVED_PLACEHOLDER_^^^^4\4^^^^ change and the virtual aperture on the wall sweeps out a larger region (&&&^^^^4\4^^^^&&&).

A common source of confusion is the word aperture. In MAS, it refers to motion-induced sampling support on the relay wall in an NLOS imaging pipeline. This is distinct from the classical aperture problem in motion perception, where local velocity measurements of elongated contours are ambiguous and are resolved through motion-based predictive coding (&&&^^^^4\4^^^^&&&). It is also distinct from motion-driven synthetic-aperture Fourier ptychography, where target rotation translates sampled patches of an object’s Fourier spectrum under fixed optics (&&&^^^^4 OR \4^^^^&&&). These neighboring usages are conceptually related in that motion increases inferential leverage, but the MAS model is specifically a confocal^^^^^^^^4 LiDAR4^^^^^^^^ measurement model for hidden-scene imaging (&&&^^^^4\4^^^^&&&).

^^^^4\4^^^^. Light-cone formulation of the canonical hidden-scene response

The MAS formulation is expressed in the light-cone–transformed domain, denoted by PRESERVED_PLACEHOLDER_^^^^4 LiDAR4^^^^, where the hidden-scene impulse response becomes a ^^^^4 OR \4^^^^D convolution. The canonical STIR is written as PRESERVED_PLACEHOLDER_^^^^4\4^^^^ for a hidden object with volumetric albedo PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^. Under the light-cone transform, depth $z$ is mapped to $v=z^2$, and time $\tau$ is mapped to $v=(c\tau/2)^2$. The forward model is

$$\boxed{ \mathcal I(x,y,v)=\bigl[\mathcal R_z\{\rho\}\bigr](x,y,v)\ast_{x,y,v} h(x,y,v) }$$

with

$h(x,y,v)=\delta\!\bigl(x^2+y^2-v\bigr).$

Here, PRESERVED_PLACEHOLDER_^^^^4 LiDAR4\4^^^^ is the remapping PRESERVED_PLACEHOLDER_^^^^4 LiDAR4 LiDAR4^^^^ of the object’s ^^^^4 OR \4^^^^D albedo, PRESERVED_PLACEHOLDER_^^^^4 LiDAR4\4^^^^ is the ^^^^4 OR \4^^^^D point-spread function in the light-cone domain, and PRESERVED_PLACEHOLDER_^^^^4 LiDAR4 OR \4^^^^ denotes convolution over PRESERVED_PLACEHOLDER_^^^^4 LiDAR44^^^^ (&&&^^^^4\4^^^^&&&).

This representation is important because it isolates a time-independent canonical response. Once PRESERVED_PLACEHOLDER_^^^^4 LiDAR4^^^^5 has been defined, subsequent frames need not recompute hidden-scene shape. Instead, later variation is introduced by shifting or sampling that canonical response. This suggests a decomposition in which geometry and dynamics are separated at the level of the forward model, with shape encoded once and motion handled through translation and sampling operators.

^^^^4 OR \4^^^^. Rigid object motion and camera-induced sampling

Rigid object motion enters MAS as a translation of the canonical STIR. If the object undergoes a rigid translation PRESERVED_PLACEHOLDER_^^^^4 LiDAR4^^^^6 at frame PRESERVED_PLACEHOLDER_^^^^4 LiDAR4^^^^7, then by shift invariance of convolution,

PRESERVED_PLACEHOLDER_^^^^4 LiDAR4^^^^8

The model explicitly states that no re-evaluation of shape is required—only a coordinate shift (&&&^^^^4\4^^^^&&&).

Camera sampling is then defined in confocal form. For a consumer^^^^^^^^4 LiDAR4^^^^^^^^ pixel at image coordinate PRESERVED_PLACEHOLDER_^^^^4 LiDAR4^^^^9 and time bin PRESERVED_PLACEHOLDER_^^^^4\4\4^^^^, with intrinsics PRESERVED_PLACEHOLDER_^^^^4\4 LiDAR4^^^^ and frame-dependent extrinsics PRESERVED_PLACEHOLDER_^^^^4\4\4^^^^, the continuous measurement is

PRESERVED_PLACEHOLDER_^^^^4\4 OR \4^^^^

The Dirac term enforces that only the wall point PRESERVED_PLACEHOLDER_^^^^4\44^^^^ projecting to pixel PRESERVED_PLACEHOLDER_^^^^4\4^^^^5 contributes; PRESERVED_PLACEHOLDER_^^^^4\4^^^^6 carries the shifted STIR, thereby encoding object shape and object motion; and camera motion enters through PRESERVED_PLACEHOLDER_^^^^4\4^^^^7 (&&&^^^^4\4^^^^&&&).

This decomposition clarifies the model’s central unification. Canonical shape is contained in PRESERVED_PLACEHOLDER_^^^^4\4^^^^8, rigid motion is a shift of that same tensor, and camera motion changes the wall sampling pattern. A plausible implication is that MAS can be read as a sampling-theoretic account of NLOS burst imaging: motion does not merely corrupt measurements, but changes which portions of a latent hidden-scene response are observed.

4. Multi-frame fusion and extended-aperture reconstruction

Because each^^^^^^^^4 LiDAR4^^^^^^^^ frame is noisy, low-resolution, and spatially cropped, MAS uses multi-frame fusion to estimate a higher-quality, extended-aperture STIR. The pipeline has three steps (&&&^^^^4\4^^^^&&&).

First, each frame is warped into a common world grid using the known camera pose. For each pixel PRESERVED_PLACEHOLDER_^^^^4\4^^^^9, the pose PRESERVED_PLACEHOLDER_^^^^4 OR \4\4^^^^ determines the world-wall coordinate PRESERVED_PLACEHOLDER_^^^^4 OR \4 LiDAR4^^^^, and the warp is written as a linear operator PRESERVED_PLACEHOLDER_^^^^4 OR \4\4^^^^:

PRESERVED_PLACEHOLDER_^^^^4 OR \4 OR \4^^^^

Second, the warped frames are fused by weighted accumulation. Using per-frame weights PRESERVED_PLACEHOLDER_^^^^4 OR \44^^^^, for example proportional to the integrated signal strength, the fused STIR estimate is

PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^5

Third, once PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^6 is sufficiently dense on PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^7 and has good signal-to-noise ratio, reconstruction of PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^8 proceeds via non-uniform backprojection or an LCT-inverse solver. The description includes a generic filtered-backprojection expression,

PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^9

where $z$^^^^4\4^^^^ is a known filter kernel (&&&^^^^4\4^^^^&&&).

For static scenes, the reconstruction problem is solved by discrete filtered backprojection on the fused $z$^^^^4 LiDAR4^^^^. No iterative solver is required if sampling density is high enough. This places MAS close to burst-fusion imaging methods, but with the additional constraint that the fused quantity is not an ordinary image stack: it is a wall-parameterized, time-resolved hidden-scene impulse response.

5. Sequential inference for tracking and localization

MAS is not limited to static reconstruction. The same forward model supports sequential Bayesian estimation when the canonical hidden-scene response is known but motion or camera pose is unknown (&&&^^^^4\4^^^^&&&).

For single-object ^^^^4 OR \4^^^^D tracking, the unknown state is the translation $z$^^^^4\4^^^^. Let the photon-histogram measurement vector at frame $z$^^^^4 OR \4^^^^ be $z$4, and let $z$5 denote the rendering of the canonical $z$6 shifted by $z$7 and sampled according to the MAS measurement equation. The posterior is

$z$8

Inference is implemented with a particle filter. Propagation samples

$z$9

and the particle weight is

$v=z^2$^^^^4\4^^^^

Particles are then resampled by importance sampling and normalized to equal weights (&&&^^^^4\4^^^^&&&).

For multi-object tracking with $v=z^2$^^^^4 LiDAR4^^^^ known shapes, the state becomes $v=z^2$^^^^4\4^^^^, and the rendered measurement is the superposition of $v=z^2$^^^^4 OR \4^^^^ shifted STIRs. The same particle-filter steps apply, after which the particles are clustered, for example by $v=z^2$4-means, to extract object positions (&&&^^^^4\4^^^^&&&).

For camera localization with a known static hidden shape, the unknown becomes the ^^^^4\4^^^^D camera translation $v=z^2$5 parallel to the wall. The line-of-sight point cloud is first used to fix camera height and orientation, leaving only planar translation unknown. The NLOS signal from the known hidden object is then rendered under each hypothesized $v=z^2$6 using the same measurement model, and a particle filter estimates camera position in real time (&&&^^^^4\4^^^^&&&).

6. Assumptions, limitations, and reported empirical performance

The MAS model is derived under a specific set of assumptions. The relay wall is planar at $v=z^2$7 and infinitely Lambertian. The hidden object undergoes rigid translation with no rotations. Reflectance is approximated as retroreflective in confocal geometry; diffuse objects still work empirically but with lower signal-to-noise ratio. Noise is modeled as dominated by Poisson photon counts and sensor jitter, while timing quantization and dark counts are neglected in the core model. The formulation excludes interreflections and occlusions beyond the first virtual bounce. Camera intrinsics $v=z^2$8 and tracking of $v=z^2$9 via onboard IMU/SLAM are assumed known. Maximum translation per frame is assumed small enough to avoid aliasing. The paper states that these simplifications preclude arbitrary rotations, non-planar relay surfaces, and strong secondary bounces (&&&^^^^4\4^^^^&&&).

These assumptions address several likely misconceptions. MAS does not claim unrestricted hidden-scene motion modeling; its motion component is rigid translation. It does not require purely static hidden scenes; instead, it treats motion as a shift of a canonical response. It is also not restricted to retroreflective targets in an absolute sense, since diffuse objects still work qualitatively, but the reported behavior includes lower SNR and larger ambiguity lobes (&&&^^^^4\4^^^^&&&).

The reported empirical validation is summarized below.

Task	Setup	Reported result
Static hidden-object reconstruction	$\tau$^^^^4\4^^^^ s handheld scans with smartphone^^^^^^^^4 LiDAR4^^^^^^^^	Quality matches research-grade phasor-field systems up to $\tau$^^^^4 LiDAR4^^^^–τ\tau^^^^4\4^^^^ cm resolution
Single-object ^^^^4 OR \4^^^^D tracking	$\tau$^^^^4 OR \4^^^^ Hz	Mean error $\tau$4 cm in a $\tau$5 m$\tau$6 volume
Multi-object tracking	Two retroreflective targets, and also human hand	Run-time $\tau$7 ms/frame on a GPU
Camera localization	Planar walls with no visual texture	Drift reduced from $\tau$8 cm to $\tau$9 cm over $v=(c\tau/2)^2$^^^^4\4^^^^ m translation

The hardware is described as an off-the-shelf smartphone^^^^^^^^4 LiDAR4^^^^^^^^ with approximately $v=(c\tau/2)^2$^^^^4 LiDAR4^^^^ SPAD pixels, a $v=(c\tau/2)^2$^^^^4\4^^^^ Hz frame rate, and an eye-safe laser (&&&^^^^4\4^^^^&&&). The paper further states that these experiments enable the first demonstrations of NLOS imaging, tracking, and localization on sub-^^^^^^^^4\4 LiDAR4\4\4^^^^ consumer^^^^^^^^4 LiDAR4^^^^^^^^ hardware with real-time on-device performance. This suggests that the principal significance of MAS lies less in a new inverse solver than in a measurement unification that makes burst fusion, tracking, and localization tractable on commodity time-of-flight hardware.

7. Relation to adjacent motion-based sampling models

MAS belongs to a broader class of models that use motion to resolve incomplete measurements, but its technical role is distinct. In the visual-motion literature, "Motion-based prediction is sufficient to solve the aperture problem" formulates a joint posterior over position and velocity, updated by Bayes–Markov filtering and implemented by Sequential Monte Carlo. There, temporally coherent features accumulate probability mass and incoherent local features are explained away, yielding a progressive solution of the classical aperture problem without explicit end-stopped filters (&&&^^^^4\4^^^^&&&). The connection to MAS is methodological rather than domain-specific: both models use motion-conditioned propagation and particle-based inference, but the latent quantities and sensors differ.

In Fourier ptychography, "Inverse Synthetic Aperture Fourier Ptychography" uses unknown target rotation to induce a spatially linear phase ramp that translates sampled patches of the object spectrum. Target motion thereby replaces illumination-angle changes to generate synthetic-aperture diversity, and a learned module estimates the induced $v=(c\tau/2)^2$^^^^4 OR \4^^^^-space coordinates from dual-plane measurements (&&&^^^^4 OR \4^^^^&&&). Relative to that framework, MAS does not operate in the object Fourier domain and does not rely on phase retrieval; instead, it samples a hidden-scene impulse response through a relay wall under confocal^^^^^^^^4 LiDAR4^^^^^^^^ geometry.

Taken together, these neighboring formulations indicate that motion-induced sampling is not a single algorithmic template but a recurring principle. In MAS, that principle takes the specific form of a light-cone-domain sampling model for NLOS imaging, in which hidden-scene shape, rigid translation, and camera pose are expressed as separable but coupled components of one forward operator (&&&^^^^4\4^^^^&&&).