---
title: Motion-Induced Aperture Sampling Model
url: https://www.emergentmind.com/topics/motion-induced-aperture-sampling-model
type: topic
---

# Motion-Induced Aperture Sampling Model

Searching arXiv for the exact topic and nearby related work to ground the article in current preprints.
arxiv_search(query="\"Motion Induced Sampling\" LiDAR", max_results=10)
arxiv_search(query="\"Motion-Induced Aperture Sampling\" OR \"Imaging Hidden Objects with Consumer LiDAR via Motion Induced Sampling\"", max_results=10)
Motion-Induced Aperture Sampling (MAS) is a measurement model for non-line-of-sight (NLOS) imaging with consumer LiDAR that unifies hidden-object shape, object motion, and camera motion in a single forward model. Introduced by Somasundaram et al. for smartphone-grade LiDAR, the model is formulated in the light-cone–transformed domain so that the hidden-scene impulse response becomes a 3D convolution, and it treats camera motion as a mechanism that sweeps a larger virtual aperture across the relay wall. Within that formulation, burst measurements from low-power, low-resolution, moving sensors can be fused for 3D reconstruction, single- and multi-object tracking, and camera localization using hidden objects [2605.17865].

## 1. Conceptual scope and terminological context

The MAS model addresses a specific NLOS imaging setting: a consumer LiDAR observes a visible relay wall while a hidden scene lies outside direct line of sight. The central claim of the model is that object shape, rigid object translation, and camera pose variation can be expressed jointly in one sampling equation, rather than treated as separate nuisance factors. In this construction, the hidden scene is represented by a canonical space–time impulse response (STIR), object motion shifts that STIR, and camera motion changes which wall locations are sampled at each frame. As the user moves the LiDAR, the camera extrinsics $\mathbf P_t$ change and the virtual aperture on the wall sweeps out a larger region [2605.17865].

A common source of confusion is the word *aperture*. In MAS, it refers to motion-induced sampling support on the relay wall in an NLOS imaging pipeline. This is distinct from the classical aperture problem in motion perception, where local velocity measurements of elongated contours are ambiguous and are resolved through motion-based predictive coding [1208.6471]. It is also distinct from motion-driven synthetic-aperture Fourier ptychography, where target rotation translates sampled patches of an object’s Fourier spectrum under fixed optics [2507.03733]. These neighboring usages are conceptually related in that motion increases inferential leverage, but the MAS model is specifically a confocal LiDAR measurement model for hidden-scene imaging [2605.17865].

## 2. Light-cone formulation of the canonical hidden-scene response

The MAS formulation is expressed in the light-cone–transformed domain, denoted by $\mathcal R$, where the hidden-scene impulse response becomes a 3D convolution. The canonical STIR is written as $\mathcal I(x,y,v)$ for a hidden object with volumetric albedo $\rho(x,y,z)$. Under the light-cone transform, depth $z$ is mapped to $v=z^2$, and time $\tau$ is mapped to $v=(c\tau/2)^2$. The forward model is

$$
\boxed{
\mathcal I(x,y,v)=\bigl[\mathcal R_z\{\rho\}\bigr](x,y,v)\ast_{x,y,v} h(x,y,v)
}
$$

with

$$
h(x,y,v)=\delta\!\bigl(x^2+y^2-v\bigr).
$$

Here, $\mathcal R_z\{\rho\}(x,y,v)$ is the remapping $v\leftarrow z^2$ of the object’s 3D albedo, $h$ is the 3D point-spread function in the light-cone domain, and $\ast_{x,y,v}$ denotes convolution over $(x,y,v)$ [2605.17865].

This representation is important because it isolates a time-independent canonical response. Once $\mathcal I(x,y,v)$ has been defined, subsequent frames need not recompute hidden-scene shape. Instead, later variation is introduced by shifting or sampling that canonical response. This suggests a decomposition in which geometry and dynamics are separated at the level of the forward model, with shape encoded once and motion handled through translation and sampling operators.

## 3. Rigid object motion and camera-induced sampling

Rigid object motion enters MAS as a translation of the canonical STIR. If the object undergoes a rigid translation $\boldsymbol\Delta_t=(\Delta x_t,\Delta y_t,\Delta v_t)$ at frame $t$, then by shift invariance of convolution,

$$
\mathcal I_t(x,y,v)=\mathcal I\bigl(x-\Delta x_t,\; y-\Delta y_t,\; v-\Delta v_t\bigr).
$$

The model explicitly states that no re-evaluation of shape is required—only a coordinate shift [2605.17865].

Camera sampling is then defined in confocal form. For a consumer LiDAR pixel at image coordinate $\mathbf u=(u_x,u_y)$ and time bin $v$, with intrinsics $\mathbf K$ and frame-dependent extrinsics $\mathbf P_t=[\mathbf R_t\mid \mathbf t_t]\in\mathbb R^{3\times 4}$, the continuous measurement is

$$
\boxed{
i_t(\mathbf u,v)=
\iint_{(x,y)\in\mathbb R^2}
\delta\!\bigl(\mathbf u-\mathbf K\,\mathbf P_t [x,y,0,1]^T\bigr)\;
\mathcal I\bigl(x-\Delta x_t,\; y-\Delta y_t,\; v-\Delta v_t\bigr)\; dx\,dy
}
$$

The Dirac term enforces that only the wall point $(x,y,0)$ projecting to pixel $\mathbf u$ contributes; $\mathcal I(\cdot)$ carries the shifted STIR, thereby encoding object shape and object motion; and camera motion enters through $\mathbf P_t$ [2605.17865].

This decomposition clarifies the model’s central unification. Canonical shape is contained in $\mathcal I$, rigid motion is a shift of that same tensor, and camera motion changes the wall sampling pattern. A plausible implication is that MAS can be read as a sampling-theoretic account of NLOS burst imaging: motion does not merely corrupt measurements, but changes which portions of a latent hidden-scene response are observed.

## 4. Multi-frame fusion and extended-aperture reconstruction

Because each LiDAR frame is noisy, low-resolution, and spatially cropped, MAS uses multi-frame fusion to estimate a higher-quality, extended-aperture STIR. The pipeline has three steps [2605.17865].

First, each frame is warped into a common world grid using the known camera pose. For each pixel $\mathbf u$, the pose $\mathbf P_t$ determines the world-wall coordinate $\bigl(x_t(\mathbf u),y_t(\mathbf u)\bigr)$, and the warp is written as a linear operator $\mathcal W_t$:

$$
i_t^{\mathrm W}(x,y,v)=\mathcal W_t\{\,i_t(\mathbf u,v)\,\}.
$$

Second, the warped frames are fused by weighted accumulation. Using per-frame weights $w_t$, for example proportional to the integrated signal strength, the fused STIR estimate is

$$
\boxed{
\widehat{\mathcal I}(x,y,v)=\sum_{t=1}^T w_t\, i_t^{\mathrm W}(x,y,v).
}
$$

Third, once $\widehat{\mathcal I}(x,y,v)$ is sufficiently dense on $(x,y)$ and has good signal-to-noise ratio, reconstruction of $\rho(x,y,z)$ proceeds via non-uniform backprojection or an LCT-inverse solver. The description includes a generic filtered-backprojection expression,

$$
\rho(x,y,z)\propto
\iiint
\widehat{\mathcal I}\!\Bigl(x+u,\; y+v,\; (x+u)^2+(y+v)^2\Bigr)\; k(u,v)\; du\,dv,
$$

where $k(u,v)$ is a known filter kernel [2605.17865].

For static scenes, the reconstruction problem is solved by discrete filtered backprojection on the fused $\widehat{\mathcal I}$. No iterative solver is required if sampling density is high enough. This places MAS close to burst-fusion imaging methods, but with the additional constraint that the fused quantity is not an ordinary image stack: it is a wall-parameterized, time-resolved hidden-scene impulse response.

## 5. Sequential inference for tracking and localization

MAS is not limited to static reconstruction. The same forward model supports sequential Bayesian estimation when the canonical hidden-scene response is known but motion or camera pose is unknown [2605.17865].

For single-object 3D tracking, the unknown state is the translation $\boldsymbol\Delta_t$. Let the photon-histogram measurement vector at frame $t$ be $\mathbf i_t\in\mathbb R^n$, and let $\hat{\mathbf i}(\boldsymbol\Delta)$ denote the rendering of the canonical $\mathcal I$ shifted by $\boldsymbol\Delta$ and sampled according to the MAS measurement equation. The posterior is

$$
P(\boldsymbol\Delta_t\mid \mathbf i_t,\boldsymbol\Delta_{t-1})
\propto
P(\boldsymbol\Delta_t\mid \boldsymbol\Delta_{t-1})\;
P(\mathbf i_t\mid \boldsymbol\Delta_t).
$$

Inference is implemented with a particle filter. Propagation samples

$$
\boldsymbol\Delta_t^{(k)}\sim \mathcal N(\boldsymbol\Delta_{t-1}^{(k)},\, r^2\mathbf I),
$$

and the particle weight is

$$
w_t^{(k)}=
\left(
\frac{
\mathbf i_t\cdot \hat{\mathbf i}(\boldsymbol\Delta_t^{(k)})
}{
\|\mathbf i_t\|\;\|\hat{\mathbf i}(\boldsymbol\Delta_t^{(k)})\|
}
\right)^\eta,
\qquad \eta>0.
$$

Particles are then resampled by importance sampling and normalized to equal weights [2605.17865].

For multi-object tracking with $M$ known shapes, the state becomes $\bigl(\boldsymbol\Delta_t^{(1)},\dots,\boldsymbol\Delta_t^{(M)}\bigr)\in\mathbb R^{3M}$, and the rendered measurement is the superposition of $M$ shifted STIRs. The same particle-filter steps apply, after which the particles are clustered, for example by $K$-means, to extract object positions [2605.17865].

For camera localization with a known static hidden shape, the unknown becomes the 2D camera translation $(t_x,t_y)$ parallel to the wall. The line-of-sight point cloud is first used to fix camera height and orientation, leaving only planar translation unknown. The NLOS signal from the known hidden object is then rendered under each hypothesized $(t_x,t_y)$ using the same measurement model, and a particle filter estimates camera position in real time [2605.17865].

## 6. Assumptions, limitations, and reported empirical performance

The MAS model is derived under a specific set of assumptions. The relay wall is planar at $z=0$ and infinitely Lambertian. The hidden object undergoes rigid translation with no rotations. Reflectance is approximated as retroreflective in confocal geometry; diffuse objects still work empirically but with lower signal-to-noise ratio. Noise is modeled as dominated by Poisson photon counts and sensor jitter, while timing quantization and dark counts are neglected in the core model. The formulation excludes interreflections and occlusions beyond the first virtual bounce. Camera intrinsics $\mathbf K$ and tracking of $\mathbf P_t$ via onboard IMU/SLAM are assumed known. Maximum translation per frame is assumed small enough to avoid aliasing. The paper states that these simplifications preclude arbitrary rotations, non-planar relay surfaces, and strong secondary bounces [2605.17865].

These assumptions address several likely misconceptions. MAS does not claim unrestricted hidden-scene motion modeling; its motion component is rigid translation. It does not require purely static hidden scenes; instead, it treats motion as a shift of a canonical response. It is also not restricted to retroreflective targets in an absolute sense, since diffuse objects still work qualitatively, but the reported behavior includes lower SNR and larger ambiguity lobes [2605.17865].

The reported empirical validation is summarized below.

| Task | Setup | Reported result |
|---|---|---|
| Static hidden-object reconstruction | $\sim 2$ s handheld scans with smartphone LiDAR | Quality matches research-grade phasor-field systems up to $1$–$2$ cm resolution |
| Single-object 3D tracking | $30$ Hz | Mean error $\sim 5$ cm in a $1$ m$^3$ volume |
| Multi-object tracking | Two retroreflective targets, and also human hand | Run-time $\sim 20$ ms/frame on a GPU |
| Camera localization | Planar walls with no visual texture | Drift reduced from $\sim 10$ cm to $\sim 2$ cm over $5$ m translation |

The hardware is described as an off-the-shelf smartphone LiDAR with approximately $100$ SPAD pixels, a $30$ Hz frame rate, and an eye-safe laser [2605.17865]. The paper further states that these experiments enable the first demonstrations of NLOS imaging, tracking, and localization on sub-\$100 consumer LiDAR hardware with real-time on-device performance. This suggests that the principal significance of MAS lies less in a new inverse solver than in a measurement unification that makes burst fusion, tracking, and localization tractable on commodity time-of-flight hardware.

## 7. Relation to adjacent motion-based sampling models

MAS belongs to a broader class of models that use motion to resolve incomplete measurements, but its technical role is distinct. In the visual-motion literature, "Motion-based prediction is sufficient to solve the aperture problem" formulates a joint posterior over position and velocity, updated by Bayes–Markov filtering and implemented by Sequential Monte Carlo. There, temporally coherent features accumulate probability mass and incoherent local features are explained away, yielding a progressive solution of the classical aperture problem without explicit end-stopped filters [1208.6471]. The connection to MAS is methodological rather than domain-specific: both models use motion-conditioned propagation and particle-based inference, but the latent quantities and sensors differ.

In Fourier ptychography, "Inverse Synthetic Aperture Fourier Ptychography" uses unknown target rotation to induce a spatially linear phase ramp that translates sampled patches of the object spectrum. Target motion thereby replaces illumination-angle changes to generate synthetic-aperture diversity, and a learned module estimates the induced $k$-space coordinates from dual-plane measurements [2507.03733]. Relative to that framework, MAS does not operate in the object Fourier domain and does not rely on phase retrieval; instead, it samples a hidden-scene impulse response through a relay wall under confocal LiDAR geometry.

Taken together, these neighboring formulations indicate that motion-induced sampling is not a single algorithmic template but a recurring principle. In MAS, that principle takes the specific form of a light-cone-domain sampling model for NLOS imaging, in which hidden-scene shape, rigid translation, and camera pose are expressed as separable but coupled components of one forward operator [2605.17865].

Source: https://www.emergentmind.com/topics/motion-induced-aperture-sampling-model