Flying-Point Artifact in Depth & Astronomical Imaging

• Flying-point artifact is a phenomenon where ambiguous pixel assignments at depth discontinuities and single exposures produce spurious 3D points or transient signals.
• Multi-hypothesis methods like the Mixture-Density Ambiguity model mitigate these errors by predicting multiple depth hypotheses and selecting the most confident mode.
• Robust training and inference, along with PSF-based cleaning pipelines in astronomy, reduce boundary errors and effectively remove cosmic-ray-like artifacts.

The flying-point artifact is a class of spurious outputs that arise in both depth estimation and astronomical image analysis. In depth estimation, flying points are erroneous 3D points placed in empty space near surface boundaries due to ambiguous pixel assignments. In astronomical imaging, flying-point-like artifacts (e.g., cosmic rays, transient events) are extraneous signals localized to individual exposures or pixels. In both contexts, these artifacts reflect fundamental limitations of modeling or measurement, especially at boundaries or under incomplete data.

1. Definition and Origin of the Flying-Point Artifact

The flying-point artifact arises when single-hypothesis models must represent pixels whose receptive fields overlap distinct surfaces with disparate depths. At strong depth discontinuities, such as object boundaries, two surfaces are present: a foreground at depth $d_{\rm fg}$ and a background at $d_{\rm bg} \gg d_{\rm fg}$. Standard depth regressors, trained with $\ell_1$ or $\ell_2$ losses, are forced to output a unique value $d$ per pixel, resulting in a compromise between $d_{\rm fg}$ and $d_{\rm bg}$. The network thus predicts a spurious depth that does not correspond to any real surface, manifesting as a "flying point" in the empty space between foreground and background surfaces. This artifact also refers, by analogy, to "cosmic-ray-like" outliers in single-exposure astronomical images, where sub-PSF-scale transients (e.g., cosmic rays, hot pixels) introduce unphysical flux values that must be detected and corrected (Bian et al., 1 Jun 2026, Desai et al., 2016).

2. The Mixture-Density Ambiguity (MDA) Model for Depth Estimation

The Mixture-Density Ambiguity (MDA) framework remedies the core ambiguity underpinning flying-point artifacts in depth estimation. For each pixel $x$, the model predicts $K$ depth hypotheses $\{d_k(x)\}_{k=1}^K$, associated uncertainties $d_{\rm bg} \gg d_{\rm fg}$0, and mixture weights $d_{\rm bg} \gg d_{\rm fg}$1 ($d_{\rm bg} \gg d_{\rm fg}$2, $d_{\rm bg} \gg d_{\rm fg}$3). The predicted depth distribution at pixel $d_{\rm bg} \gg d_{\rm fg}$4 is

$d_{\rm bg} \gg d_{\rm fg}$5

where each Gaussian (or Laplacian, analogously) component can specialize to a surface mode. Mixture weights are softmaxed logits, and $d_{\rm bg} \gg d_{\rm fg}$6. This explicit multi-modal representation enables the model to retain both plausible depths at boundaries, avoiding the spurious averaging that produces flying points (Bian et al., 1 Jun 2026).

3. Inference and Decoding Strategies

At inference, standard practice demands a unique depth per pixel. Averaging mixture components reintroduces flying points by interpolation. MDA instead decodes via mode selection: for each pixel, it chooses the hypothesis with maximal self-density at its mean,

$d_{\rm bg} \gg d_{\rm fg}$7

Because $d_{\rm bg} \gg d_{\rm fg}$8, this procedure ensures that the depth map places points exactly on predicted surfaces. Alternative strategies (expectation, mixture-argmax) are less effective: taking the expectation (mean) re-creates flying points and increases boundary errors (Acc $d_{\rm bg} \gg d_{\rm fg}$9 100 mm), while mixture-argmax yields identical boundary scores but is $\ell_1$0 slower than mode-select (Bian et al., 1 Jun 2026).

4. Training Objective and Architectural Considerations

MDA is trained to minimize the negative log-likelihood (NLL) of the mixture over all training pixels $\ell_1$1:

$\ell_1$2

with per-component loss

$\ell_1$3

A small probability clamp prevents all weight from collapsing to one component. The architecture is modular and backbone-agnostic: MDA replaces the final regressor head in any depth backbone (e.g., DA3, VGGT) with a $\ell_1$4-channel mixture head reporting $\ell_1$5, $\ell_1$6 (log-precision), and $\ell_1$7 (mixture-logit) per pixel. Computational overhead for $\ell_1$8 is $\ell_1$9, maintaining $\ell_2$0 FPS on $\ell_2$1 inputs (Bian et al., 1 Jun 2026).

5. Quantitative Performance and Robustness

Empirical evaluations span boundary reconstruction, video depth, 3D multi-view reconstruction, and blur robustness. On NRGBD, 7Scenes, and HiRoom boundary clouds, MDA reduces mean predictor$\ell_2$2GT accuracy from $\ell_2$3 mm (DA3 backbone) and $\ell_2$4 mm (VGGT), a $\ell_2$5 reduction. Chamfer Distance (CD) improves correspondingly. MDA is $\ell_2$680$\ell_2$7 faster (33 FPS) than diffusion-based baselines (PPD/PPVD $\ell_2$81 FPS). On Sintel/Bonn/KITTI, MDA matches or slightly improves AbsRel and $\ell_2$91.25 metrics over DA3 and VGGT, outperforming PPD/PPVD. For 3D reconstruction, all metrics (accuracy, completeness, normal consistency) are preserved within $d$0. Simulated blur degradations ($d$1) show unimodal baselines degrade by $d$2 mm in accuracy at $d$3, while MDA degrades only by $d$4–$d$5 mm (Bian et al., 1 Jun 2026).

Backbone	Baseline Acc (mm)	MDA Acc (mm)	CD/Boundary
DA3	57	25	Canny cloud, NRBGD
VGGT	60	28	Canny cloud, NRBGD

Mode-selection and mixture-argmax inference yield optimal boundary scores; expectation (mean) recreates flying points. For $d$6, boundary metrics are stable.

6. Extensions: Transparent Objects and Sky Regions

MDA accommodates transparent objects by replacing the mixture softmax with independent sigmoids, allowing multiple components to activate at transparent pixels. Training then supervises heads on distinct surfaces with single-component NLLs and includes a weight-sum regularizer. Synthetic and real-world experiments show front-layer AbsRel improves from $d$7, multi-layer boundary Acc from $d$8 mm, and real-world ordering accuracy from $d$9. For unbounded sky, an extra fixed-Gaussian component with large mean/variance absorbs sky pixels. Pixels are classified as sky if $d_{\rm fg}$0. Sky IoU on Sintel ($d_{\rm fg}$10.95) matches or exceeds dedicated segmenters (DA3: 0.96) (Bian et al., 1 Jun 2026).

7. Flying-Point Artifacts in Astronomical Images and Correction Methods

Astronomical imaging introduces "cosmic-ray-like" flying-point artifacts—transient or single-exposure events such as cosmic rays and hot pixels removed by coadd-based image cleaning pipelines. The static-sky coadd is built by PSF homogenizing all single-epoch images and performing median coaddition. Source catalogs are fit to the coadd, then forward-modeled via the local PSF and WCS to predict clean, artifact-free images for each epoch. Deviations above threshold in the difference image

$d_{\rm fg}$2

indicate transients to mask. Two-pass expansion identifies faint extended wings. For sub-PSF-scale events (cosmic rays), masked pixels are replaced by a PSF-weighted interpolant over unmasked neighbors. Detection efficiency of non-imaged artifacts is $d_{\rm fg}$3100\%, with residual flux bias $d_{\rm fg}$4 and added noise $d_{\rm fg}$55–10\%. The workflow robustly removes sub-PSF artifacts and flags extended ones, provided sufficient multi-epoch coverage and high-fidelity PSF/catlog modeling (Desai et al., 2016). A plausible implication is the generalization of this approach to any survey with well-sampled seeing and robust PSF modeling.

Summary

The flying-point artifact, in both depth estimation and astronomical imaging, reflects a failure to accommodate multimodal or transient phenomena at boundaries or in single-hypothesis models. The Mixture-Density Ambiguity (MDA) method eliminates flying points in depth estimation by predicting multiple depth hypotheses and selecting surface-consistent modes, achieving substantial reductions in boundary error and computational overhead. Astronomical pipelines, using median coaddition and PSF-based modeling, achieve near-100\% detection and safe interpolation of flying-point-like artifacts, maintaining negligible bias and robust depth. Both cases exemplify the importance of explicitly modeling ambiguity and transients to overcome the limitations of single-hypothesis or single-epoch procedures (Bian et al., 1 Jun 2026, Desai et al., 2016).