PoInit-of-View: Poisoning Initialization of Views Transfers Across Multiple 3D Reconstruction Systems

Abstract: Poisoning input views of 3D reconstruction systems has been recently studied. However, we identify that existing studies simply backpropagate adversarial gradients through the 3D reconstruction pipeline as a whole, without uncovering the new vulnerability rooted in specific modules of the 3D reconstruction pipeline. In this paper, we argue that the structure-from-motion (SfM) initialization, as the geometric core of many widely used reconstruction systems, can be targeted to achieve transferable poisoning effects across diverse 3D reconstruction systems. To this end, we propose PoInit-of-View, which optimizes adversarial perturbations to intentionally introduce cross-view gradient inconsistencies at projections of corresponding 3D points. These inconsistencies disrupt keypoint detection and feature matching, thereby corrupting pose estimation and triangulation within SfM, eventually resulting in low-quality rendered views. We also provide a theoretical analysis that connects cross-view inconsistency to correspondence collapse. Experimental results demonstrate the effectiveness of our PoInit-of-View on diverse 3D reconstruction systems and datasets, surpassing the single-view baseline by 25.1% in PSNR and 16.5% in SSIM in black-box transfer settings, such as 3DGS to NeRF.

• The paper introduces PoInit-of-View, which targets SfM initialization to induce transferable adversarial effects across diverse 3D reconstruction systems.
• The methodology injects imperceptible, geometry-aligned perturbations that disrupt keypoint matching and lead to failures in pose estimation and triangulation.
• Experimental results reveal significant drops in PSNR and SSIM, alongside increased LPIPS scores, demonstrating the attack’s efficacy under strict perceptual constraints.

Targeted Poisoning of Structure-from-Motion Initialization for Transferable 3D Reconstruction Attacks

Motivation and Problem Statement

The paper "PoInit-of-View: Poisoning Initialization of Views Transfers Across Multiple 3D Reconstruction Systems" (2604.16540) investigates the adversarial vulnerability of 3D reconstruction pipelines at their geometric initialization stage, specifically the Structure-from-Motion (SfM) module. SfM serves as the geometric core in most contemporary pipelines—including Multi-View Stereo (MVS), Neural Radiance Fields (NeRF), and 3D Gaussian Splatting (3DGS)—by providing camera poses and sparse 3D points. Most prior attack strategies have targeted downstream modules or operated by backpropagating gradients through the composite pipeline. This paper proposes that disrupting the multi-view geometric initialization yields highly transferable adversarial effects, destabilizing subsequent dense or implicit reconstruction algorithms regardless of architectural details.

Figure 1: Imperceptible perturbations are injected into input views, inducing cross-view inconsistency and propagating errors from SfM initialization through subsequent stages of 3D reconstruction.

Attack Methodology

The PoInit-of-View attack strategically injects visually imperceptible perturbations into carefully selected input images, aiming not for single-view distortion but for targeted cross-view gradient inconsistencies at the projections of corresponding 3D points. The approach manipulates local edge and texture structures so that keypoint descriptors (e.g., SIFT, SuperPoint) in the poisoned views diverge from their clean reference counterparts. This divergence disrupts spatial feature correspondence, undermining pose estimation and triangulation, and causing collapse of the geometric backbone for downstream rendering modules.

Figure 2: Framework overview of PoInit-of-View, with a proxy renderer providing cross-view gradients critical for optimizing the inconsistency loss.

The attack is formulated as a constrained projected gradient ascent problem, maximizing a differentiated cross-view inconsistency loss (CVIPLoss) subject to strict $\ell_\infty$-norm perceptual budgets. A differentiable proxy (e.g., 3DGS) models cross-view interactions and provides gradients. The optimization combines imperceptibility regularizers including SSIM and total variation to preserve visual realism.

Theoretical Analysis

The paper presents a formal theoretical foundation linking local gradient inconsistencies to exponential reduction in keypoint matching probability and global collapse of SfM pose graph. By bounding descriptor divergence via local Lipschitz continuity and analyzing RANSAC acceptance thresholds, it derives a condition: once cross-view inconsistency exceeds a threshold dictated by gradient and descriptor mismatch, the probability of accepting critical edges in the pose graph drops, causing severe under-constraint and geometric collapse.

Figure 3: Cross-view gradient inconsistency violating the epipolar constraint, leading to projection errors and mismatched correspondences.

Experimental Results

Extensive experiments on public 3D datasets (NeRF-Synthetic, Tanks and Temples, Mip-NeRF360) and multiple reconstruction pipelines (COLMAP, Instant NGP, Mip-Splatting) demonstrate the downstream-agnostic impact of attacking SfM initialization. Under a small perturbation budget ($\rho = 16/255$), PoInit-of-View induces average drops of 25.1% in PSNR and 16.5% in SSIM, surpassing single-view baselines by these margins in black-box transfer scenarios. LPIPS perceptual distance nearly doubles, evidencing view-inconsistent artifacts and geometric collapse. Registered images, triangulated keypoints, and total sparse 3D points steeply decline, with collapse ratios approaching 80–90%.

Figure 4: Qualitative comparison showing severe geometric distortion and collapse in poisoned reconstructions versus clean inputs across diverse pipelines and scenes.

Figure 5: In the "Auditorium" scene, poisoned COLMAP reconstruction fails to register the majority of cameras, even though apparent reprojection error is lower; geometry is unrecoverable.

The poisoned images remain nearly indistinguishable from clean views in pixel space (PSNR $>27$ dB, SSIM $\approx 0.86$, LPIPS $<0.21$), confirming high stealthiness.

Figure 6: Clean and poisoned images appear visually similar; amplified difference maps reveal only subtle perturbations.

Ablations show the importance of structured, geometry-aligned perturbations over random noise and highlight that increasing the perturbation budget escalates collapse rates.

Figure 7: Larger perturbations intensify degradation in registration and reconstruction, validating theoretical predictions.

Figure 8: Edge-aligned discrepancies induced by the attack distinguish it from random noise, disrupting SfM-critical feature structures.

Increasing CVIPLoss consistently predicts degradation in SfM stability, corroborating theoretical breakdown thresholds.

Figure 9: Registered images and triangulated keypoints collapse as cross-view inconsistency increases.

Implications and Future Research

This work establishes that the front-end geometric initialization in multi-view 3D reconstruction is intrinsically vulnerable to subtle, coordinated adversarial perturbations. The attack demonstrates strong transferability, destabilizing systems regardless of underlying representation (volumetric, implicit, explicit). Practically, this vulnerability affects safety-critical domains reliant on robust 3D memory—autonomous driving, surgical navigation, AR/VR localization, and robotic manipulation.

Theoretically, the analysis motivates new robustness criteria for multi-view consistency and descriptor repeatability. It also challenges the implicit assumptions underlying photometric and geometric initialization, and suggests that augmenting SfM with adversarial regularization or robust feature matching is necessary.

Potential future directions include:

• Developing cross-model generalizable defenses for geometric consistency.
• Designing robust pose optimization methods resilient to cross-view adversarial disruption.
• Investigating attack transferability across heterogeneous SfM implementations.
• Formulating multi-view statistical detection for imperceptible poisoning.

Conclusion

The PoInit-of-View attack exposes a fundamental, transferable vulnerability in multi-view 3D reconstruction pipelines, linked to their reliance on geometric initialization via SfM. The method achieves strong adversarial and stealthy disruption across diverse settings, validated both theoretically and empirically. Addressing this vulnerability necessitates a paradigm shift in geometric initialization robustness and multi-view adversarial defense.