PoInit-of-View: Poisoning SfM Initialization

• PoInit-of-View is a poisoning attack on multi-view 3D reconstruction that targets the structure-from-motion (SfM) initialization by introducing cross-view gradient inconsistencies.
• It employs a differentiable proxy and PGD-based optimization under ℓ∞ constraints to craft subtle perturbations that reduce keypoint detection, feature matching, and pose estimation.
• Empirical evaluations show significant drops in PSNR, SSIM, and LPIPS along with degraded SfM statistics, demonstrating transferable disruption across reconstruction systems.

Searching arXiv for the named paper and closely related viewpoint-initialization / reconstruction references.

Search result 1: "PoInit-of-View: Poisoning Initialization of Views Transfers Across Multiple 3D Reconstruction Systems" (Wang et al., 17 Apr 2026) — matches the topic and provides the primary basis for the article.

Search result 2: "Viewpoint Consistency in 3D Generation via Attention and CLIP Guidance" (Zhang et al., 2024) — relevant background on viewpoint consistency as a systems issue in 3D generation.

Search result 3: "Chain-of-View Prompting for Spatial Reasoning" (Zhao et al., 8 Jan 2026) — relevant background on viewpoint selection as an explicit reasoning component. PoInit-of-View is a poisoning attack on multi-view 3D reconstruction systems that targets Structure-from-Motion (SfM) initialization rather than a particular downstream reconstructor. It perturbs a subset of input views so as to introduce cross-view gradient inconsistencies at projections of corresponding 3D points, with the goal of degrading keypoint detection, feature matching, pose estimation, and triangulation. The central claim is that SfM is the geometric core shared by many widely used reconstruction pipelines, so attacking it yields transferable degradation across systems such as COLMAP-based SfM+MVS, Instant NGP, and Mip-Splatting (Wang et al., 17 Apr 2026).

1. Definition and threat model

PoInit-of-View studies a data-poisoning setting for multi-view reconstruction. Starting from a clean image set $\mathcal{I}$, the attacker produces poisoned images $\tilde{\mathcal{I}}$ under an $\ell_\infty$ constraint,

$$\|\tilde{\mathcal{I}}-\mathcal{I}\|_\infty \le \varepsilon.$$

The objective is not to make the inputs obviously corrupted, but to cause the reconstruction pipeline to fail internally: fewer images are registered, fewer keypoints are triangulated, the sparse geometry degrades, and downstream rendered novel views become poor (Wang et al., 17 Apr 2026).

The paper frames this as a black-box attack on the victim SfM service. The attacker does not need access to the target system’s code, descriptors, or thresholds. A local differentiable proxy is used for optimization, while the deployed victim remains off-the-shelf. This design is meant to maximize transferability: instead of overfitting perturbations to the idiosyncrasies of a specific NeRF or 3D Gaussian Splatting implementation, the attack focuses on the shared initialization stage that many systems inherit.

A plausible implication is that the method redefines the attack surface of 3D reconstruction. Rather than treating reconstruction quality as a property of the downstream renderer alone, PoInit-of-View treats the initialization of views as the point where small perturbations can induce system-wide failure.

2. SfM initialization as the geometric core

The paper characterizes SfM as the geometric core because it performs keypoint detection, descriptor extraction, feature matching, geometric verification, relative pose estimation, triangulation, and bundle adjustment (Wang et al., 17 Apr 2026). In the notation of the paper, the input views are

$\{I_i\}_{i=1}^N,$

with keypoints $p_{i,k} \in \mathbb{R}^2$ and descriptors $\phi(p_{i,k}) \in \mathbb{R}^d$. Correspondences are checked by the epipolar constraint

$p_{j,\ell}^{\top} \mathbf{F}_{ij}\, p_{i,k} = 0,$

and, when intrinsics are known, the essential matrix is

$$\mathbf{E}_{ij} = \mathbf{K}_j^{\top} \mathbf{F}_{ij}\mathbf{K}_i.$$

Given correspondences and poses, SfM refines global poses $\{(\mathbf{R}_i,\mathbf{t}_i)\}$ and 3D points $\tilde{\mathcal{I}}$0 by minimizing a robust reprojection objective,

$\tilde{\mathcal{I}}$1

The paper also emphasizes calibrated two-view geometry through

$\tilde{\mathcal{I}}$2

PoInit-of-View argues that corruption at this stage propagates into later modules. If keypoints and matches are weakened, pose graph construction becomes unstable, triangulation yields fewer 3D points, and downstream systems receive poor camera poses or sparse geometry. This is why the paper presents SfM initialization as a more universal leverage point than attacks that directly optimize against a single renderer’s photometric loss.

3. Cross-view inconsistency objective and proxy-guided optimization

The attack’s central mechanism is cross-view gradient inconsistency. For corresponding 3D points that project to two views, the paper assumes that clean local neighborhoods should exhibit similar gradient structure. Using Sobel gradients

$\tilde{\mathcal{I}}$3

the attack maximizes the discrepancy between a clean reference view $\tilde{\mathcal{I}}$4 and a poisoned target view $\tilde{\mathcal{I}}$5,

$\tilde{\mathcal{I}}$6

The intended causal chain is explicit: local gradient inconsistency degrades keypoint repeatability and descriptor agreement; feature matching quality drops; epipolar verification and RANSAC become unstable; relative pose estimation and triangulation deteriorate; and the downstream reconstruction collapses (Wang et al., 17 Apr 2026).

The full surrogate objective is

$\tilde{\mathcal{I}}$7

Here the CVI term drives the attack, the SSIM term preserves perceptual similarity, and the TV term keeps perturbations smooth. The practical optimization uses PGD with the update

$\tilde{\mathcal{I}}$8

Because COLMAP/SfM is not differentiable end-to-end, the method introduces a differentiable 3DGS proxy. The proxy is used only to compute gradients; evaluation remains on unmodified off-the-shelf systems. In all experiments, 3DGS is used as the proxy model, and the proxy is refreshed every $\tilde{\mathcal{I}}$9 iterations. The reported implementation uses perturbation budget $\ell_\infty$0, step size $\ell_\infty$1, inner PGD steps per adversarial view of 15, outer optimization iterations of 1000, poisoning ratio $\ell_\infty$2, and loss weights $\ell_\infty$3, $\ell_\infty$4, and $\ell_\infty$5 on an NVIDIA A100 (40GB) (Wang et al., 17 Apr 2026).

A plausible implication is that PoInit-of-View is best understood as a pairwise cross-view attack, not a single-image attack. Its optimization target is the relative consistency that SfM depends on, rather than the isolated quality of any one view.

4. Theoretical connection to correspondence collapse

The paper supplements the attack with a stylized theory that links cross-view inconsistency to descriptor divergence, reduced inlier ratios, and SfM failure (Wang et al., 17 Apr 2026). The first assumption is clean cross-view gradient consistency: $\ell_\infty$6 The second assumption states local Lipschitz continuity of descriptors: $\ell_\infty$7 A third assumption bounds the probability that descriptor deviations remain below a matching threshold.

Under these assumptions, Lemma 1 states that if

$\ell_\infty$8

then descriptor distance must satisfy

$\ell_\infty$9

Lemma 2 then bounds the expected inlier ratio $$\|\tilde{\mathcal{I}}-\mathcal{I}\|_\infty \le \varepsilon.$$0 by

$$\|\tilde{\mathcal{I}}-\mathcal{I}\|_\infty \le \varepsilon.$$1

The central result, Theorem 1, gives an SfM breakdown condition. If each critical view pair is poisoned such that

$$\|\tilde{\mathcal{I}}-\mathcal{I}\|_\infty \le \varepsilon.$$2

then the global SfM optimization fails with probability

$$\|\tilde{\mathcal{I}}-\mathcal{I}\|_\infty \le \varepsilon.$$3

where

$$\|\tilde{\mathcal{I}}-\mathcal{I}\|_\infty \le \varepsilon.$$4

and the threshold

$$\|\tilde{\mathcal{I}}-\mathcal{I}\|_\infty \le \varepsilon.$$5

marks the onset of correspondence collapse on critical edges. In plain terms, once gradient inconsistency exceeds a tolerance tied to descriptor stability, matching probability drops enough that important pose-graph edges disappear.

This theoretical analysis does not provide a defense, but it makes the mechanism explicit: SfM failure is modeled as the consequence of correspondence collapse induced by structured cross-view appearance perturbation.

5. Empirical performance and transfer across reconstruction systems

The evaluation spans NeRF-Synthetic, Tanks and Temples, and Mip-NeRF360, with both internal SfM statistics and downstream rendering metrics. The primary downstream targets are COLMAP-based SfM+MVS, Instant NGP, and Mip-Splatting, while optimization is always carried out with a 3DGS proxy (Wang et al., 17 Apr 2026).

On Tanks and Temples, the paper reports the following clean-to-poisoned degradation:

Target system	Clean	Poisoned
COLMAP (SfM+MVS)	PSNR 11.92, SSIM 0.436, LPIPS 0.606	PSNR 8.96, SSIM 0.372, LPIPS 0.693
Instant NGP	PSNR 21.62, SSIM 0.712, LPIPS 0.340	PSNR 16.24, SSIM 0.605, LPIPS 0.440
Mip-Splatting	PSNR 23.93, SSIM 0.833, LPIPS 0.166	PSNR 17.63, SSIM 0.684, LPIPS 0.327

These results correspond to average drops of -24.8% PSNR and -14.7% SSIM for COLMAP, -24.8% PSNR and -15.0% SSIM for Instant NGP, and -26.0% PSNR and -18.4% SSIM for Mip-Splatting. The paper emphasizes that lower PSNR and SSIM, and higher LPIPS, indicate a stronger attack. It also states that PoInit-of-View surpasses the single-view baseline by 25.1% in PSNR and 16.5% in SSIM in black-box transfer settings, specifically in the comparison on Tanks and Temples with Mip-Splatting.

The internal SfM statistics support the same conclusion. On Tanks and Temples, clean SfM reports 93.5% registered images, 73.6k triangulated keypoints, and 3.07M 3D points, whereas PoInit-of-View reduces these to 24.3%, 13.7k, and 0.41M, respectively. On NeRF-Synthetic, the corresponding values change from 98.7% / 52.3k / 2.11M to 28.5% / 12.4k / 0.32M; on Mip-NeRF360, from 96.2% / 61.7k / 2.64M to 26.9% / 11.8k / 0.34M. Collapse ratios rise above 0.80 across all three datasets. A striking example is the Auditorium scene, where cameras drop from 298 to 7 and points from 73k to 9k.

The perturbations remain visually subtle: the paper reports PSNR: $$\|\tilde{\mathcal{I}}-\mathcal{I}\|_\infty \le \varepsilon.$$6 dB, SSIM: $$\|\tilde{\mathcal{I}}-\mathcal{I}\|_\infty \le \varepsilon.$$7, and LPIPS: $$\|\tilde{\mathcal{I}}-\mathcal{I}\|_\infty \le \varepsilon.$$8 between clean and poisoned images. Ablations further indicate that the cross-view inconsistency term is the critical ingredient. A photometric-only loss does not collapse SfM, whereas removing SSIM or TV weakens both stealthiness and attack strength. The paper also states that attack strength increases as $$\|\tilde{\mathcal{I}}-\mathcal{I}\|_\infty \le \varepsilon.$$9 increases, with a sharp collapse once $\{I_i\}_{i=1}^N,$0, and near-total failure at $\{I_i\}_{i=1}^N,$1.

6. Position within viewpoint-centric research

PoInit-of-View belongs to a broader line of work in which viewpoint is treated as a first-class systems variable rather than a nuisance parameter. In text-to-3D generation, “Viewpoint Consistency in 3D Generation via Attention and CLIP Guidance” argues that the gap between the expected viewpoint in the 3D optimization loop and the actual viewpoint encoded by the diffusion model output is a root cause of the Janus problem, and it introduces Attention and CLIP Guidance (ACG) to reduce that mismatch (Zhang et al., 2024). In embodied question answering, “Chain-of-View Prompting for Spatial Reasoning” turns viewpoint choice into part of the reasoning process through coarse-grained view selection and fine-grained view adjustment, treating anchor views as the starting point for exploration (Zhao et al., 8 Jan 2026).

PoInit-of-View is distinct from these systems because it inverts the role of viewpoint control. Instead of using viewpoint alignment to improve generation or reasoning, it weaponizes initialization of views as a transferable attack surface. This suggests a general principle: when a pipeline depends on view-conditioned geometry, any persistent gap between the system’s assumed view relationships and the effective evidence induced by its inputs can become either a performance bottleneck or a vulnerability.

The paper’s limitations are correspondingly specific. Its theory relies on assumptions such as cross-view gradient consistency and local Lipschitz continuity of descriptors. Its optimization depends on a differentiable proxy and a poisoning ratio of $\{I_i\}_{i=1}^N,$2. Its transfer claims are strongest for pipelines that rely on SfM-like initialization. The paper does not develop a defense, but it points toward robust feature matching, pose optimization with adversarial regularization, and more generally defenses for multi-view geometry as plausible directions (Wang et al., 17 Apr 2026).