Emergent Multi-View Geometry Through Self-Distillation

Abstract: Over a century ago, Henri Poincaré argued that a motionless observer cannot acquire the notion of space. Yet, most visual representation learning methods operate on individual images, while those that leverage multiple views rely on RGB reconstruction, entangling geometry with appearance. We propose Poincar3, a self-supervised method that learns representations from multiple views through self-distillation instead of RGB reconstruction. We combine masked patch and image-level distillation with a teacher that observes additional views, enabling training from scratch without explicit 3D supervision. Poincar3 outperforms both previous single and multi-view self-supervised approaches such as DINOv3, MuM, and Muskie on correspondence estimation, camera pose estimation, and 3D reconstruction. Using a lightweight Poincaré adapter, we also find that our learned features encode camera motion more accurately than existing self-supervised representations.

• The paper introduces Poincar3, a multi-view self-distillation model that achieves exceptional zero-shot geometric correspondence without explicit 3D supervision, outperforming baseline models such as Muskie and Dino v3.
• Performance evaluations show Poincar3's strong correspondence accuracy on benchmark datasets, such as 83.7% PCK@25 on ScanNet with attention matching, which is higher than other evaluated models.
• Poincar3’s advantages include robustness to limited training budgets and effective geometric feature transfer capability, but it also points out limitations like optimization fragility and assumptions about scene coherence in training sequences.

Problem setting and central claim

“Emergent Multi-View Geometry Through Self-Distillation” (2609.39227) addresses a specific weakness of contemporary visual SSL: single-image objectives can produce strong semantic features, but they do not directly impose consistency under viewpoint changes. Existing multi-view SSL methods such as CroCo, MuM, and Muskie use cross-view RGB reconstruction, which encourages geometric reasoning but also requires representations to encode view-dependent appearance. The paper asks whether multi-view geometric structure can instead emerge from a DINO/iBOT-style teacher–student objective without RGB reconstruction, correspondence labels, camera poses, or explicit 3D supervision.

The proposed answer is Poincar3, a 650M-parameter multi-view ViT trained from scratch on unlabeled image sequences. Its central claim is that self-distillation can produce representations with zero-shot geometric correspondence and camera-motion structure, and that this objective can outperform both single-view SSL and prior multi-view reconstruction objectives. The claim is unusually strong because the model is not trained to predict depth, pose, optical flow, RGB values, or correspondence maps; geometric behavior is evaluated only after pretraining.

The motivation is well aligned with the structure of the downstream tasks. A representation intended for pose estimation, correspondence, or reconstruction should identify scene elements that remain stable across views while discounting appearance changes. RGB reconstruction is not an ideal proxy for this requirement because the target itself varies with illumination, occlusion, texture, and view-dependent effects. Poincar3 instead transfers teacher representations across masked and augmented views, using the multi-view transformer to communicate information between frames.

Multi-view self-distillation

Poincar3 receives a sequence of $M$ images depicting the same scene. The student processes a masked, photometrically augmented subset of these views, while the EMA teacher receives the corresponding unmasked views and, crucially, $T$ additional frames. The additional teacher frames are not directly supervised; they expand the teacher’s geometric context through cross-view attention.

The loss has three principal components:

1. Masked patch distillation: student patch predictions are matched to teacher predictions only at masked patch locations, following iBOT.
2. Image-level distillation: student and teacher [CLS] predictions are aligned independently for each frame, following DINO.
3. KoLeo and Sinkhorn–Knopp regularization: these prevent representational collapse and promote a distributed feature space.

The objective is approximately

$$\mathcal{L} = \mathcal{L}_{\mathrm{patch}} + 0.5\mathcal{L}_{\mathrm{global}} + 0.1\mathcal{L}_{\mathrm{KoLeo}}.$$

The image-level term is not a minor auxiliary loss. The ablation results indicate that it is the principal stabilizing mechanism that redirects multi-view distillation toward geometric rather than merely appearance-preserving features. The teacher’s additional views provide a second, distinct source of improvement: they allow the teacher to construct a more complete scene representation while the student is required to predict the teacher’s outputs from less information.

Architecturally, Poincar3 uses a ViT-L image encoder followed by a 12-layer multi-view transformer decoder with alternating frame-wise and global attention. RoPE is used within frame-wise attention, and register attention supports longer sequences. Training uses sequences of 2–24 views, images resized to $256\times256$, 400K AdamW steps, and an EMA decay of $0.999$. The reported training cost is three days on eight H200 GPUs.

The training design is summarized below.

Figure 1: The EMA teacher processes full image sequences, including additional views, while the masked student is trained using image-level and masked-patch distillation.

The asymmetry between teacher and student is essential. A conventional multi-view extension of DINO-style training performs poorly in the paper’s ablations, and the authors report that local/global crop strategies inherited from single-image SSL are inappropriate for this setting. Full images preserve the spatial evidence needed for cross-view matching, while extra teacher views provide information that cannot be recovered from a single frame.

Emergent correspondence representations

The strongest evidence for emergent geometry comes from zero-shot correspondence estimation. The evaluation samples eight-view sequences and queries patches in one image. Correspondences are obtained either by nearest-neighbor matching in feature space or by selecting the maximum activation in the multi-view attention maps. No correspondence labels or attention supervision are used during pretraining.

Poincar3 produces the best reported results among the compared SSL and feed-forward geometry models. Feature nearest-neighbor matching reaches PCK@25 values of 79.5% on ScanNet and 68.7% on NAVI, compared with 70.1% and 64.7% for Muskie, and 66.9% and 60.2% for MuM. Attention matching is stronger still, reaching 83.7% on ScanNet and 74.5% on NAVI at PCK@25, with PCK@50 values of 94.9% and 86.5%, respectively.

Method	ScanNet PCK@25	NAVI PCK@25
DINOv3	56.8	59.6
Muskie	70.1	64.7
MuM	66.9	60.2
Poincar3, feature NN	79.5	68.7
Poincar3, attention	83.7	74.5

These results substantiate the paper’s principal claim more directly than downstream finetuning does: the frozen representation itself supports dense multi-view tracking. The attention result is particularly notable because it demonstrates that the model’s cross-view interaction pattern, rather than only its final patch embedding, contains highly accurate geometric correspondences.

Figure 2: Attention-derived tracks recover corresponding patches across views without correspondence labels or explicit attention supervision.

The qualitative tracks show that Poincar3 generally follows the same physical scene points across frames, whereas RGB-reconstruction features yield more diffuse and appearance-sensitive responses. This supports the authors’ interpretation that the objective suppresses photometric variation and emphasizes cross-view identity.

Figure 3: Poincar3’s predicted multi-view tracks remain close to ground-truth trajectories across an image sequence.

The paper also reports that correspondence accuracy is distributed throughout the network. This contrasts with supervised feed-forward reconstruction models, whose later-layer features can become less suitable for generic patch matching. The observation matters for transfer: geometric information is not confined to a specialized output head and can therefore support multiple downstream decoders.

Feed-forward reconstruction and pose estimation

Poincar3 is evaluated in three transfer regimes: training only pose and depth heads over frozen features, training a transformer over the frozen backbone, and finetuning the full model. Across these settings, it improves relative pose estimation and point-cloud prediction over DINOv3, MuM, and Muskie.

With only heads trained, Poincar3 obtains relative-pose AUC@30 values of 51.9% on RE10K, 48.7% on ScanNet++, and 67.5% on MegaDepth. With an additional transformer, these increase to 62.2%, 52.1%, and 69.5%. Full finetuning yields 68.2%, 67.0%, and 74.6%, respectively. Point-cloud results likewise improve, particularly on ETH3D, where the full model reaches 0.81 mm accuracy and 0.80 normal consistency.

The implication is that Poincar3 is not merely a correspondence specialist. Its representations are also useful as initialization for feed-forward geometric prediction, especially when the downstream adaptation budget is limited. The paper reports that Poincar3 reaches more than 65.0% AUC@30 on RE10K after only 10K finetuning steps at a low learning rate.

The comparisons are consequential because the baselines include both single-view SSL and multi-view models. Poincar3’s advantage is therefore not attributable solely to exposure to multiple images; the objective and its teacher–student asymmetry are necessary to obtain the reported transfer behavior.

Camera-motion structure and the Poincaré adapter

To assess whether the representation encodes camera motion in a structured form, the paper fits a lightweight per-scene Poincaré adapter. Given frame features, the adapter maps feature differences to the relative $\mathrm{SE}(3)$ twist between camera poses. The evaluation uses 20 held-out ScanNet++ scenes and reports $R^2$ on held-out frame pairs.

Without multi-view input, Poincar3 obtains an average $R^2$ of only 0.053, close to DINOv3’s 0.046. With multi-view processing, Poincar3 rises to 0.098, compared with 0.081 for MuM and 0.061 for Muskie. It also achieves the largest fraction of scenes with positive predictive power, 55.7%, and the largest fraction exceeding $R^2>0.3$, 7.1%.

Method	Multi-view input	Average $R^2$	$T$0	$T$1
DINOv3	No	0.046	42.5%	0.9%
Muskie	No	0.042	39.7%	0.4%
Muskie	Yes	0.061	46.3%	3.6%
MuM	Yes	0.081	51.7%	6.5%
Poincar3	No	0.053	42.4%	2.0%
Poincar3	Yes	0.098	55.7%	7.1%

These results should be interpreted as evidence of decodable camera-motion structure, not as proof that the embedding is globally equivariant to $T$2. The adapter is trained separately for each scene, and the reported metric measures predictive decodability after nonlinear adaptation. Nevertheless, the multi-view gain is consistent with the correspondence results: the representation becomes more organized with respect to scene motion when the network processes multiple views jointly.

Figure 4: A lightweight Poincaré adapter exposes the alignment between Poincar3 feature displacements and camera trajectories.

Ablations and compute efficiency

The ablations isolate the contribution of the proposed components under a fixed architecture, data distribution, and compute budget. Multi-view DINOv2 reaches PCK@25 values of 49.7% on ScanNet and 35.9% on NAVI. Removing local crops and using a multi-view iBOT-style objective raises these values to 55.7% and 46.4%. Adding image-level distillation increases them to 66.7% and 58.1%; allowing the teacher to see additional views raises them again to 70.2% and 61.5%. Scaling to the final Poincar3 configuration produces 83.7% and 74.5%.

Figure 5: Correspondence performance improves steadily during self-distillation rather than exhibiting the dense-feature degradation reported for some single-view SSL systems.

The ablation establishes a cumulative rather than substitutive design logic. Masked patch prediction alone is insufficient; the global loss and teacher context each provide independent improvements. The authors also report that the VGGT-$T$3 SSL MSE objective fails when trained from scratch, which is an important negative result: prediction-based geometric SSL cannot necessarily be transferred directly from a supervised reconstruction model to an annotation-free setting.

Poincar3’s compute efficiency is another central claim. The authors state that it uses approximately 100 times less compute than DINOv3 while achieving stronger geometric performance. In the compute–accuracy comparison, Poincar3 reportedly matches DINOv3 after one day on eight H200 GPUs.

Figure 6: Poincar3 reaches DINOv3-level correspondence performance substantially earlier in training under the reported compute comparison.

The comparison is informative but should not be treated as a universal scaling law. The models differ in objective, data mixture, architecture, and target capabilities. The result establishes an empirical Pareto advantage for the reported experimental configuration, not that multi-view self-distillation is intrinsically 100 times more compute-efficient in all regimes.

Limitations and open questions

The paper explicitly reports several limitations. Poincar3 prioritizes geometric performance over semantic performance, so it should not be regarded as a general replacement for large single-image semantic encoders. Its scale is also much smaller than DINOv3’s, and the compute comparison therefore does not compare equally scaled systems.

Optimization is fragile. Performance degrades when the EMA coefficient or weight decay deviates from the selected values, and the learning-rate adaptation to batch size must be handled carefully. This is a substantive limitation because the proposed objective adds multi-view attention and asymmetric teacher context to an already sensitive self-distillation framework. The paper leaves open whether better normalization, collapse prevention, or teacher parameterization can reduce this sensitivity.

The training sequences are also selected using handcrafted frame sampling. Consequently, the method assumes that sampled frames depict sufficiently overlapping views of a static or approximately static scene. Internet videos can violate this assumption through scene cuts, dynamic objects, low overlap, and camera motion unrelated to rigid-scene geometry. The paper does not establish how performance changes under systematic violations of these conditions.

Finally, the Poincaré analysis depends on a per-scene adapter and therefore demonstrates decodability rather than a canonical, scene-independent coordinate system. An open question is whether the same representation supports a shared $T$4 decoder across scenes, or whether the observed structure is primarily local to each scene’s feature manifold.

Conclusion

Poincar3 demonstrates that multi-view geometric representations can be learned from unlabeled image sequences using self-distillation rather than RGB reconstruction or explicit 3D supervision. Its defining ingredients are masked patch distillation, image-level alignment, full-image processing, and a teacher with access to additional views. The resulting features support zero-shot correspondence estimation, camera-motion decoding, relative pose estimation, and point-cloud prediction, with particularly strong correspondence scores of 83.7% PCK@25 on ScanNet and 74.5% on NAVI using attention matching.

The paper’s main contribution is therefore methodological and empirical: multi-view geometry need not be specified through pixel reconstruction or geometric labels when the teacher–student information asymmetry is designed to make cross-view consistency the most useful solution to the distillation objective. The remaining questions concern optimization stability, robustness to imperfect video sampling, semantic transfer, and whether the learned geometric structure can be decoded without scene-specific adaptation.