DEFOM Foundation Model Overview

• DEFOM Foundation Model is a deep learning architecture that fuses pretrained monocular depth priors with recurrent stereo refinement and explicit scale updates to achieve robust performance across diverse domains.
• The model employs a novel fusion of cost-volume construction and multiscale feature encoders to precisely estimate depth in challenging scenes such as UAV forestry and material fracture simulations.
• Specialized variants balance inference speed and accuracy, enabling real-time embedded applications and setting a new standard for cross-domain stereo matching and scientific computing.

The DEFOM Foundation Model encompasses a modern class of deep learning models that integrate large-scale vision foundation architectures, initially applied to monocular or multimodal perception tasks, with domain-specific modules to achieve robust generalization in stereo matching and material fracture prediction. DEFOM and its derivatives, notably DEFOM-Stereo, are distinguished by their incorporation of pretrained monocular depth priors, recurrent refinement modules, and explicit scale adjustment mechanisms. This integration yields stable, accurate performance across out-of-distribution domains, including vegetation-dense UAV imagery and heterogeneous materials under complex loading for fracture tasks. DEFOM establishes itself as the gold standard in cross-domain stereo matching for UAV forestry and as a unifying architecture for mesh-based physical simulation.

1. Architectural Foundations and Model Variants

DEFOM models are characterized by a modular architecture that leverages large-scale vision transformer backbones (typically based on DINOv2-ViT), often in conjunction with Dense Prediction Transformer (DPT) decoders for dense output tasks. In stereo matching contexts, the monocular depth prior module—trained as a foundation model (e.g., Depth Anything V2)—is injected into a recurrent stereo refinement backbone. Key DEFOM-Stereo components include:

• Monocular Prior Module: Produces an initial per-pixel depth (or disparity) using global scene semantics, mitigating ambiguities in textureless regions. Trained with a scale- and shift-invariant objective, e.g.,

$$L_{mono}(\theta) = \min_{\alpha, \beta} \frac{1}{N} \sum_{i=1}^N \left| \alpha z_i + \beta - z^{\dagger}_i \right|$$

• Combined Feature and Context Encoders: Multiscale features from DPT are aligned and fused with conventional CNN encoders for both matching and context branches.
• Recurrent Disparity Refinement: Disparity is initialized from the monocular depth output and iteratively updated via a combination of delta-update and scale-update modules, following the update rule:

$$d_n(u, v) = s_n(u, v) \cdot d_{n-1}(u, v) + \Delta d_n(u, v)$$

where $s_n(u, v)$ is a learned local scale adjustment and $\Delta d_n(u, v)$ is the recurrent update from cost-volume features (Jiang et al., 16 Jan 2025).

• Cost-Volume Construction: Standard local 1D correlation at multiple resolutions; pyramid-based in DEFOM-Stereo.

Five prominent DEFOM-Stereo variants have been developed to balance inference speed and depth accuracy in embedded and real-time UAV applications, ranging from the full ViT-L (depth, slowest) to the highly compressed PruneNano (fastest, least accurate) (Lin et al., 27 Mar 2026).

2. Training Procedures and Loss Formulation

DEFOM-Stereo and related DEFOM models are pretrained on large synthetic datasets (e.g., Scene Flow / FlyingThings3D, 35,454 training pairs) exploiting the abundance of dense ground-truth disparity maps. The main training regime comprises:

• Supervised Disparity Loss: L₁ or smooth-L₁ end-point error (EPE) averaged over valid pixels:

$L = \frac{1}{N} \sum_{i=1}^N |d_i - d^{gt}_i|$

• Augmentation: Strong photometric and geometric augmentation to encourage generalization (brightness/contrast, scale, rotation, erasure).
• Domain Adaptation: Absent in primary DEFOM evaluations; all cross-benchmark results are strictly zero-shot (Lin et al., 3 Dec 2025).
• Optimization: AdamW (β₁=0.9, β₂=0.999, weight decay=1e−4), initial LR 4e–4 with exponential decay.

In fracture prediction, the model is trained first on surrogate rule-based simulations, followed by fine-tuning on phase-field data, employing task-aligned MSE or L₁ losses depending on output type (Marcato et al., 30 Jul 2025).

3. Cross-Domain Generalization and Benchmark Performance

DEFOM achieves state-of-the-art cross-domain robustness suitable for both structured urban scenes and unstructured, vegetation-dense forestry imagery:

• Stereo Matching Zero-Shot Benchmarks (Lin et al., 3 Dec 2025, Lin et al., 27 Jan 2026):

| Model | ETH3D EPE | KITTI12 EPE | KITTI15 EPE | Middlebury EPE | Avg. Benchmark Rank | CV(EPE) | |----------|-----------|-------------|-------------|----------------|--------------------|---------| | DEFOM | 0.35 | 0.84 | 1.04 | 4.65 | 1.75 | 0.58 | | BridgeDepth | 0.23 | 0.83 | 1.07 | 20.03 | 2.5 | 0.73 | | IGEV++ | 0.36 | 1.20 | — | 6.77 | >2 | >1 |

DEFOM demonstrates the lowest coefficient of variation across datasets, indicating unmatched stability and absence of catastrophic failure.

• Vegetation Scenes and UAV Forestry: Qualitative evaluations confirm DEFOM produces smooth, artifact-free disparities, accurately localizes thin branches under strong occlusion and illumination variation, and is chosen as gold-standard pseudo-ground-truth for real-world vegetation datasets (e.g., Canterbury Tree Branches, 5,313 pairs) (Lin et al., 27 Jan 2026).
• Material Fracture Prediction: The multimodal DEFOM model generalizes to unseen materials (titanium: MAE ≈ 0.03 after one shot; concrete: ~20 samples to reach parity) and simulation types, achieving downstream MAE reductions (e.g., FDEM MAE=0.0218, ~100× speedup over CPU-MPI) (Marcato et al., 30 Jul 2025).

4. Practical Deployments and Variants for Embedded Systems

Specialized DEFOM-Stereo variants are crafted to balance accuracy and real-time computation on constrained hardware (e.g., NVIDIA Jetson Orin):

Model	FPS (Jetson)	EPE (px)	Depth MAE (cm)	δ₁ (%)	Backbone	Conv Layers
ViT-S	2.2	1.74	23.40	95.90	DINOv2 ViT-S	~66
PrunePlus	3.3	5.87	64.26	87.59	DINOv2 ViT-S	~14
PruneNano	8.5	13.06	112.16	68.96	DINOv2 ViT-S	~3

DEFOM-PrunePlus is identified as the optimal real-time UAV guidance solution (3.3 FPS, MAE_z = 64.26 cm at 2 m range), offering a practical trade-off between depth accuracy and latency for high-speed, closed-loop control scenarios. Full-capacity ViT-S serves as the reference for maximum achievable accuracy (Lin et al., 27 Mar 2026).

5. Mechanisms for Improved Robustness and Generalization

DEFOM achieves superior out-of-distribution generalization via several architectural and training innovations:

• Monocular Depth Priors: Coverage of non-textured, ambiguous regions by learning single-view geometric cues from large-scale datasets.
• Fusion with Stereo Cost Volumes: Blending monocular predictions with learned stereo features fosters artifact-free, consistent outputs at occlusion boundaries.
• Scale Update Modules: Explicit per-pixel or global scale corrections resolve the metric-scale ambiguity inherent in monocular priors, improving error resilience across datasets with varying disparity ranges.
• Augmentation Diversity: Pretraining with diverse lighting and geometric conditions imparts robustness against real-world scene variability.
• Multimodal Task Fusion (in fracture prediction): LLM-based text encoding and cross-attention modules allow flexible adaptation to new simulation regimes without architectural modification (Marcato et al., 30 Jul 2025).

6. Limitations and Future Directions

Limitations:

• Opaque Internal Structure: DEFOM is primarily evaluated as a black box, with limited architectural disclosure, constraining research on transparency and reproducibility.
• Inference Efficiency: Full-capacity models exhibit lower inference rates on embedded hardware; aggressive compression results in significant accuracy drop.
• Application Scope: While DEFOM excels at 2D extensional fracture and outdoor vegetation stereo, extension to shear, compression, 3D, or uncertainty quantification remains open (Marcato et al., 30 Jul 2025).
• Ground-truth Validation: In dense vegetation scenes, lack of LiDAR ground-truth means performance is established solely through cross-domain consistency and visual assessment.

Future research avenues identified include interpretability (attention map analysis), integration of physics constraints for extrapolation beyond training regimes, per-pixel uncertainty estimation, light-weight deployment optimization, and domain adaptation to close sim-to-real gaps in fielded UAV systems (Lin et al., 27 Jan 2026, Lin et al., 3 Dec 2025).

7. Impact and Outlook

DEFOM inaugurates a new paradigm in vision and scientific computing models, serving as the reference pseudo-ground-truth generator for UAV-based forestry datasets, a unifying surrogate for material fracture simulation, and a test-bed for efficient real-time stereo matching on embedded systems. By combining large-scale pretraining, sophisticated fusion of monocular and stereo cues, and systematic benchmarking, DEFOM establishes both a strong empirical and methodological foundation for future domain-agnostic generalist models in scientific vision and simulation (Lin et al., 3 Dec 2025, Lin et al., 27 Mar 2026, Marcato et al., 30 Jul 2025, Lin et al., 27 Jan 2026, Jiang et al., 16 Jan 2025).