- The paper proposes a Smart Transfer framework that leverages a DINOv3 vision foundation model for efficient post-earthquake damage mapping using minimal annotations.
- It employs pixel-wise clustering and distance-penalized triplet loss to ensure robust domain generalization and accurate delineation of building damage.
- Empirical results show improved F1 scores and mIoU, demonstrating that FM-based adaptation outperforms traditional methods even in data-scarce scenarios.
Smart Transfer: Vision Foundation Models for Rapid Building Damage Mapping with Post-Earthquake VHR Imagery
Problem Statement and Motivation
Accurate, rapid assessment of building damage in the immediate aftermath of an earthquake is a critical operational priority for disaster response agencies. Traditional field-based surveys and classic EO pipelines are labor intensive and unable to scale to the required spatial or temporal extent. Even existing GeoAI approaches, while powerful, typically require copious region-specific labeled data and thus lack the scalability necessary for ad hoc, global deployment in diverse urban contexts. The increasing availability of high-resolution EO imagery, combined with the evolving capabilities of large vision Foundation Models (FMs), motivates the development of frameworks that can generalize rapidly across geographic domains, with minimal training or annotation effort.
The Smart Transfer framework addresses these challenges directly. It proposes a sophisticated approach for leveraging the strong generalization and transfer learning properties of FMs in remote sensing—especially DINOv3 ViT-L, pretrained on large-scale remote sensing data—to enable automated, scalable, and robust building damage mapping from post-disaster very-high-resolution (VHR) satellite imagery.
Data and Experimental Design
The study is grounded in a rigorous experimental context: nine diverse urban regions from the 2023 Türkiye-Syria earthquake are selected, spanning a gradient of seismic intensity and urban morphologies. The authors assemble a comprehensive dataset: post-disaster Pléiades 1A/B VHR imagery at 0.3 m GSD, open-access building footprint polygons from GlobalBuildingAtlas, and a high-quality benchmark of manually and expertly annotated building damage masks from the KATE-CD dataset (further curated and extended). The full dataset comprises 119,014 image tiles and 288,567 building outlines, with exhaustive ground-truth annotations for 1,340 tiles. This design provides a challenging and representative testbed for cross-region domain adaptation and generalization.
Methodology
Vision Foundation Model Backbone
The core encoder is the DINOv3 ViT-L/16, pretrained in a self-supervised fashion on the SAT493M remote sensing corpus. The pretraining objective integrates global semantic consistency (DINO loss), local representation reconstruction (iBOT MIM loss), and geometric alignment (Gram loss), yielding a backbone that encodes both robust global context and fine-grained local structure. The input images are tessellated into patches, and both pixel-level and patch-level embeddings are used for downstream adaptation.
Smart Transfer Strategies
The main methodological contribution is the design of two complementary FM adaptation strategies:
- Pixel-wise Clustering (PC): Clusters pixel embeddings from the FM encoder to construct class-specific prototypes (damaged vs non-damaged). Each pixel is pseudo-labeled by proximity to a prototype, and a masked BCE loss is computed over reliable assignments (agreement with manual mask). PC encourages intra-class compactness and inter-class separability in the embedding space, aligning FM features with true morphological damage classes.
- Distance-Penalized Triplet (DPT): Operates at the patch level, enforcing spatial autocorrelation by constructing triplets of patch embeddings—anchor, positive (same label), negative (different label). A penalty term based on spatial adjacency (Euclidean patch-center distance) modulates the loss: semantically inconsistent yet spatially adjacent triplets are penalized more strongly. This regularization enforces spatial coherence in the damage mapping, improving boundary delineation and robustness to domain shift.
Both are deployed via lightweight decoders attached to the frozen FM encoder; decoder parameters are trained with a combination of segmentation loss, PC, and DPT losses.
Domain Generalization Settings
Two rigorous transfer learning benchmarks are used:
- Leave One Domain Out (LODO): Models are trained on all but one region and tested on the held-out region, emulating practical rapid deployment to unseen disaster zones.
- Specific Source Domain Combination (SSDC): Measures the impact of training with varied combinations of regions, focusing on source diversity in terms of damage severity and geographic location.
The framework is tested with varying amounts of labeled data (down to "few-shot" or near zero-shot scenarios), and key metrics include F1, accuracy, recall, precision, and mean Intersection-over-Union (mIoU) for both pixel and instance-level damage detection.
Results
Supervised FS Baseline
DINOv3 achieves a substantial performance gain over classical CNN architectures (ResNet-18, ResNet-152, YOLO-like baselines), with overall F1 = 0.62 and mIoU = 0.68 in the fully supervised regime. Notable is its strong recall and stable generalization across highly heterogeneous urban morphologies, validating the selection of a VFM backbone over conventional, task-specific architectures.
- LODO: PC delivers the highest mIoU (0.69 ± 0.03) and F1 (0.64 ± 0.06), with consistent improvements in recall and balanced precision, demonstrating that prototype-level feature alignment yields robust transfer across domain shifts. DPT and combined PC+DPT show regionally variable but competitive results. The methods notably excel with limited annotations, maintaining strong accuracy even at low training ratios (20–40%).
- SSDC: DPT becomes more valuable when the source domains encompass multiple, geographically and structurally diverse damaged regions. The highest overall mIoU (0.71 ± 0.03) and F1 (0.67 ± 0.07) are observed when the training data covers all major axes of damage diversity across the study area. This empirically supports that patch-level spatial regularization benefits from diversity in both morphology and severity.
- Few-shot Learning: Minimal gains are achieved by fine-tuning the FM encoder in few-shot regimes; most transfer benefit accrues from decoder adaptation, indicating the FM backbone is already highly reusable as a general-purpose EO feature extractor.
Data Efficiency and Qualitative Assessment
The Smart Transfer framework shows substantial resilience to reduced annotation availability, outperforming the FS baseline in data-scarce settings. Qualitative mapping in dense, severely damaged urban sub-regions confirms improved coherence in damage classification and better handling of true positives/negatives. Notably, high precision can be traded for higher recall depending on operational requirements (critical in humanitarian use cases).
Limitations and Future Work
Despite strong cross-region generalization, several limitations are acknowledged:
- Current approach is limited to post-event VHR satellite imagery and binary/graded damage classification. Finer-grained interpretation demands complementary data (Street View, social media) and multimodal V(L)M architectures.
- Evaluation is intra-event; truly event-agnostic transfer (cross-quake or cross-hazard) is not yet validated.
- Integration with autonomous, real-time GIS workflows and anticipatory risk modeling is still an open challenge.
- Further development is needed to close the gap between automated post-disaster assessment and actionable, localized disaster resilience planning.
Potential future directions include extension towards multimodal VFM architectures, unsupervised or self-supervised pre-disaster vulnerability mapping, and integration with autonomous agent-based workflows for real-time disaster response.
Conclusion
This paper provides a principled, effective framework for leveraging Vision FMs in post-disaster building damage mapping from VHR satellite imagery. Smart Transfer, through its strategic use of pixel-wise clustering and patch-level spatial regularization, substantially improves FM adaptation for domain-shifted, data-scarce, and operationally urgent disaster mapping. Empirical results underscore the dominance of FM-based adaptation over traditional approaches—especially in geodiverse and annotation-limited settings.
The framework demonstrates both the practical feasibility of scaling GeoAI to support rapid humanitarian response and reinforces the theoretical value of representation learning in bridging the disaster resilience gap for vulnerable regions. By releasing code and data, the work provides a benchmark for continued advancement in automated, generalizable EO-based disaster mapping.
Citation: "Smart Transfer: Leveraging Vision Foundation Model for Rapid Building Damage Mapping with Post-Earthquake VHR Imagery" (2604.02627).