WeatherPrompt: Robust Geo-Localization

Updated 3 July 2026

WeatherPrompt is a multi-modality framework employing weather-invariant representation learning via dynamic fusion of visual and textual features to address cross-view geo-localization.
It utilizes training-free weather reasoning with chain-of-thought prompting and synthetic weather augmentation to generate detailed, open-set weather descriptions.
Empirical results show significant performance gains under diverse weather conditions, validating its effectiveness for drone-based retrieval and visual matching tasks.

WeatherPrompt

WeatherPrompt refers to a class of multi-modality learning frameworks and benchmarks designed for all-weather robust visual geo-localization, particularly in the context of drone-based cross-view retrieval and matching tasks. Unlike conventional geo-localization methods that are severely hampered by adverse weather perturbations (e.g., rain, fog, night conditions), WeatherPrompt exploits weather-invariant representation learning by leveraging dynamic fusion of visual and semantic (textual) information to achieve high recall rates under diverse and unseen weather scenarios (Wen et al., 13 Aug 2025).

1. Problem Domain: All-Weather Cross-View Geo-Localization

WeatherPrompt addresses the domain of cross-view geo-localization, wherein the task is to match an oblique drone-view query image to its corresponding geo-referenced satellite image. This operation is foundational for autonomous navigation, search and rescue, and environment monitoring by aerial vehicles. The principal challenge arises from weather-induced domain gaps: standard convolutional or transformer-based models are empirically brittle under image corruption from atmospheric perturbations.

Conventional CVGL frameworks fail in two key aspects under weather: (i) they generalize poorly to continuous, open-set weather conditions outside limited annotated categories; (ii) they exhibit suboptimal disentanglement of scene structure from transient weather artifacts, preventing robust matching (Wen et al., 13 Aug 2025).

2. WeatherPrompt Framework: Multi-Modality and Weather Reasoning

The core WeatherPrompt paradigm (Wen et al., 13 Aug 2025) comprises three foundational elements:

Training-free Weather Reasoning: WeatherPrompt leverages large vision–LLMs (LVLMs), such as Qwen-2.5-VL, with chain-of-thought (CoT) prompting to synthesize detailed, per-image weather descriptions. Synthetic weather augmentation (e.g., fog, snow, rain, night) is applied to drone images. Each weather-perturbed instance is subjected to a multi-phase CoT reasoning pipeline, producing an open-set, structured caption reflecting both global atmospheric visibility and local meteorological cues.
Dynamic Gating Multimodal Fusion: A frozen LVLM backbone encodes both the visual (augmented drone image) embedding $f_I \in \mathbb{R}^{B \times D}$ and the structured text embedding $f_T \in \mathbb{R}^{B \times D}$ . WeatherPrompt employs a weather-driven channel gating module that computes a gate vector $g \in (0,1)^{B \times D}$ via MLPs. The final fused representation is $f_{fuse} = g \odot f_I + (1-g) \odot f_T$ , which adaptively emphasizes visual or semantic features conditional on weather content.
Cross-Modal Objectives: The network is supervised with (a) an image-text contrastive loss (ITC), (b) an image-text matching loss (ITM) with hard-negatives, and (c) a localized alignment loss (LA) for spatial region alignment. The resulting joint optimization encourages the representations of identical scenes—differing only by weather overlays—to be mapped closer together in the learned metric space.

3. Synthetic Weather Benchmarking Protocol

WeatherPrompt establishes a standardized, open-set corruption suite by synthetically applying ten weather conditions to drone images in standard datasets such as University-1652 and SUES-200. The conditions include:

Fog, Rain, Snow
Fog + Rain, Fog + Snow, Rain + Snow (compound events)
Night (under-exposure/dark), Over-exposure (bright)
Wind (motion blur)

Augmented images are paired with their corresponding uncorrupted satellite views for cross-view retrieval. Each weather variant is annotated by the LVLM-generated textual description, ensuring that the model can exploit fine-grained, continuous weather semantics rather than relying on discrete or pseudo weather categories. The WeatherPrompt protocol provides a robust stress test for cross-view retrieval algorithms (Tran et al., 12 May 2026).

4. Model Architecture and Training

Backbone: WeatherPrompt utilizes a pre-trained XVLM as the fixed vision–language module: Swin-Transformer (vision) and BERT (text).
Fusion Head: The dynamic gating head serves to reweight and merge visual and textual features, producing a weather-invariant embedding for downstream retrieval/classification.
Supervision: The optimization combines
- Image-Text Contrastive Loss:
$L_{ITC} = -\frac{1}{2B} \sum_{i=1}^B [ \log p_{I \to T}^{(i)} + \log p_{T \to I}^{(i)} ]$ - Image-Text Matching Loss:

$L_{ITM} = -\frac{1}{3B} \sum_{k=1}^{3B} [ y_k \log p_k + (1 - y_k) \log(1-p_k) ]$ - Localized Alignment Loss:

$L_{LA} = \mathbb{E}[1 - IoU(l^j, \hat l^j) + \|l^j - \hat l^j\|_1 ]$ - Classification Head (geo-label prediction): Cross-entropy loss
End-to-End Loss: $L_{total} = L_{ITC} + L_{ITM} + L_{LA} + L_{CE}$

WeatherPrompt is trained on synthetic weather-augmented datasets, using an SGD optimizer, momentum 0.9, and standard data augmentations (Wen et al., 13 Aug 2025).

5. Empirical Performance and Robustness

WeatherPrompt achieves state-of-the-art cross-view retrieval performance under both clean and corrupted conditions. Noteworthy results:

On University-1652 (drone → satellite):
- Prior SOTA (LPN): 64.33% R@1 (mean, 10 weather styles)
- WeatherPrompt: 77.14% (+12.81 pp)
- Under night: +13.37% recall gain over SOTA; under fog + snow: +18.69% gain.
On SUES-200:
- Drone → Satellite: 62.52% R@1 for WeatherPrompt vs. 52.02% for MuSe-Net.
On real-world videos (dark/rain/fog): R@1 = 44.44% for WeatherPrompt vs. 22.22% for prior methods.

Ablation studies confirm (i) the superiority of dynamic gating (+1.41 pp over static gates), and (ii) monotonic improvement as CoT prompting steps are increased (6-step CoT yields +2.04 pp over direct captioning). Performance in unseen compound corruptions (e.g., Fog+Snow) remains robust.

WeatherPrompt redefines the robustness evaluation paradigm for cross-view geo-localization by moving beyond categorical label-based weather augmentation toward open-set, textually described weather semantics. The explicit disentanglement of scene and weather—amplified by the dynamic fusion framework—demonstrably mitigates the empirical fragility of SOTA visual matching backbones under real-world noise and occlusion (Wen et al., 13 Aug 2025).

The WeatherPrompt weather-corruption suite and its protocol have already served as comparative benchmarks for new model architectures such as SkyPart (Tran et al., 12 May 2026), where further advances in prototype-based semantic part discovery and uncertainty weighting yield additional improvements specifically on WeatherPrompt tasks.

7. Future Directions and Open Challenges

Despite WeatherPrompt’s advances in robustness and generalization, open challenges remain:

Generalization to longitudinal, geographical, and seasonal weather patterns without synthetic augmentation pipelines.
Integration of physics-informed models for causal weather reasoning (as in physics-informed simulation frameworks (Qian et al., 27 Jun 2026)).
Efficient extension to real-time, resource-constrained deployment on edge platforms.
Development of unsupervised or continual-learning versions for rapidly evolving or rarely observed weather conditions.

WeatherPrompt thus constitutes both a practical toolset for current robust geo-localization and a platform for continued research into open-set model generalization, scene-weather disentanglement, and all-weather visual intelligence (Wen et al., 13 Aug 2025, Tran et al., 12 May 2026).