BTCChat: Multimodal Temporal Change Analysis

• BTCChat is a multimodal framework for bi-temporal remote sensing change understanding, integrating dedicated change extraction and prompt augmentation modules.
• It employs a Change Extraction module that fuses spatial and temporal features using cosine similarity and convolution operations for refined change captioning.
• The system achieves state-of-the-art performance on change captioning and VQA benchmarks, supporting applications in urban monitoring, disaster assessment, and environmental analysis.

BTCChat is a multimodal LLM (MLLM) framework specifically designed to advance bi-temporal remote sensing change understanding—particularly the captioning and analysis of satellite image pairs taken at different times. Unlike previous approaches that process image pairs via naïve concatenation, BTCChat explicitly models temporal and spatial semantic change using a dedicated Change Extraction module and a Prompt Augmentation mechanism. The system retains both bi-temporal change captioning and single-image interpretation capabilities, and demonstrates state-of-the-art performance on established change captioning and visual question answering (VQA) benchmarks (Li et al., 7 Sep 2025).

1. Model Architecture

BTCChat adheres to the modern MLLM paradigm with an architecture composed of:

• Visual Encoder ($\mathcal{E}_v$): Processes input remote sensing imagery. For $k=2$ (bi-temporal), the input $I \in \mathbb{R}^{2 \times H \times W \times C}$ is encoded into $F \in \mathbb{R}^{2 \times L_V \times D_V}$, where $L_V$ is the number of patch tokens.
• Change Extraction Module: Enhances spatiotemporal feature extraction by explicitly modeling spatial and temporal differences.
• Multimodal Projector ($\mathcal{M}_m$): Compresses visual features into the word embedding space (typically a 1/4 downsampling), aiding semantic extraction and reducing computational load.
• Prompt Augmentation Mechanism: Uses a frozen base MLLM to generate contextual clues from the images, which are integrated with the user task prompt to enrich spatial reasoning.
• LLM Backbone: Receives the concatenated projected visual embeddings ($E_I$) and word embeddings ($E_L$) of the augmented prompt $P$, producing the final textual output $a = \mathcal{M}_l([E_I; E_L])$.

This structure supports both single-image and bi-temporal interactions, facilitating multi-tasking within remote sensing analytics.

2. Change Extraction Module

The Change Extraction (CE) module addresses the inadequacy of direct feature concatenation in prior models by:

• Spatial Enhancement ( $\mathcal{M}_{cs}$ ): Adds learnable positional embeddings to retain spatial location fidelity, reshaping visual features to a 2D grid ($\sqrt{L_V} \times \sqrt{L_V}$).
• Cosine Similarity Mapping: Calculates local patch similarity across the temporal pair:

$$f_\mathrm{cos} = \uparrow ( \mathrm{Cos}(f_1, f_2) )$$

where $\uparrow$ denotes a broadcast to match concatenated dimensions.

• Feature Fusion ( $\mathcal{M}_{cf}$ ): Concatenates $[f_1; f_2] + f_\mathrm{cos}$ and applies a stack of three 2D Conv layers with kernels [1×1, 3×3, 1×1], residual connections, and ReLU activations to yield enhanced features $F'$.

This design captures, fuses, and projects local spatiotemporal changes, preserving geometric and textural discontinuities essential for accurate change captioning.

3. Prompt Augmentation Mechanism

To exploit spatially detailed priors and enrich overall prompt signal without costly retraining, the Prompt Augmentation (PA) mechanism:

• Applies a fixed guiding prompt $P_g$ (e.g., “Please describe the remote sensing image(s) in detail”) to a frozen base model $\mathcal{M}_b$ with input image(s) $I$.
• Collects the base model’s generated context $P_c = \mathcal{M}_b(I, P_g)$ containing high-level visual descriptions.
• Combines $P_c$ with the task-specific user prompt $P_t$ via a fixed template to produce the final model input $P$:

$P = \mathrm{Template}(P_c, P_t)$

This context-rich prompt drives the LLM to stronger spatial reasoning and descriptive fidelity, without increasing training complexity.

4. Performance Metrics

BTCChat demonstrates substantial empirical improvement:

• Change Captioning: On the LEVIR-CC dataset, achieves a CIDEr-D of 139.12, with consistently higher BLEU-1, METEOR, and ROUGE-L scores compared to prior MLLMs and task-specific baselines.
• Visual Question Answering: Scores an average accuracy of 92.21% on RSVQA-LR; achieves 73.15% in zero-shot mode on RSVQA-HR.
• Ablation Studies: Removal of the CE module or PA mechanism degrades performance, confirming both are essential for optimal efficacy.

These metrics establish BTCChat as the state of the art in multi-temporal change captioning and VQA for remote sensing image pairs.

5. Applications

BTCChat’s ability to robustly model spatial and temporal change has immediate utility in:

• Urban Development Monitoring: Automatic description of infrastructure expansion, land use shifts, and construction activities over time.
• Disaster Assessment: Rapid, high-fidelity description of damage pre- and post-event (e.g., floods, fires, earthquakes), supporting emergency management and resource allocation.
• Environmental and Agricultural Change: Detection and captioning of gradual or abrupt biome, crop, or landscape changes to inform stakeholders in agriculture and conservation.
• General Single-Image Interpretation: The architecture retains single-image understanding, allowing its deployment in VQA and descriptive analytics with only one timestamp as input.

6. Visual-Semantic Alignment

BTCChat addresses the visual-semantic misalignment common in bi-temporal analysis through:

• Explicit Spatiotemporal Feature Fusion: The CE module’s use of local patchwise cosine similarity and convolutional fusion ensures that the model forms representations of change—not merely scene content—preserving both magnitude and location.
• Prompt Augmentation: By augmenting textual prompts with contextual clues derived from a frozen vision-LLM, BTCChat guides the LLM to map subtle local visual evidence directly to changes emphasized in generated descriptions.

The combined approach ensures accurate alignment between pixel-level change and high-level linguistic output, surpassing the descriptive performance of concatenation-based or single-modality MLLMs.

7. Comparative Position and Significance

BTCChat’s methodological advancements directly address the limitations of prior approaches in bi-temporal change understanding, which have generally failed to model temporal correlation and fine-grained spatial-semantic shifts. The integration of both advanced change extraction and prompt enrichment mechanisms, validated by superior performance across standardized benchmarks, positions BTCChat as a reference model for future work in multi-temporal remote sensing captioning and visual question answering (Li et al., 7 Sep 2025).