RS-HyRe-R1: A Hybrid Reward Mechanism to Overcome Perceptual Inertia for Remote Sensing Images Understanding

Abstract: Reinforcement learning (RL) post-training substantially improves remote sensing vision-LLMs (RS-VLMs). However, when handling complex remote sensing imagery (RSI) requiring exhaustive visual scanning, models tend to rely on localized salient cues for rapid inference. We term this RL-induced bias "perceptual inertia". Driven by reward maximization, models favor quick outcome fitting, leading to two limitations: cognitively, overreliance on specific features impedes complete evidence construction; operationally, models struggle to flexibly shift visual focus across tasks. To address this bias and encourage comprehensive visual evidence mining, we propose RS-HyRe-R1, a hybrid reward framework for RSI understanding. It introduces: (1) a spatial reasoning activation reward that enforces structured visual reasoning; (2) a perception correctness reward that provides adaptive quality anchors across RS tasks, ensuring accurate geometric and semantic alignment; and (3) a visual-semantic path evolution reward that penalizes repetitive reasoning and promotes exploration of complementary cues to build richer evidence chains. Experiments show RS-HyRe-R1 effectively mitigates "perceptual inertia", encouraging deeper, more diverse reasoning. With only 3B parameters, it achieves state-of-the-art performance on REC, OVD, and VQA tasks, outperforming models up to 7B parameters. It also demonstrates strong zero-shot generalization, surpassing the second-best model by 3.16%, 3.97%, and 2.72% on VQA, OVD, and REC, respectively. Code and datasets are available at https://github.com/geox-lab/RS-HyRe-R1.

• The paper introduces RS-HyRe-R1, a framework that tackles perceptual inertia in RL-trained remote sensing vision-language models.
• It integrates spatial reasoning, task-specific perception, and visual-semantic path evolution rewards to enforce deep, diversified reasoning processes.
• Experiments demonstrate significant improvements in REC, OVD, and VQA tasks, achieving robust performance in both in-domain and zero-shot cross-domain settings.

RS-HyRe-R1: A Hybrid Reward Mechanism to Overcome Perceptual Inertia for Remote Sensing Image Understanding

Introduction

The paper "RS-HyRe-R1: A Hybrid Reward Mechanism to Overcome Perceptual Inertia for Remote Sensing Images Understanding" (2604.17504) systematically addresses the issue of perceptual inertia in RL-trained remote sensing vision-LLMs (RS-VLMs). Perceptual inertia, as defined in this work, is the RL-induced tendency for models to overfit to prominent local features in remote sensing imagery (RSI), causing shallow reasoning and poor generalization, especially in multi-task RS scenarios.

By constructing and integrating reward signals that operate at both the process and outcome levels, the RS-HyRe-R1 framework establishes a unified, robust, and flexible interpretation protocol for diverse RS tasks—specifically, Referring Expression Comprehension (REC), Open Vocabulary Object Detection (OVD), and Visual Question Answering (VQA). The approach demonstrably achieves leading performance in both in-domain and zero-shot, cross-domain settings, with computational efficiency due to a 3B parameter backbone vastly outperforming larger models on a range of SOTA benchmarks.

Figure 1: Overview of the RS-HyRe-R1 framework, illustrating the hybrid reward flow and task unification pipeline.

Problem Statement: Perceptual Inertia in RL-trained RS-VLMs

Remote sensing imagery interpretation is particularly sensitive to superficial optimization because domain-relevant features are spatially diffuse, semantically complex, and often occluded by noise or irrelevant context. Standard RL reward schemes, predominantly relying on outcome-level correctness (ORMs), incentivize shortcut learning: the model collapses its attention onto the most salient cues for rapid reward acquisition, ignoring global and fine-grained evidence chains. This leads to two principal limitations: (1) at the cognitive level, a deficit in constructing comprehensive reasoning chains; (2) at the executive level, poor adaptability and localization drift across tasks.

The paper formalizes this class of RL-induced failure as "perceptual inertia," distinct from reward sparsity or mode collapse. Eliminating perceptual inertia necessitates integrated feedback that not only evaluates task success but also explicitly regulates the reasoning process.

Methodology

Unified RS Task Environment

RS-HyRe-R1 frames REC, OVD, and VQA within a shared RL environment. Only 1,600 samples are required for fine-tuning, emphasizing the importance of reward design over brute-force dataset expansion.

Group Relative Policy Optimization (GRPO)

GRPO is the backbone RL algorithm, eliminating value networks and using group sampling to stabilize optimization via relative advantage estimation. This enables efficient, high-variance exploration, critical for supporting reward signals at both the global (output) and local (process) levels.

Hybrid Reward Mechanism

RS-HyRe-R1’s reward is a weighted sum of three terms:

• Spatial Reasoning Activation Reward: Enforces explicit reasoning chains with structural tags, functioning as both an output and supervision constraint.
• RS-task Perception Correctness Reward: Task-adaptive anchors ensure that bounding box localization and semantic alignment are properly reinforced for each task, using IoU, set-based F1, and normalized text match metrics.
• Visual-Semantic Path Evolution Reward: The core innovation, this reward penalizes response path redundancy. It uses intra-group embedding similarity to force the model to diversify its reasoning chains, actively discouraging shortcut solutions and mitigating perceptual inertia.

Figure 2: Qualitative comparisons; RS-HyRe-R1 displays superior spatial localization (REC) and small-object sensitivity (OVD) compared to all baselines.

Experimental Validation

Main Results

REC: The model achieves [email protected] of 51.36% and [email protected] of 32.25% on VRSBench-test, outperforming the previous best (Geo-R1-REC, [email protected] 49.60%) despite having fewer parameters.

OVD: On NWPU VHR-10-val, RS-HyRe-R1 attains mAP@[0.5:0.95] 0.2600 (vs. Geo-R1-OVD’s 0.1887) and [email protected] 0.5225, a substantial increase over all prior models including those with ORMs or PRMs.

VQA: Pass@1 reaches 59.87% on RSVQA-LR, exceeding RL baselines by 9-14%.

Figure 3: RS-HyRe-R1 produces responses of 350–400 tokens, more than double those of baseline ORMs-RL, indicating deeper multi-hop visual reasoning.

Zero-Shot and Cross-Domain Generalization

In zero-shot transfer, such as VRSBench-REC → DIOR-RSVG, RS-HyRe-R1 maintains a >2% edge over GeoChat and other high-capacity models, and similar margins are seen on OVD and VQA transfer tasks. This validates that the model learns semantically robust, dataset-agnostic visual-language correspondences rather than overfitting to spurious domain-specific cues.

Training Stability and Evolution

Figure 4: The RL reward curve displays monotonic ascent and stability, confirming the hybrid reward signal’s efficacy and robustness.

Ablation studies provide further insight. Removing the visual-semantic path evolution reward induces premature convergence on shallow features, with subsequent plateauing; omitting the perception correctness reward catastrophically degrades all metrics; and dropping the spatial reasoning constraint causes volatile, unstable learning, evidencing the necessity of all three reward streams.

Figure 5: Ablation shows only the complete model (with all reward terms) achieves optimal, stable and robust performance across all RS tasks.

Implications and Future Directions

The hybrid reward formulation in RS-HyRe-R1 demonstrates the critical importance of process-aware RL signals for vision-LLMs, particularly in domains characterized by feature sparsity and task heterogeneity. The work highlights that outcome-only constraints are insufficient to produce genuinely interpretable, robust, and transferable reasoning policies.

This blueprint is not fundamentally restricted to RSI; the paradigm is extensible toward general multimodal or multi-sensor tasks, and to other domains where shortcut solutions prevail. Beyond improving interpretability and robustness, the technique presents new avenues for efficient, small-model RL fine-tuning, minimizing annotation cost through intelligent reward engineering.

Potential future work includes leveraging the hybrid reward strategy for multi-modal sensor fusion (RGB, multi-spectral, SAR), scaling to larger, more diverse datasets, and advancing the theoretical understanding of process-level reward effects in RL for reasoning.

Conclusion

RS-HyRe-R1 sets a new standard for reward design in RL post-training for RS-VLMs by directly addressing perceptual inertia. By enforcing explicit structured reasoning, robust outcome correctness, and maximal path diversity, it enables compact models to outperform larger, less carefully optimized counterparts. The hybrid reward mechanism not only boosts in-domain accuracy but also confers pronounced cross-domain generalization, underscoring the essential role of tightly coupled process-outcome feedback in deep multimodal models.

(2604.17504)