---
title: 'SenseNova-MARS: Agentic Vision-Language Framework'
url: https://www.emergentmind.com/topics/sensenova-mars
type: topic
---

# SenseNova-MARS: Agentic Vision-Language Framework

SenseNova-MARS is a multimodal agentic vision-language framework designed to enable seamless interleaved high-level reasoning and dynamic tool use for vision-language models (VLMs). The system empowers a pretrained VLM to strategically coordinate image search, text search, and image cropping tools via reinforcement learning, focusing on complex, knowledge-intensive, and visually demanding tasks that require coordinated tool use beyond text-oriented reasoning or isolated tool invocation [2512.24330].

## 1. System Architecture

SenseNova-MARS is constructed on top of a frozen vision encoder ("vision backbone") and a multimodal projector, which together produce joint image–text embeddings. This foundation supports a two-tiered system:

- **Reasoning Core:** A large language model (exemplar: Qwen3-VL-8B) autoregressively processes the cumulative interaction state—including prompts, model-generated <think> steps, and tool outputs—to sequentially generate both natural-language reasoning and structured tool calls.
- **Tool-Invocation Module:** Provides a lightweight interface for invoking precisely one tool per reasoning turn, using a JSON-parameter schema for reproducibility and structured action tracing. The available toolset comprises:
  - ```text_search(query: string) → top-5 summarized web snippets```
  - ```image_search(image_index: int) → set of related image thumbnails and captions```
  - ```image_crop(bbox: [x1, y1, x2, y2], image_index: int) → cropped image region```

After every tool action, the resulting observation—textual or visual—is appended to the interaction history, incrementally informing subsequent reasoning and tool selection. The interaction protocol enforces a cycle: exactly one <think>→<tool_call> per step (non-final) and a single reasoning step plus answer in the terminal step.

## 2. Batch-Normalized Group Sequence Policy Optimization (BN-GSPO)

The RL optimization component of SenseNova-MARS is based on BN-GSPO, which extends Group Sequence Policy Optimization (GSPO) for improved stability and effectiveness on heterogeneous, multi-tool trajectories.

For each trajectory $\tau$ in a batch, BN-GSPO introduces two levels of advantage normalization:

1. **Group (Prompt-Level) Normalization:** For each prompt $x_b$ with sampled trajectories $\{y_{b,g}\}$, normalize the reward $r_{b,g} = R(\tau_{b,g})$ within the group:
   $$
   \bar{A}_{b,g} = \frac{r_{b,g} - \mathrm{mean}_{g'} r_{b,g'}}{\mathrm{std}_{g'} r_{b,g'}}
   $$
2. **Batch-Wide Normalization:** Normalize across all batch elements:
   $$
   \tilde{A}_{b,g} = \frac{\bar{A}_{b,g} - \mathrm{mean}_{b',g'} \bar{A}_{b',g'}}{\mathrm{std}_{b',g'} \bar{A}_{b',g'}}
   $$

The length-normalized importance ratio $s_{b,g}(\theta)$ adjusts for variable-length sequences:
$$
s_{b,g}(\theta) = \left[ \frac{\pi_\theta(y_{b,g}|x_b)}{\pi_{\theta_\text{old}}(y_{b,g}|x_b)} \right]^{1/|y_{b,g}|}
$$

The clipped training objective, penalized by $\beta$ times the KL divergence from a reference policy, is:
$$
J(\theta) = \mathbb{E}_{b,g} \left[ \min(s_{b,g}(\theta) \cdot \tilde{A}_{b,g}, \mathrm{clip}(s_{b,g}(\theta), 1-\epsilon_\text{low}, 1+\epsilon_\text{high}) \cdot \tilde{A}_{b,g}) \right] - \beta \cdot D_{\text{KL}}(\pi_\theta \| \pi_\text{ref})
$$

The composite sequence-level reward $R(\tau)$ combines:
- $R_{\text{acc}} = 1$ if the answer is correct, per GPT-4o LLM-judge; 0 otherwise.
- $R_{\text{format}} = 0.5$ if the format protocol (<think>→<tool_call>, JSON-conforming) is strictly obeyed; 0 otherwise.

## 3. Dynamic Tool Integration and Agentic Reasoning

At each reasoning step $t$, the agent processes the complete multimodal history $\mathcal{O}_t$ and chooses an action $a_t$ from the discrete action space:
- $A = \{$text_search$(q)$, image_search$(i)$, image_crop$(\text{bbox}, i)$, produce final answer$\}$

The enforced protocol requires each non-terminal step to output a reasoning thought and exactly one tool action, while the final step must generate a reasoning thought and produce the answer. Correct adherence is rewarded via $R_{\text{format}}$. Reinforcement learning with BN-GSPO allows the agent to learn optimal policies for tool selection and invocation order, such as resolving pixel-level queries with image_crop or retrieving external information via search tools. This interleaved tool-reasoning cycle is necessary to address knowledge-intensive and visually complex questions insufficiently solvable via isolated tool or static reasoning approaches.

## 4. HR-MMSearch Benchmark

HR-MMSearch is a high-resolution, knowledge-intensive benchmark created to evaluate search-driven multimodal reasoning agents. Core characteristics include:
- 305 images at native 4K resolution, covering 8 thematic domains (Sports, Culture, Science & Technology, Business & Finance, Games, Academic Research, Geography & Travel, Others).
- Each image paired with a single manually crafted question that is search-driven, knowledge-intensive, and targets small or inconspicuous content (<5% of image area) unanswerable from pixels alone.
- At least one tool invocation is requisite for each query, with quality control and difficulty assessment performed by master’s-level professionals.
- Difficulty split by agent pass rates: "Hard" set (0/8 success for Qwen2.5-VL-7B) and "Easy" set (≥1/8 success).
- Evaluation via Agentic Pass@1 (zero-temperature decoding), judgements from GPT-4o.

## 5. Experimental Results and Comparative Performance

SenseNova-MARS-8B demonstrates state-of-the-art results across major multimodal and search-oriented benchmarks. The table below summarizes the Pass@1% on MMSearch and HR-MMSearch, comparing SenseNova-MARS-8B to relevant baseline and proprietary models:

| Model                      | MMSearch | HR-MMSearch |
|----------------------------|----------|-------------|
| Qwen3-VL-8B (direct)       |  11.7    |   12.1      |
| Qwen3-VL-8B (zero-shot agent)| 47.4   |   27.9      |
| MMSearch-R1 (open-source)  |  53.8    |   20.3      |
| GPT-5 (proprietary agent)  |  52.6    |   38.4      |
| Gemini-3-Flash (agent)     |  62.6    |   41.6      |
| SenseNova-MARS-8B (agent)  |  67.8    |   41.6      |

Across high-resolution perception tasks (Exact-Match%):

| Model            | V* Bench | HR-4K | HR-8K | MME-RealWorld | Avg.  |
|------------------|----------|-------|-------|---------------|-------|
| Mini o3          |  88.2    | 77.5  | 73.3  | 65.5          | 76.1  |
| DeepEyesV2       |  81.8    | 77.9  | 73.8  | 64.9          | 74.6  |
| SenseNova-MARS-8B|  92.2    | 83.1  | 78.4  | 67.9          | 80.4  |

These results reflect substantial gains over direct and zero-shot agentic VLM baselines, achieving +14.0 points on MMSearch and +3.0 points on HR-MMSearch relative to GPT-5.

## 6. Ablation Studies, Limitations, and Prospective Directions

- **BN-GSPO comparison:** On the 7B model variant, BN-GSPO yields higher Pass@1 on MMSearch (56.7%) compared to GRPO (50.9%) and GSPO (53.8%), and increases V* Bench scores by 12–25 points.
- **RL Data Mixing:** A hybrid RL dataset (search-oriented + perception data) surpasses specialized approaches; for example, exclusive perception training is detrimental to MMSearch, whereas hybridization provides +5.2 points.
- **Tool-Use Patterns:** The framework adapts tool selection: cropping is almost exclusive on V* Bench, search is dominant on MMSearch, and a mixed approach is observed on HR-MMSearch. Reinforcement learning further reduces average tool invocations per query from ~4 to ~2.
- **Identified Limitations:** Failure analysis identifies two leading sources of error:
  - Retrieval noise (misinterpretation of similar data, e.g., "born in" vs. "based in") causes hallucinations.
  - Tool parameter misconfigurations (imprecise cropping, overly generic queries) hinder fine-grained retrieval.
- **Future Work:** Directions include expanding robust alignment strategies to counter distractor information, enriching the suite of available tools (e.g., OCR, knowledge-base access), and adopting self-supervised strategies to reduce the dependence on costly human-confirmed data.

SenseNova-MARS substantiates the potential for end-to-end RL with BN-GSPO to produce multimodal agentic systems that emulate human-like proficiency in dynamic tool-use and multimodal reasoning, establishing new standards on complex, high-resolution, search-driven visual tasks [2512.24330].

Source: https://www.emergentmind.com/topics/sensenova-mars