S-Agent-8B: Distilled Spatial Reasoning Agent

• S-Agent-8B is an 8B-parameter spatial agent that redefines spatial reasoning as iterative, evidence-based accumulation over continuous multi-view inputs.
• It leverages a three-level tool stack and dual memory systems to plan, invoke specialized spatial experts, and consolidate geometric and semantic evidence.
• Benchmark evaluations show the model significantly outperforms non-tool-augmented baselines, highlighting its practical gains in 3D scene understanding.

Searching arXiv for the specified paper and closely related context. /arxiv_search (Dai et al., 18 Jun 2026) S-Agent-8B is an 8 B-parameter spatial agent introduced within the S-Agent framework, a spatial tool-use paradigm for understanding and reasoning over continuous multi-view images and videos. Rather than treating spatial reasoning as isolated frame-level prediction, the system formulates it as spatio-temporal evidence accumulation over a continuous and evolving 3D world. In this formulation, a base vision-LLM acts as a semantic planner that iteratively requests evidence, invokes specialized spatial tools and experts, maintains temporal memory, and produces an answer only after sufficient structured evidence has been collected. The model is presented as a compact agent distilled from S-Agent-generated trajectories, and is reported to substantially outperform similar-scale baselines while approaching advanced closed-source multimodal systems on several spatial-reasoning benchmarks (Dai et al., 18 Jun 2026).

1. Conceptual framing

S-Agent-8B emerges from an “agentic” paradigm in which a base vision-LLM is not asked to answer spatial questions in one shot. Instead, it is cast as a lightweight semantic planner that iteratively calls specialized spatial tools, accumulates their outputs in memory, and answers only when adequate evidence has been gathered. This redefines spatial reasoning from frame-centric recognition to scene-centric understanding grounded in evidence aggregation across views, frames, and reasoning steps (Dai et al., 18 Jun 2026).

The central distinction is between direct answer generation and explicit evidence acquisition. Existing VLMs and tool-augmented agents are characterized as largely tied to static, stateless inference from isolated visual observations. S-Agent instead treats the problem as one of maintaining an evolving scene representation and procedural reasoning trace. This suggests that the model’s competence is not defined solely by the parametric content of the backbone VLM, but by the interaction between planning, tool invocation, memory, and structured aggregation.

A common misconception addressed by the reported results is that stronger spatial performance necessarily requires larger closed-source models. The S-Agent paper argues that a compact 8 B model can become substantially more capable when trained on tool-use trajectories that encode evidence-integration policies, rather than relying only on direct end-to-end answering (Dai et al., 18 Jun 2026).

2. Architecture and operational loop

At its core, S-Agent-8B uses Qwen3-VL-8B-Instruct as a planner $\pi_\theta$. At each reasoning step $t$, the planner issues an evidence request based on the question $q$, the visual inputs $\mathcal{F}$, and two memory states, Scene Memory $S_t$ and Agent Memory $H_t$:

$r_t = \pi_\theta(q, \mathcal{F}, S_t, H_t).$

A selected spatial tool $T(r_t)$ executes the request and returns an observation $o_t$. The two memories are then updated as

$(S_{t+1}, H_{t+1}) = Update(S_t, H_t; r_t, o_t).$

Once the planner judges that it has enough structured evidence stored in memory, it emits the final answer (Dai et al., 18 Jun 2026).

The tool stack is organized into three levels. Level 1 performs 2D grounding and includes open-vocabulary detectors, VLM-grounding votes, and keyframe selectors; GroundingDINO is explicitly named. Level 2 performs 3D lifting through a Depth-Anything-3-based module that recovers metric depth, 3D coordinates, and camera poses from boxed regions or points, producing per-region point clouds or bird’s-eye projections. Level 3 consists of five spatial experts: Counting Expert, Metric Measurement Expert, Relative Position Expert, Visual Orientation Expert, and Object-Centric View Expert. These experts deterministically convert lower-level 2D and 3D evidence into structured observations such as distances, counts, orientations, and relative spatial relations (Dai et al., 18 Jun 2026).

The architecture is complemented by two memory systems. Scene Memory is an entity-centric store containing grounded object names, visual boxes, accumulated geometric attributes, and high-level relations. New tool outputs are merged into the corresponding scene entries or used to create new ones. Agent Memory records the procedural trajectory, including planner thoughts, tool calls, returned observations, failures, and intermediate conclusions. In practical terms, Scene Memory supports reuse of scene facts, while Agent Memory suppresses redundant tool use and enables strategy refinement. The combination operationalizes evidence accumulation across both time and reasoning depth (Dai et al., 18 Jun 2026).

3. Spatial tools, experts, and reasoning behaviors

The reasoning behaviors attributed to S-Agent-8B are defined through the five expert modules and the interaction between the three tool levels. Counting is handled by the Counting Expert, which merges detections across frames, applies non-maximum suppression, and answers questions of the form “How many $t$0?” including those with attribute or relational filters such as “How many red blocks on the left shelf?” (Dai et al., 18 Jun 2026).

Measurement is performed by the Metric Measurement Expert. It lifts 2D boxes into 3D via the depth tool, samples representative points, and computes Euclidean distances, including center-to-center and object-edge-to-edge distances, as well as object dimensions. Orientation is handled by the Visual Orientation Expert, which examines object-centric cues such as front and back surfaces, handles, and symmetry within a grounded crop to answer directional questions such as “Which way is the handle facing?” Relative Position is handled by the Relative Position Expert, which transforms relevant objects into a common 3D frame, potentially using egocentric or world axes, and deterministically evaluates left, right, front, back, and cardinal relations (Dai et al., 18 Jun 2026).

These modules are not presented as generic chain-of-thought components; they are spatially specialized operators that convert raw detections and geometry into higher-level facts. The data further indicate that Level 2 raw geometry “adds little or distracts unless interpreted,” whereas Level 3 experts “unlock +7 pt over L1+L2.” A plausible implication is that the principal gain does not come from access to geometric quantities alone, but from deterministic spatial interpretations that map those quantities into task-relevant relational abstractions.

Evidence accumulation is a further defining behavior. Across multi-view or multi-frame inputs, Scene Memory suppresses duplicates and consolidates new detections, while Agent Memory prevents repeated requests to the same tool. The planner therefore reasons over a compact set of structured facts rather than directly over raw pixels. This suggests a shift from perceptual redundancy toward symbolic- or record-like intermediate state, though the paper frames the representation specifically in terms of structured observations and scene facts rather than formal symbolic reasoning (Dai et al., 18 Jun 2026).

4. Supervised fine-tuning and the S-300K trajectory corpus

S-Agent-8B is trained through supervised fine-tuning on S-Agent-generated trajectories collected in the S-300K dataset. The starting point is 800 K spatial questions in SenseNova-SI, from which 100 K examples were sampled that Qwen3-VL-8B found uncertain. A frozen S-Agent with planner set to GPT-5.4 generated full tool-use trajectories for these 100 K questions, logging every $t$1, $t$2, $t$3, $t$4, and final answer. Only trajectories with valid executions and correct final answers were retained, using three criteria: MCQ exact match, numeric MRA $t$5, and text match. This filtering yielded approximately 51.6 K traces (Dai et al., 18 Jun 2026).

Each retained trajectory was decomposed into three supervised forms: 51.6 K full “final-answer” sequences, 154.6 K intermediate “turn-level” planner calls, and 86.2 K individual tool or expert invocations. The total number of supervised fine-tuning samples was 292,391, which motivates the designation “S-300K” (Dai et al., 18 Jun 2026).

The fine-tuning objective is standard next-token cross-entropy over serialized assistant responses, including planner thoughts, tool calls, observations, and the final answer:

$t$6

The reported training setup uses 8× NVIDIA B200 GPUs, a maximum sequence length of 8,192, a learning rate of $t$7, cosine learning-rate decay with 3% warmup, one epoch, and no extra regularization beyond the default weight decay. The resulting model is S-Agent-8B, described as an 8 B-parameter model with built-in spatial tool-use policy (Dai et al., 18 Jun 2026).

This training procedure is significant because the supervision target is not only the final answer. It also includes the intermediate planner-tool interaction sequence. A plausible implication is that the model is being taught both what answer to produce and how to orchestrate evidence collection under uncertainty, thereby distilling an agentic policy rather than merely improving answer priors.

5. Benchmark performance and ablation findings

The reported evaluation separates training-free inference-time augmentation from the behavior of the compact agent after supervised fine-tuning. In the zero-shot setting, S-Agent is used as a wrapper around an existing VLM. On MMSI-Bench, a multi-image benchmark with 13 K questions, GPT-5.4 baseline achieves 41.9% average, Gemini 3 Pro 45.2%, and S-Agent with planner = GPT-5.4 reaches 46.4%, corresponding to +4.5 points versus GPT-5.4 and +1.2 points versus Gemini 3 Pro. On ViewSpatial-Bench, GPT-5.4 scores 45.6% while S-Agent reaches 60.0%, a gain of +14.4 points. On ReVSI, a video 3D reasoning benchmark, the best open-source general model is reported as 54.1% for InternVL 38 B, whereas S-Agent reaches 58.8%, ranking second overall and as the best open-source agent on multiple choice; across subtasks including relative direction, route planning, and camera or object motion, zero-shot S-Agent is consistently top-3 (Dai et al., 18 Jun 2026).

For the distilled compact agent, the paper reports the following trajectory-distillation results.

Model	MMSI	ViewSpatial	ReVSI
Qwen3-VL-8B (no tools)	31.1	42.2	49.1
S-Agent (Qwen3-VL-8B)	30.7	44.1	49.5
S-Agent-8B (after S-300K)	41.6	46.8	52.8
Gemini 3 Pro	45.2	50.4	60.9
GPT-5.4	41.9	45.6	—

These results show that S-Agent-8B improves by +10.5 points on MMSI relative to its Qwen3-VL-8B backbone and improves by 4–7 points on ViewSpatial and ReVSI. The paper characterizes this as closing in on Gemini 3 Pro and GPT-5.4 on MMSI, while outperforming the original backbone on the other evaluated tasks (Dai et al., 18 Jun 2026).

The ablation study clarifies which architectural components matter. Using only Level 1 2D grounding yields +3.4 points versus VLM-only on ViewSpatial. Level 2 raw geometry adds little or can distract unless interpreted. Level 3 experts provide an additional +7 points over L1+L2. Scene Memory and Agent Memory each add 1–2 points, and the full stack yields 60.0%. These findings constrain interpretation: the improvement is not attributable to any single ingredient in isolation, and geometry appears most effective when coupled to deterministic expert interpretation and stateful memory (Dai et al., 18 Jun 2026).

6. Significance, applications, and interpretive boundaries

The paper identifies three practical implications. First, S-Agent can augment any off-the-shelf VLM, whether open- or closed-source, without retraining; this yields training-free gains of 4–15 points on the reported benchmarks. Second, explicit evidence acquisition and planner orchestration are presented as mechanisms for filling the semantic-to-geometric gap in VLMs. Third, trajectory distillation on S-300K shows that even an 8 B-parameter open-weight model can learn tool-use policies and evidence-integration patterns that approach or rival closed-source systems on some tasks (Dai et al., 18 Jun 2026).

The proposed application domains are those requiring accurate 3D awareness, including robotic manipulation and navigation, AR/VR object placement, and autonomous driving scene understanding. The reported advantage is not restricted to final answer accuracy: the system also produces explicit structured evidence such as measurements and bounding boxes, which can be audited or passed to downstream control systems. This suggests relevance for settings in which interpretability of intermediate spatial state is operationally useful (Dai et al., 18 Jun 2026).

Several interpretive boundaries are also clear from the reported evidence. The benchmark claims concern multi-view and video spatial reasoning rather than general multimodal performance. The strongest zero-shot results are often obtained by using S-Agent as inference-time augmentation around a stronger planner, such as GPT-5.4, whereas S-Agent-8B is the distilled compact version. It would therefore be misleading to conflate the performance of the wrapper configuration with that of the standalone 8 B model. Likewise, the ablations indicate that access to depth and 3D cues alone is insufficient; the benefits depend on the full evidence pipeline, especially expert interpretation and memory. These distinctions matter when comparing S-Agent-8B either to one-shot VLMs or to closed-source multimodal agents (Dai et al., 18 Jun 2026).

In aggregate, S-Agent-8B is best understood as a distilled spatial agent whose defining property is not merely its backbone architecture, but a learned policy for sequencing tool calls, accumulating spatio-temporal evidence, and reasoning over structured scene state. Within the evidence reported in the original study, it represents a compact instantiation of the broader claim that spatial intelligence can be materially improved by recasting multimodal inference as stateful, tool-grounded evidence accumulation rather than direct frame-level prediction (Dai et al., 18 Jun 2026).