STAgent: Autonomous Exploration & Tool Use

• STAgent is a class of autonomous agentic systems that combines exploratory stateful software testing with LLM-driven spatio-temporal reasoning.
• It employs decentralized decision-making with explicit state representation, heuristic planning, and robust fault detection to navigate complex environments.
• Empirical results for the LLM variant highlight improved planning fidelity, multi-tool orchestration, and scalable performance across benchmarking tasks.

STAgent refers to a class of agentic systems spanning distinct research lines in automated reasoning, software testing, spatio-temporal tool use, and agent-based LLM integration. Notably, the term "STAgent" labels both a conceptual exploratory test agent for stateful software systems (Karlsson, 2020) and a specialized agentic LLM for spatio-temporal reasoning and tool use (Hu et al., 31 Dec 2025). These threads are unified by a focus on autonomy, interactive exploration, and dynamic adaptation in complex environments.

1. Conceptual Foundations and Scope

STAgent, in its foundational articulation, designates an exploratory approach for automating the testing of stateful software systems. The architecture leverages agent-based abstraction to traverse, interact with, and probe the state space of a software system under test (SUT), aiming to expose faults and gain a systematic understanding of stateful behaviors (Karlsson, 2020). Independently, the STAgent model for spatio-temporal reasoning is positioned as a specialized LLM, augmented for planning, multi-tool interaction, and itinerary generation in domains requiring explicit temporal and spatial reasoning (Hu et al., 31 Dec 2025).

These divergent uses highlight a common set of principles: decentralized decision-making, dynamic state tracking, integration with external tools or interfaces, and explicit learning-driven exploration.

2. Formal Environment and Interaction Models

The original exploratory STAgent formalization postulates the SUT as a transition system, typically not fully specified but characterized in terms of:

• States $S$: Representing abstract screens, API endpoints, or process-states.
• Actions $A$: Interactions such as API calls or GUI events that trigger transitions.
• Transition function $\delta: S\times A \to S$: Mapping state-action pairs to successor states.

Future extensions envision formal Markov Decision Process (MDP) models, e.g., $(S, A, \delta, R)$ with a reward function $R: S \times A \times S \to \mathbb{R}$ capturing exploration utility or fault discovery (Karlsson, 2020). However, a fully worked-out MDP is not present in the original proposal, but remains a core part of the research agenda.

In the spatio-temporal LLM-based STAgent, the environment is constituted by a suite of ten domain-specific tools (for mapping, navigation, travel, weather, and retrieval), accessed via a unified JSON-RPC protocol within a FastMCP sandbox and ROLL asynchronous rollout system (Hu et al., 31 Dec 2025). This environment is designed for high-fidelity, concurrent tool calls, supporting multi-tool workflows central to planning tasks.

3. Agent Architecture and Core Modules

The exploratory STAgent posits an internal structure based on functionally distinct subsystems:

• Knowledge Base/Memory: Recording prior states, action histories, and coverage metrics.
• State Representation: Either explicit via finite-state machine node identifiers or implicit through environment snapshots (e.g., GUI, API responses).
• Action Selection/Planning: Spanning random (fuzzer-style), heuristic, or learning-based schedules.
• Fault Detection/Asserting: Monitoring for crashes, invariants, or anomalies (Karlsson, 2020).

Subsystems explicitly highlighted are:

• Sensing: Extracting state and metric data from the SUT via hooks or logs.
• Deciding/Planning: Determining the next interaction.
• Asserting: Enforcing property-based or difference-based checks to discriminate faults.

For the spatio-temporal STAgent, architectural emphasis is on robust tool orchestration, high-throughput inference, and ability to chain tool calls for intermediate verification and reasoning. The backbone is a mixture-of-experts (MoE) LLM (Qwen3-30B-A3B), selected for specialization capacity without sacrificing general performance (Hu et al., 31 Dec 2025).

4. Data Curation, Training Paradigms, and Learning

A major focus for the LLM-based STAgent is the hierarchical data curation framework:

• Lexical Filtering: Removal of near-duplicates from logs.
• Semantic Filtering: Diversity maximization in embedding space using Faiss.
• Geometric Filtering: K-Center-Greedy selection for broad coverage.
• Difficulty & Diversity Stratification: Each sample receives a scalar score for cognitive load, depth, and constraint complexity.
• Learnability Potential: Defined by $$S_i = \hat\sigma_i^2 \times \max(\hat\mu_i,\,\epsilon)$$, combining uncertainty and empirical reward mean.

Training proceeds through a three-phase regime:

1. Seed SFT (Supervised Fine-Tuning) Guardian: Strong filtering, policy initialized on high-certainty expert-corroborated samples.
2. High-Certainty SFT: Additional fine-tuning on high-certainty, simple trajectories.
3. SFT-Guided RL: Asynchronous GRPO/GSPO reinforcement learning focused on uncertain, boundary cases, maintaining KL-divergence control.

By filtering down from 30M logs to 200k high-quality samples (1:10,000), data curation is extreme, with curriculum learning supported by difficulty balancing. This approach is tailored to maximize diversity and advance the agent's capabilities on the hardest, most instructive samples (Hu et al., 31 Dec 2025).

The classical exploratory STAgent posits but does not implement MDP-based learning, Q-learning, or coverage-guided exploration. Existing prototypes execute only continuous smoke-test loops over APIs, with advanced sequence generation and prioritization remaining for future work (Karlsson, 2020).

5. Empirical Results and Evaluation

For the spatio-temporal STAgent, extensive benchmarking is performed:

• TravelBench (in-domain): Achieves 66.6, 73.4, and 71.0 on Multi-turn, Single-turn, and Unsolvable, with an overall score of 70.3, outperforming baselines including Qwen3-30B and even some 235B models.
• Online Evaluation: Delivers +12.8% on reasoning/planning, +15.3% on fidelity, and +9.8% on presentation over Qwen3-30B-Thinking.
• General Benchmarks: Matches or exceeds backbone in function calling (BFCL v3: 76.8), mathematics, code synthesis, and knowledge (MMLU-Pro, C-Eval), confirming no degradation or loss of generality post-specialization (Hu et al., 31 Dec 2025).

For the exploratory STAgent, the only operational result reported is a smoke-test agent running in a staging environment. No quantitative fault rates, state coverage, or comparison to baselines are available. All full-scale empirical studies are identified as future work (Karlsson, 2020).

6. Lessons, Challenges, and Research Directions

Key challenges and open research questions are distilled in the exploratory STAgent vision (Karlsson, 2020):

1. Interaction Model Induction: Determining how best to represent the SUT—via GUI, explicit FSMs, or derived from API specifications.
2. Agent Goals and Reward Specification: Elevating agent objectives to user-meaningful coverage criteria or resilience metrics.
3. Assertions and Oracles: Developing robust difference- or property-based oracles for fault discrimination.

The STAgent vision also advocates for a multi-agent ecosystem where smoke testers, fuzzers, chaos agents, and domain specialists collaborate, leveraging shared goals and results. Intrinsic-motivation RL (e.g., curiosity-driven) is specifically highlighted as promising for overcoming manual reward engineering bottlenecks.

In the spatio-temporal LLM-based STAgent, open themes include scaling to new domains, extending toolboxes, and integrating more sophisticated reasoning models without incurring catastrophic forgetting of general abilities (Hu et al., 31 Dec 2025).

7. Synthesis and Contextual Significance

"STAgent" uniquely represents both a roadmap for autonomous software test agents and a realized agentic LLM architecture for spatio-temporal reasoning. Commonalities are evident in their commitment to agentic autonomy, systematic exploration, the necessity for diverse and high-utility data, and the imperative to balance specialization with generality. While the original exploratory test agent remains largely visionary—charting research questions and sketching an agenda—the LLM-based STAgent manifests these agentic principles in a concrete, empirically validated system. Both strands position STAgent at the intersection of AI planning, automated reasoning, interactive learning, and robust system understanding, with the latter demonstrating cross-domain transfer and empirical superiority in applied benchmarks.

For foundational exploration and future directions, see (Karlsson, 2020) for the conceptual model and (Hu et al., 31 Dec 2025) for current-generation LLM-based STAgent design and performance.