Beyond Next-Token Prediction: An RLVR Proof of Concept for Tool-Use Agents on Atlassian Workflows

Abstract: LLMs are trained to predict the next token, not to act inside a specific API. In niche enterprise SaaS workflows -- where success means hitting the right endpoint with the right nested arguments in the right order -- this objective mismatch shows up as silent failures: dropped required fields, hallucinated tools, or early stops after a single read. We ask whether Reinforcement Learning with Verifiable Rewards (RLVR), applied directly in the target environment, closes the gap. As a proof of concept we build a suite of five synthetic environments emulating the Jira REST v3 and Confluence v2 APIs at schema fidelity; rewards are computed entirely from the tool-call trace, with no live API, no learned judge, and no human label in the loop. Scoring prompted Qwen3-1.7B and Qwen3.5-4B on the same checkers that drive GRPO training, we find that on the four scenarios whose rewards are non-degenerate the RL-trained policy lifts average reward from a 4B-baseline range of 0.35--0.92 to 0.95--1.00, with the largest single gain on Confluence page creation ($0.35 \rightarrow 1.00$). We position this as a preliminary step toward outcome-optimised small models for niche enterprise APIs, and foreground two limitations a workshop reader should weigh: hand-crafting verifiable rewards does not scale beyond the handful of endpoints reported here, and one of our five scenarios (ticket-transition) has a saturating reward shape that the prompted 4B already maxes out.

• The paper demonstrates that RLVR-trained LLMs can significantly improve the correctness of API call structures over traditional next-token predictions.
• It employs fully synthetic, deterministic environments that mimic Atlassian APIs and uses programmatic rewards for argument accuracy, proper sequencing, and error penalization.
• The approach paves the way for automating reward synthesis and scaling RL-driven agent development for robust enterprise workflow automation.

RLVR for LLM Tool-Use Agents on Enterprise Atlassian Workflows

Motivation and Problem Statement

The work addresses a fundamental misalignment in most LLM-driven tool-use agents: LLMs are optimized for next-token prediction, not for generating structured API calls following the rigorous schemas prevalent in enterprise SaaS systems. In operational contexts such as Jira and Confluence automation, failure manifests as plausibly-formed but incorrectly-structured API calls, omission of required arguments, hallucination of tools, and premature stoppage after a single read. The paper investigates whether Reinforcement Learning with Verifiable Rewards (RLVR)—implemented in fully synthetic, schema-faithful environments—can bridge this gap by directly optimizing for successful outcome-based API interaction instead of mere text prediction (2607.01465).

Synthetic Environments and Reward Design

Five synthetic environments are created, emulating the Atlassian Jira REST v3 and Confluence v2 APIs at full schema fidelity. These environments are deterministic, stateful, and feature per-batch resets to support reproducible and stable reward computation. Each scenario presents realistic automation tasks involving ticket transitions, sub-task creation, page creation, page labeling, and an integrated Jira-Confluence workflow.

The reward functions are programmatic, dense, and decomposed over three axes:

• Per-argument correctness (R1): Rewards for precise matching of expected argument values.
• Structural bonuses (R2): Rewards for executing validate–mutate–verify interaction sequences, encouraging environment-grounded procedural competence.
• Penalties (R3): Deductions for invalid payloads, duplicate/hallucinated tool calls, missing required calls, and exceeding call budgets.

By constraining rewards to be deterministic functions of the tool-call trace, the approach circumvents the need for learned judges, human-in-the-loop feedback, or live access to production APIs during training.

RLVR Training Protocol and Implementation

Qwen3-1.7B and Qwen3.5-4B serve as policy backbones. Training employs Grouped Relative Policy Optimization (GRPO) in the TRL framework, with convergence-based early stopping. Hardware requirements are kept modest (single to dual GPU nodes). All evaluation—both baseline and post-RL—is performed in the same synthetic environments for comparability.

Experimental Results

Prompted baseline performance (static Qwen3-1.7B and Qwen3.5-4B) is measured against the same reward functions used for RL. On all non-trivial scenarios, RLVR yields substantial reward gains:

• Confluence page creation: Reward increases from 0.35 (prompted baseline) to 1.00 (RLVR-trained), an absolute lift of +0.65.
• Confluence labeling: 0.52 to 0.95 (+0.43).
• Jira sub-task creation: 0.68 to 1.00 (+0.32).
• Cross-product (integrated Jira+Confluence): 0.92 to 1.00.

In all cases, the RL-trained policy produces strictly correct, schema-compliant tool-call sequences, as opposed to the prompted models' frequent argument shape errors, dropped fields, or early termination after a read. The one degenerate scenario (ticket transition) is saturated by baseline performance due to a trivial reward shape.

Figure 2: RLVR-trained agents outperform prompted Qwen3-1.7B and Qwen3.5-4B baselines on all non-degenerate scenarios; baselines suffer from slot-filling and schema errors.

Qualitative Behavioral Analysis

Pre-training trajectories reveal two main failure modes for prompted models: (1) interruption after only performing read actions, and (2) argument mis-shaping leading to repeated, penalized retries. Post-training rollouts consolidate behavior into the expected read-before-write sequence, with all nested arguments and call orderings correct, no hallucinations or extraneous calls, and no reward hacking observed in the trace-level output.

Limitations and Scalability

The approach's primary limitation is engineering overhead: reward functions and scenario prompts are hand-crafted, which does not scale to hundreds of endpoints or full production schemas. The authors note that much of the reward logic (R1, R3) can be mechanically derived from OpenAPI specifications, leaving only structural rewards (R2) to be scenario-tuned, suggesting automatic reward synthesis as a clear direction for future work.

The environments do not simulate real-world API failure modes (e.g., rate limits or permission errors). Thus, zero-shot transfer to live APIs remains unaddressed and will require additional validation regimes such as replay of captured responses, fault injection, or limited live-tenant canaries.

The baseline comparison isolates the effect of RL, but does not preclude the efficacy of SFT on successful traces, plan–execute scaffolding, or larger models in similar workflows. Multi-turn user simulators and expanded prompt diversity are necessary for robust generalization assessment.

Implications and Future Directions

This work demonstrates that RLVR can close the observed gap between general-purpose LLMs and the requirements of outcome-optimized, schema-constrained agents for enterprise workflows. The RLVR-plus-synthetic-environment paradigm enables small-footprint, reproducible policy improvement in narrow application domains, but its utility as a generalized approach depends on automating reward derivation and validating real-world transferability.

Future developments will likely focus on end-to-end pipelines where programmatic reward construction and training environment synthesis are fully automated from API specifications, coupled with curriculum learning over scenario complexity and agentic abstraction layers that span multiple enterprise platforms. Integration of failure-mode simulators and controlled online RL in canary environments are pressing requirements for production-grade deployment.

Conclusion

The paper provides an authoritative proof-of-concept that RLVR, when applied within schema-faithful synthetic environments and equipped with deterministic, programmatic rewards, substantially improves the reliability of LLM-based tool-use agents over prompted and instruction-tuned models, specifically in workflow automation for enterprise APIs. While not without limitations in scalability and generalization, this work lays critical groundwork for future modular, outcome-optimized agent architectures targeting complex schema-driven environments (2607.01465).