Customer-Agent: Overcoming Context Limitations in Ultra-Long Shopping Trajectories via Tool-Augmented Agents and RLVR

Abstract: Understanding customer shopping trajectories is essential for enabling personalized shopping experiences. However, shopping records (i.e., customer's search, clicks, purchases, etc.) often span long time horizons over multiple years, resulting in extremely long trajectories that pose significant challenges for existing LLMs. Despite the importance of this problem, existing benchmarks are limited to short customer trajectories, while real-world trajectories from large e-commerce platforms are rarely accessible due to data privacy constraints. To address this gap, we introduce ShopTrajQA, a long-context evaluation benchmark constructed from real-world product information and simulated shopping trajectories. The dataset includes variants of up to 32k and 64k tokens, enabling systematic evaluation of model robustness under varying context lengths. Through comprehensive benchmarking of frontier LLMs, we identify critical performance gaps in reasoning over long shopping trajectory data. To address these challenges, we propose a Customer Agent Framework for ultra-long context management. Leveraging a Reinforcement Learning with Verifiable Rewards (RLVR) agentic training paradigm, our approach stores trajectories as external local files and trains the agent to autonomously retrieve and parse them through code-interpreter interactions (e.g., SQL queries), effectively bypassing the fixed in-context window constraints of LLMs. Experimental results demonstrate that our framework achieves strong performance for ShopTrajQA and shows generalization to other complex reasoning tasks.

• The paper introduces a long-context benchmark (ShopTrajQA) and a tool-augmented Customer-Agent framework that overcomes LLM context bottlenecks.
• The methodology integrates supervised fine-tuning with RLVR to improve tool invocation, achieving accuracy gains of over 49 points on extended trajectories.
• The approach decouples full trajectory context from direct LLM input, enabling scalable reasoning over complex, multi-year shopping data.

Overcoming LLM Context Bottlenecks in Shopping Trajectory Reasoning via Tool-Augmented Customer Agents and RLVR

Introduction

Ultra-long customer interaction histories in e-commerce platforms, manifesting as multi-year shopping trajectories, encode fine-grained behavioral intent but substantially exceed the context limits of current LLMs. The paper "Customer-Agent: Overcoming Context Limitations in Ultra-Long Shopping Trajectories via Tool-Augmented Agents and RLVR" (2606.07995) addresses the core challenge of robust long-context modeling and reasoning over realistic shopping data by proposing both a new long-context benchmark (ShopTrajQA) and a tool-augmented agentic framework trained with RLVR. The work delineates context window limitations of state-of-the-art LLMs and provides a modular solution for scalable reasoning that overcomes these constraints.

ShopTrajQA: A Long-Context Evaluation Benchmark

To facilitate rigorous study of long-context reasoning in the shopping domain, the authors construct ShopTrajQA, a QA-centric benchmark based on real-world product catalogs and LLM-driven trajectory simulation. This resource notably exceeds previous datasets (e.g., OPeRA) in both diversity and sequence length, with carefully constructed variants at 32k and 64k tokens that probe the limits of current architectures.

Figure 1: ShopTrajQA construction pipeline compared to OPeRA and analysis of user/action/product distribution, highlighting increased trajectory length and heterogeneity.

Benchmark trajectories are synthesized by prompting LLMs with precise rules to generate behaviorally consistent action sequences (search, click, cart, purchase), with product diversity enhanced by public datasets such as ESCI. Executable, code-verifiable QA pairs are constructed in tandem, enabling precise, automatic evaluation (Figure 2).

Figure 2: Retrieval QA construction workflow integrating programmatic answer verification.

Token length and action distributions are presented for quantitative scrutiny. The token histogram and corresponding action count clearly indicate ShopTrajQA's effectiveness for stress-testing LLM context handling:

Figure 3: Distribution of action counts across dataset variants; 64k trajectories systematically support more complex behavior sequences.

Figure 4: Token length distribution of ShopTrajQA, facilitating controlled, systematic context-expansion experiments.

Tool-Augmented Customer Agent Framework

Directly inputting complete, multi-year trajectories into LLMs leads to computational intractability (due to quadratic scaling of attention/memory) and degraded retrieval and reasoning, especially as window length approaches or exceeds 32k/64k. To address this, the Customer Agent Framework externalizes the trajectory context—storing interactions as local files or databases—and employs an agentic model that learns to invoke tools (e.g., code interpreters running SQL queries) for targeted retrieval and parsing.

The architecture supports:

• Context Offloading: The trajectory is never loaded in full into the prompt. Instead, only dynamically retrieved fragments are accessed via tool calls, bypassing nominal LLM context barriers.
• Interpreter Feedback Masking and KV-Cache Reuse: System feedback tokens are masked from gradient computation for purity of agent reasoning, and execution state reuse limits memory overhead.
• Sandboxed, Asynchronous Execution: Secure code invocation is decoupled from text generation for multi-turn tool-based learning.

  Figure 5: Customer-Agent architecture: model interacts in multiple turns with external shopping databases, orchestrating tool calls for precise context extraction and stepwise reasoning.

RLVR Training Paradigm

Although tool use provides a viable pathway for context scale-up, generic LLMs struggle with tool syntax and error recovery in the absence of explicit supervision. The framework therefore follows a two-phase optimization protocol:

1. Supervised Fine-Tuning (SFT) on Tool Use Traces: Cold-start data is synthesized by generating multi-turn, tool-integrated rollouts, filtering only those which execute successfully and yield correct answers. This bootstrap phase enables basic tool invocation ability.
2. Reinforcement Learning with Verifiable Rewards (RLVR): The core agent is then refined with task-grounded rewards, integrating both intermediate code executability and final answer correctness (with explicit reward shaping, strict exact match evaluation, and regulated by a clipped reward function to suppress tool-use overfitting).

Experimental Results

ShopTrajQA Evaluation

Baseline LLMs (gpt-oss 20B/30B/120B, Qwen3 variants) display marked performance collapse as context exceeds their fixed window, with accuracy degradation particularly severe at 64k tokens. In contrast, the tool-augmented agent framework, following SFT+RLVR, yields substantial improvements: Qwen3-4B advances from 13.8% to 62.5% accuracy on 32k and from 9.3% to 60.1% on 64k after full RLVR (absolute improvements exceeding 49 and 51 points, respectively).

Key empirical findings:

• Pure text ingestion becomes infeasible above ~64k context, resulting in drastic accuracy loss and frequent parsing/tool-use errors.
• Tool-based retrieval after agentic RLVR training enables nearly order-of-magnitude accuracy gains, simultaneously scaling context capacity and increasing retrieval precision.

Generalization

General reasoning ability is preserved; RLVR-trained agents maintain or slightly exceed baseline models when transferred to mathematical (AIME) and generic multi-hop QA datasets, thus no significant domain overfitting is observed.

Tool Call and Execution Metrics

Tool call rate (TCR) and successful tool call rate (SuccTCR) increase post-RLVR, with TCR approaching 0.95 for best models. Challenges remain in further elevating SuccTCR due to the inherent complexity of multi-turn, programmatic tool orchestration.

Case Study Diagnostics

Detailed case analyses show that naively embedding trajectory context within generated code produces systematic failure, including context omission, NameError due to execution state loss, and text format misparsing. The use of explicit local storage and tool-use templates is shown to be critical for system stability and scaling.

Implications and Future Directions

The separation of trajectory context from LLM input fundamentally alters the boundaries of tractable agentic reasoning: the primary constraint becomes one of storage and retrieval design, not transformer context length. This architectural decoupling opens the door to handling trajectories of millions of actions or tokens, provided that (i) tool-use grammars remain robust and (ii) reward specification remains properly aligned.

From a practical perspective, this approach enables deployment of LLM-powered agents for long-horizon personalization, recommendation, and behavior analysis in e-commerce under real-world data regimes where customer history exceeds ephemeral context. Given the modular tool integration, similar methodologies can be ported to CRM, medical records, and any discipline where historical trace mining is central.

Theoretical implications include reinforcement of the view that agent-environment interaction, facilitated via code and structured queries, provides a critical pathway for scalable reasoning beyond monolithic context ingestion. Future work should consider (1) extension to unstructured context, (2) more advanced memory management (e.g., hierarchical retrieval, cache optimization), and (3) meta-reasoning over complex tool usage (discovery, selection, composition).

Conclusion

This work provides a rigorous route for overcoming fixed-context limitations in LLMs via tool-augmented reasoning and RLVR, evidenced by both the construction of a realistic long-context behavioral benchmark and the empirical demonstration of dramatic accuracy improvements. The Customer Agent Framework offers a practical and extensible architecture for robust question answering and reasoning over ultra-long e-commerce trajectories, laying the groundwork for further advances in agentic LLMs with open-ended context and tool-integration capabilities.