Agentic Aggregation for Parallel Scaling of Long-Horizon Agentic Tasks

Abstract: We study parallel test-time scaling for long-horizon agentic tasks such as agentic search and deep research, where multiple rollouts are generated in parallel and aggregated into a final response. While such scaling has proven effective for chain-of-thought reasoning, agentic tasks pose unique challenges: trajectories are long, multi-turn, and tool-augmented, and outputs are often open-ended. Aggregating only final answers discards rich information from trajectories, while concatenating all trajectories exceeds the model's context window. To address this, we propose AggAgent, an aggregation agent that treats parallel trajectories as an environment. We equip it with lightweight tools to inspect candidate solutions and search across trajectories, enabling it to navigate and synthesize information on demand. Across six benchmarks and three model families (GLM-4.7, Qwen3.5, MiniMax-M2.5), AggAgent outperforms all existing aggregation methods-by up to 5.3% absolute on average and 10.3% on two deep research tasks-while adding minimal overhead, as the aggregation cost remains bounded by a single agentic rollout. Our findings establish agentic aggregation as an effective and cost-efficient approach to parallel test-time scaling.

• The paper introduces AggAgent, a new agentic aggregation paradigm that consolidates parallel trajectories for long-horizon tasks to synthesize solutions leveraging full evidence.
• It employs in-memory tools for selective search and deep reads, yielding performance improvements of up to 10.3 points over heuristic baselines.
• AggAgent maintains bounded aggregation cost with minimal overhead while effectively scaling performance across complex, multi-turn tasks.

Agentic Aggregation for Parallel Scaling of Long-Horizon Agentic Tasks

Motivation and Problem Setting

Parallel test-time scaling is an established technique for boosting LLM performance, but prior efficacy has largely been confined to chain-of-thought (CoT) tasks. Long-horizon agentic tasks—such as web navigation, deep research, and software engineering—introduce unique aggregation challenges. Here, agent trajectories are multi-turn, tool-augmented, and often open-ended, rendering naive voting, selection, or summary-based aggregation suboptimal. Concatenating all trajectory content exceeds context window constraints; compressing trajectories via summarization is lossy and costly; aggregating only final answers ignores rich reasoning evidence.

The central task is to aggregate $K$ parallel agentic trajectories $T_1,\ldots,T_K$ into a final solution $\hat{y}$ that maximally utilizes distributed evidence, preserves full trajectory fidelity, and maintains computational efficiency.

AggAgent: Agentic Aggregation Paradigm

AggAgent reframes aggregation itself as an agentic task, treating the set of completed trajectories as an interactive environment. The agent is equipped with lightweight, in-memory tools—get_solution, search_trajectory, get_segment, and finish—for on-demand examination and synthesis. This enables cross-trajectory reasoning without bulk loading trajectories into LLM context, maintaining bounded aggregation cost and latency.

The aggregation workflow is coarse-to-fine:

• Survey metadata and candidate solutions;
• Selectively inspect trajectories (via keyword search or segment reads);
• Validate key reasoning steps against tool outputs;
• Synthesize a standalone solution (with justification).

AggAgent's unique strength lies in full-fidelity access to trajectory evidence while incurring minimal overhead, fundamentally addressing the lossy or context-limited nature of existing solution or summary aggregation.

Figure 1: High-level comparison: Solution Aggregation discards intermediate reasoning, Summary Aggregation uses lossy compression, AggAgent enables agentic tool-based cross-trajectory reasoning.

Empirical Evaluation

AggAgent was evaluated across six long-horizon benchmarks—BrowseComp, BrowseComp-Plus, HLE, DeepSearchQA, Healthbench-Hard, ResearchRubrics—using three LLM families (GLM-4.7-Flash, Qwen3.5-122B, MiniMax-M2.5). The same model served as both rollout agent and aggregator. Across all settings, AggAgent consistently outperformed heuristic and LLM-based baselines, improving average score by 2.4–5.3 points over the best baseline, and up to 10.3 points on deep research tasks.

Strong numerical results include:

• AggAgent achieves 13.3–17.9 points improvement over Pass@1.
• Aggregation cost overhead of only 5.7% (compared to 41% for Summary Aggregation at $K=8$), with bounded latency.
• AggAgent surpasses Pass@8 in several settings, demonstrating synthesis capabilities beyond any single trajectory.

  Figure 2: AggAgent outperforms existing aggregation methods on average across six benchmarks as K increases.

  

  Figure 3: Pareto-optimal performance-efficiency tradeoff: AggAgent achieves higher performance at comparable or lower cost and latency.

Architectural Variants and Ablations

Ablation studies confirm:

• Using a stronger model for aggregation (MiniMax-M2.5 over GLM-4.7-Flash) boosts performance, supporting asymmetric multi-agent system designs.
• Synthesis-based output yields substantial gains over best-trajectory selection, especially in open-ended research settings where quality is distributed across trajectories.

  Figure 4: Employing a stronger aggregator (MiniMax-M2.5) improves performance for all LLM-based aggregation methods, especially AggAgent.

  

  Figure 5: Synthesis outperforms best-trajectory selection; selection variant underperforms on deep research.

Tool usage statistics reveal that AggAgent relies primarily on keyword search for efficient evidence retrieval, invoking full segment reads only when necessary for deeper validation.

Figure 6: Average tool calls per query; search_trajectory dominates, get_segment reserved for selective deep reads.

Qualitative Behaviors

Qualitative analysis highlights four key capabilities:

1. Minority answer identification: recognizing correct solutions even when they are a minority.
2. Disagreement resolution: resolving contradictory evidence across trajectories.
3. Cross-trajectory synthesis: composing correct solutions from individually incorrect rollouts.
4. Heuristic interpretation: leveraging (but not solely relying on) confidence and majority signals.

These behaviors illustrate AggAgent's capacity for nuanced, evidence-grounded aggregation.

Figure 7: Qualitative examples showcase minority answer identification, disagreement resolution, synthesis, and heuristic guidance.

Analysis of Performance Scaling

AggAgent's performance scales robustly with parallel sample size $K$. Unlike heuristic baselines, which plateau or degrade on poorly calibrated tasks, AggAgent maintains consistent gains by leveraging full trajectory evidence rather than shallow confidence or frequency signals.

Performance-efficiency tradeoff is optimal, as aggregation cost remains nearly constant with varying $K$ due to the in-memory tool mechanism.

Figure 8: Performance-efficiency tradeoff across six benchmarks with GLM-4.7-Flash at varying K.

Figure 9: Performance-efficiency tradeoff with Qwen3.5-122B as both rollout agent and aggregator.

Figure 10: Performance-efficiency tradeoff with MiniMax-M2.5 as both agent and aggregator.

Confidence calibration analysis demonstrates that AggAgent scales reliably even on tasks where confidence-based selection fails due to poor calibration.

Figure 11: Confidence calibration across benchmarks; only agentic aggregation methods reliably benefit from scaling.

Implications and Future Directions

Practically, AggAgent enables efficient deployment in production agentic systems where multiple parallel rollouts are generated and aggregated. It opens the door to asymmetric multi-agent architectures with strong aggregators. Theoretically, agentic aggregation sets a new paradigm for scalable evidence synthesis in complex tasks, decoupling the aggregation cost from trajectory length and count.

Potential future developments include fine-tuning aggregator agents, randomized tool selection policies for efficient exploration, adaptive tool invocation guided by uncertainty, and extending agentic aggregation principles to sequential scaling and lifelong learning.

Conclusion

Agentic aggregation via AggAgent is a principled and cost-efficient approach for parallel test-time scaling of long-horizon agentic tasks. Empirical results establish its superiority across diverse task settings and model families, with strong synthesis capabilities and optimal efficiency. This work significantly advances the state-of-the-art in agentic aggregation, providing both practical methods and theoretical foundations for robust AI system scaling.