Select-then-Solve: Paradigm Routing as Inference-Time Optimization for LLM Agents

Abstract: When an LLM-based agent improves on a task, is the gain from the model itself or from the reasoning paradigm wrapped around it? We study this question by comparing six inference-time paradigms, namely Direct, CoT, ReAct, Plan-Execute, Reflection, and ReCode, across four frontier LLMs and ten benchmarks, yielding roughly 18,000 runs. We find that reasoning structure helps dramatically on some tasks but hurts on others: ReAct improves over Direct by 44pp on GAIA, while CoT degrades performance by 15pp on HumanEval. No single paradigm dominates, and oracle per-task selection beats the best fixed paradigm by 17.1pp on average. Motivated by this complementarity, we propose a select-then-solve approach: before answering each task, a lightweight embedding-based router selects the most suitable paradigm. Across four models, the router improves average accuracy from 47.6% to 53.1%, outperforming the best fixed paradigm at 50.3% by 2.8pp and recovering up to 37% of the oracle gap. In contrast, zero-shot self-routing only works for GPT-5 at 67.1% and fails for weaker models, all trailing the learned router. Our results argue that reasoning paradigm selection should be a per-task decision made by a learned router, not a fixed architectural choice.

• The paper demonstrates that task-specific paradigm selection improves LLM performance by up to 17.1 percentage points compared to any fixed approach.
• It rigorously evaluates six reasoning paradigms across diverse tasks, revealing trade-offs between structured reasoning protocols and computational cost.
• The embedding-based router achieves higher cost-efficiency and significantly narrows the gap to oracle performance, highlighting adaptive meta-control in LLM agents.

Select-then-Solve: Paradigm Routing as Inference-Time Optimization for LLM Agents

Motivation and Context

The paper "Select-then-Solve: Paradigm Routing as Inference-Time Optimization for LLM Agents" (2604.06753) performs a comprehensive and controlled evaluation of six distinct reasoning paradigms for LLM-based agents. The central inquiry is whether improvements in task performance derive primarily from advances in model architecture or from the inference-time reasoning protocol applied. Notably, the authors provide empirical results demonstrating that reasoning structure can enhance, degrade, or have neutral impact on task performance, contingent on the task type and underlying LLM. No single paradigm dominates across tasks, and substantial accuracy gains can be realized through per-task paradigm selection.

Figure 1: Direct prompting (gray), best single paradigm per dataset (colored), and oracle per-task selection (blue) for GPT-5. The best paradigm differs across tasks, and the oracle substantially exceeds any fixed choice, motivating the select-then-solve approach.

Experimental Framework

The study examines six paradigms—Direct, Chain-of-Thought (CoT), ReAct, Plan-Execute, Reflection, and ReCode—wrapped around four frontier LLMs (GPT-5, Gemini-3-Flash, Qwen3-Max, Qwen3-30B) and evaluated on ten diverse benchmarks spanning code generation, mathematics, QA, knowledge, and tool-intensive agent tasks. All paradigms share a unified implementation, differing only in inference organization and tool invocation. Tool-using paradigms employ both web-search and Python code execution interfaces. The evaluation encompasses 761 tasks per model-paradigm pair, totaling $\approx$18k completed runs.

The Marginal Value of Reasoning Structure

The empirical results establish that the marginal value of reasoning structure is sharply task-dependent. For information-seeking tasks, structured paradigms that incorporate tools yield dramatic improvements (e.g., ReAct improves over Direct by 44pp on GAIA). Conversely, on tasks where parametric knowledge suffices, notably in knowledge-centric or mathematical benchmarks, minimal or no reasoning structure matches or surpasses complex paradigms at a fraction of the computational cost. In some cases, structure is actively detrimental: CoT degrades performance by 15pp on HumanEval due to step-by-step reasoning disrupting code synthesis.

Figure 2: Success rate heatmap for GPT-5 across paradigms (rows) and datasets (columns). Clear task-paradigm interactions are visible: no single row dominates all columns.

Paradigm complementarity is observed both at the dataset and individual task level. Jaccard similarity analyses between paradigm success sets reveal genuine complementarity: lower overlap indicates distinct subsets of tasks are solved by different paradigms.

Figure 3: Jaccard similarity between paradigm success sets (aggregated across all models). Lower values indicate greater complementarity; paradigms solve different subsets of tasks.

Routing and the Select-then-Solve Framework

The key finding is that oracle per-task paradigm selection consistently outperforms the best fixed paradigm by 17.1pp average accuracy, indicating substantial headroom for optimized inference strategies. To practically approach the oracle, the authors introduce the select-then-solve pipeline—a paradigm router that analyzes task text and routes the task to the most suitable paradigm, including Direct when appropriate.

Figure 4: The select-then-solve pipeline. A Paradigm Selector encodes, classifies, and routes each task to one of six paradigms. Only the selected paradigm is executed.

Three routing strategies are evaluated: handcrafted features, embedding-based features (using text-embedding-3-small), and zero-shot self-routing. Embedding-based routers consistently outperform Direct and best-single baselines, improving average accuracy from 47.6\% (Direct) to 53.1\%, and recovering up to 37\% of the oracle gap, with gains largest for lower-capability models.

Figure 5: Router comparison across four models. The embedding router (green) consistently outperforms Direct and Best-single baselines. Self-routing (red) shows mixed results. The oracle (blue) shows substantial remaining headroom.

Advanced routers make fine-grained task-level decisions, optimizing for both accuracy and cost-efficiency. The embedding router achieves higher accuracy than ReAct at half its token cost, showing significant improvements in cost-effectiveness.

Figure 6: Cost-effectiveness scatter plot: success rate vs. average tokens per task. Points closer to the top-left corner represent better cost-efficiency.

Oracle gap recovery analysis shows the embedding-based router consistently narrows the gap to the oracle upper bound, particularly for Gemini and GPT-5.

Figure 7: Oracle gap recovery across models. Each group shows the progression from Direct (gray) through handcrafted router (orange) and embedding router (green) to the Oracle upper bound (blue). The embedding router consistently narrows the gap, with the largest recovery on Gemini.

Self-Routing and Meta-Reasoning Failure Modes

Self-routing—in which the model is prompted to select its own paradigm—only benefits the strongest LLM (GPT-5), achieving 67.1\% accuracy, but fails for weaker models (Qwen3-Max and Qwen3-30B), which over-select tool-using paradigms and underperform their Direct baselines. Distribution analysis indicates self-routing shows degenerate biases, whereas the learned router produces model-adapted, diversified distributions.

Figure 8: Paradigm distribution comparison: learned router (left) vs. zero-shot self-routing (right). The learned router produces diverse, model-adapted distributions, while self-routing shows degenerate biases (GPT-5 selects Direct 100\%; Qwen3-30B over-selects ReAct at 48\%).

The results underscore that reliable paradigm selection is a distinct meta-reasoning capability not readily achieved via zero-shot prompting except for frontier models.

Practical and Theoretical Implications

The select-then-solve paradigm routing approach offers a principled, inference-time optimization axis orthogonal to model size or architecture. Structured reasoning protocols are not universally beneficial; agent performance is maximized not by always imposing complex scaffolds, but by matching the paradigm to the task. Given substantial accuracy and cost improvements, this routing framework can be integrated within agent architectures as a lightweight, adaptive control layer. The demonstrated complementarity suggests further development in paradigm composition, multi-objective routing (balancing accuracy and cost), and cross-model router transfer.

The findings suggest that future developments in AI agent systems will increasingly prioritize adaptive meta-control strategies that optimize for inference-time computation allocation, rather than static model or scaffold selection. Fine-tuned routers may ultimately become standard components in agent stacks, particularly as LLMs diversify in capability and deployment scenarios.

Conclusion

The paper provides rigorous evidence that inference-time reasoning paradigms are complementary and task-dependent, and that embedding-based routers can approach oracle-level selection, outperforming any fixed paradigm at substantially improved cost-efficiency. Self-routing remains unreliable except for the most capable LLMs, highlighting paradigm selection as a distinct meta-reasoning challenge. Optimal agent performance depends not on elaborate reasoning scaffolds, but on the ability to selectively deploy them only when beneficial. The select-then-solve framework establishes paradigm routing as a critical axis for LLM agent optimization, with practical implications for adaptive agent design and theoretical ramifications for understanding the limits of reasoning structure in contemporary LLMs.