AgentIR: End-to-End Retrieval and Code Synthesis

Updated 12 March 2026

AgentIR is a framework that treats reasoning traces as first-class elements to enable unified retrieval and program synthesis.
It employs a joint embedding method combining agent reasoning and query data with contrastive loss to significantly enhance retrieval accuracy.
The framework uses an agentic graph intermediate representation for systematic task refinement, ensuring traceability and reproducibility in code generation.

AgentIR is a framework paradigm for end-to-end agentic workflows, primarily concerned with leveraging advanced retrieval, reasoning-awareness, and verification for LLM agents tasked with information synthesis and software generation. The AgentIR approach manifests in two major forms: (1) as a reasoning-aware retrieval architecture for Deep Research agents, and (2) as an agentic graph intermediate representation for program synthesis. This dual use serves both the information retrieval and the software engineering domains, with a particular emphasis on systematic traceability, reproducibility, composability, and robust evaluation.

1. Definitions, Motivation, and Scope

AgentIR refers to a class of frameworks in which agent reasoning traces and the intermediate results of natural language processing are treated as first-class citizens in retrieval and workflow compilation. In reasoning-aware retrieval, AgentIR jointly embeds the agent’s explicit reasoning trace (τₜ) together with the search query (qₜ), capturing the entire interaction context up to the search call, including agent hypotheses, contextual disambiguation, and intent specification. In agentic compilation, AgentIR denotes the use of structured intermediate representations (typed DAGs annotated with effect and type metadata) to capture the progressive refinement of natural language tasks into executable programs (Chen et al., 4 Mar 2026, Chivukula et al., 24 Nov 2025).

The motivation arises from the limitations of conventional retrieval and code-generation systems, which neglect agent-internal state, leading to ambiguous query results or non-reproducible workflows. The AgentIR paradigm closes this gap by formalizing reasoning traces and intermediate representations, thus enabling agents to exploit their explicit contextual and operational state at all execution stages.

2. Reasoning-Aware Retrieval: Formalization and Methodology

The AgentIR retrieval paradigm is defined by the joint embedding model $f(\tau_t, q_t)$ , where $\tau_t$ is the agent’s natural language reasoning trace immediately preceding query $q_t$ . The retrieval pipeline is as follows:

Interaction History: $H_T = (\tau_1, q_1, o_1, ..., \tau_T, q_T, o_T)$ , capturing all agent reasoning and actions.
Joint Embedding: Queries are encoded as $f(\tau_t, q_t) \in \mathbb{R}^d$ ; documents are encoded as $g(d) \in \mathbb{R}^d$ .
Similarity Scoring: Retrieval relevance is determined by $S(f(\tau_t, q_t), g(d)) = \langle f(\tau_t, q_t), g(d) \rangle$ .
Objective: A contrastive loss over joint input pairs ([τₜ, qₜ], positive $d^+$ , and negatives $d^-$ ) is used, with hard negatives selected via LLM-based reranking:

$L = -\log \frac{\exp( sim([ \tau_t, q_t ], d^+) / T )}{\exp( sim([ \tau_t, q_t ], d^+) / T ) + \sum_{d^-} \exp( sim([ \tau_t, q_t ], d^- ) / T )}$

where $\tau_t$ 0 is the temperature coefficient (Chen et al., 4 Mar 2026).

This approach outperforms both sparse (BM25) and conventional dense retrievers, achieving substantial improvements in benchmarked accuracy and requiring fewer retrieval calls for Deep Research agents.

3. Data Synthesis and Training Regime

High-quality training data for reasoning-aware retrieval are synthesized via the DR-Synth pipeline:

Agent Rollouts: For a question-answer-document tuple (Q, A, P) from standard QA datasets, an agent is rolled out to collect the trajectory of search sub-queries $\tau_t$ 1.
Oracle Reranking: For each sub-query $\tau_t$ $τ_{t}$ 2:
- Retrieve top-50 candidate docs using a query-only retriever.
- Merge in gold-positive docs $\tau_t$ 3.
- Use an LLM-based reranker (prompted with (Q, A, $\tau_t$ 4)) to produce a relevance-ordered list, assigning the top-1 to $\tau_t$ 5 and the last 7 as $\tau_t$ 6.
This process yields [ (τₜ, qₜ), $\tau_t$ 7, $\tau_t$ 8 ] instances (Chen et al., 4 Mar 2026).

The AgentIR-4B model builds on Qwen3-Embedding-4B (with LoRA adaptation), incorporating input formatting that explicitly separates reasoning and query via prompt templating. Training employs a dual encoder architecture and is conducted with long-context settings (8,192 max query tokens, 4,096 document tokens) and multi-epoch fine-tuning on datasets of >5,000 synthesized instances.

4. Intermediate Representation for Agentic Compilation

In software engineering contexts, AgentIR signifies a typed, effect-aware code DAG $\tau_t$ 9. The components are:

$q_t$ 0: nodes representing tasks or subtasks (with payloads of type: text, schema, spec, code).
$q_t$ 1: edges denoting dependencies.
$q_t$ 2: type floor (with a six-tier lattice: TEXT, TYPED, SPEC, STUB, SHIM, PURE).
$q_t$ 3 Effects annotates side effects (e.g., FileRead, Network, DBWrite).

Type floors order the refinement progression:

$q_t$ 4

ensuring only upward transitions in compilation. Each node $q_t$ 5 is represented as $q_t$ 6, with context for refinement. Data-flow edges propagate pure values, while control/effect edges support runtime effect management and rollback (Chivukula et al., 24 Nov 2025).

The hierarchical graph rewrite rules promoting nodes through type floors are locality-preserving, enabling parallel upgrading of independent subgraphs. Fallback rules (e.g., node decomposition, virtual stub insertion) handle LLM failures or unresolved logic.

5. End-to-End Software Agent Workflow and Toolchain

The agentic IR paradigm is operationalized by the Agint toolchain:

dagify: Multi-pass DAG compiler for composition, refinement, and resolution of natural language instructions through type floors.
dagent: JIT hybrid runtime/interpreter executing the typed DAG with effect tracking, speculative execution, parallelism, and isolation of side effects.
schemagin: Schema generator for producing and refining database schemas from NL.
datagin: Data synthesizer/transformer for ingesting and transforming test data according to schema.
Interaction Flow:

Compose NL specification to a plain TEXT DAG.
Refine with contextual prompts.
Resolve up through TYPED, SPEC, STUB, SHIM, PURE.
Compile to target artifact (e.g., LangChain, CrewAI).
Execute/interpret via dagent, partitioning and paralleling tasks, with deterministic effect management.

The entire toolchain is orchestrated using Flyte for programmatic LLM calls and Hydantic for structured outputs, supporting accelerated development and robust, reproducible outputs (Chivukula et al., 24 Nov 2025).

6. Verification and Human-in-the-Loop Evaluation

AgentIR frameworks incorporate comprehensive evaluation via automatic and human-in-the-loop pipelines. The FROAV implementation exemplifies a full system for Retrieval-Augmented Generation (RAG) and agent verification:

Retrieval: PostgreSQL+pgvector (HNSW) with OpenAI embeddings, cosine similarity for vector search.
Generation: Prompt engineering injects context and question into LLM calls for controlled output.
Verification: "LLM-as-a-Judge" multi-model evaluation, scoring outputs along reliability, completeness, understandability, and relevance, with outputs aggregated via the median:

$q_t$ 7

Feedback Loop: Human annotators (via Streamlit UI) score agent answers, enabling correlation analysis and systematic prompt refinement (Lin et al., 12 Jan 2026).

No-code orchestration (n8n) and modular FastAPI microservices enable researchers to rapidly prototype and evaluate AgentIR pipelines without extensive infrastructure code.

7. Empirical Results, Limitations, and Future Directions

Benchmarks such as BrowseComp-Plus demonstrate that reasoning-aware AgentIR retrieval achieves 66.27% accuracy for Tongyi-DeepResearch, compared to 48.67% (Qwen3-Embed-4B), 50.72% (Qwen3-Embed-8B), and 33.98% (BM25), reducing average search calls per question from ~33 to ~26. The “visit” tool (full document retrieval) further raises AgentIR accuracy to 68.07%. AgentIR outperforms LLM-based reranking by ~10 percentage points without additional inference cost (Chen et al., 4 Mar 2026).

Limitations include the necessity for explicit reasoning traces (limiting black-box agent applicability), dependence on LLM scoring accuracy for data synthesis and reranking, and the marginal gains from including full historical traces due to redundancy. Future research focuses on principled context engineering for retrievers, cumulative embeddings over novel reasoning increments, and distillation-based adaptation to closed-weight agents.

A plausible implication is that the integration of AgentIR principles—reasoning trace exposure, type- and effect-aware IRs, and systematic human/LLM verification—will be necessary for scalable, interpretable, and reliable agentic systems in both retrieval and code synthesis domains.