Papers
Topics
Authors
Recent
Search
2000 character limit reached

Graph-Based Agentic AI with LangGraph: Workflow Pathways for Long-Running Stateful Business Processes

Published 21 Jul 2026 in cs.AI and cs.SE | (2607.19297v1)

Abstract: This paper is a practitioner guide to graph-based workflow pathways for long-running, stateful, multi-step generative AI systems in business processes. Rather than treating LangGraph, a low-level orchestration framework for stateful agents, as a model-quality benchmark target, we present three executable recipes -- SQL analytics with repair loops, agentic retrieval-augmented generation with evidence gating, and human-in-the-loop policy review with interrupt and checkpoint recovery -- to show how typed state, conditional routing, deterministic tools, retries, interrupts, checkpoints, and traces fit together. LangGraph is positioned by workflow-complexity fit, not as a universal default: simpler ReAct-style or plain SDK loops may be better for basic tool use, schema-first tools for structured extraction and validation, and DSPy when prompt or program optimization is the main goal. Each recipe explains when LangGraph is worth the extra structure and which implementation patterns make routes, pauses, and audit trails explicit product behavior rather than hidden prompt logic.

Summary

  • The paper presents LangGraph, a framework that enables durable, auditable, stateful workflows with explicit pause/resume and human-in-the-loop operations.
  • The methodology leverages a graph-based state abstraction with typed state objects, nodes, and conditional routing to enforce precise workflow control.
  • The study demonstrates practical applications, including SQL analytics, retrieval-augmented generation, and HITL policy reviews, emphasizing auditability and error recovery.

Graph-Based Agentic AI with LangGraph: Structured Orchestration for Stateful Business Workflows

Motivation and Problem Framing

The paper "Graph-Based Agentic AI with LangGraph: Workflow Pathways for Long-Running Stateful Business Processes" (2607.19297) offers a practitioner-focused examination of LangGraph, a low-level orchestration framework for stateful agentic workflows. The key motivation is to provide transparency, durability, and explicit control in LLM-powered business applications that require long-duration, branched, auditable sequences with persistent state and human-in-the-loop (HITL) operations. Rather than optimizing for model performance or prompt engineering, the paper delineates the types of workflow requirements—such as pause/resume, conditional routing, retry/repair paths, traceability, and durable state—that justify the graph-based paradigm.

The paper argues that conventional approaches, including ReAct-style reasoning-act loops [yao2023react], schema-first validation agents [geng2025jsonschemabench, pydanticai2026], and program-optimization frameworks like DSPy [khattab2024dspy], are each best suited for distinct problem classes. Graph orchestration with LangGraph is uniquely valuable when workflow management, not just LM invocation, is central to the software product's contract.

LangGraph Architecture and Principles

LangGraph exposes core primitives—typed shared state, nodes, conditional edges, persistent checkpoints, and explicit interrupts—as first-class constructs. Unlike higher-level agent libraries or end-to-end frameworks, it does not subsume domain tool logic or define modeling protocols but makes workflow control flow concrete and inspectable.

The architecture maps to a StateGraph abstraction, in which:

  • Typed state objects carry user inputs, intermediate results, route decisions, retry counts, and outputs across nodes,
  • Nodes represent executable code units: tool invocations, validation steps, retrieval actions, or review pauses,
  • Conditional edges (routing functions) determine subsequent nodes based on explicit state evaluation,
  • Checkpointing is integrated for durable persistence and HITL pause/resume,
  • Traceability and auditability are by design: workflow decisions and their rationale are manifest in state and routing, not buried in logs or LM outputs.

The separation of concerns is emphasized both in repository layout (graph structure versus node logic) and in the interaction between workflow and domain semantics.

Executable Recipes: Three Canonical Pathways

The paper advances three executable "recipes" as canonical instantiations where LangGraph's abstraction provides engineering advantages over alternative paradigms:

1. SQL Analytics with Repair Loops

Use Case: Natural language-to-SQL agents where validation and execution failures require persistent, auditable, and retryable repair loops.

  • Core State: Tracks question, schema, candidate SQL, error messages, retry attempts, query results, and status.
  • Key Structure: Schema lookup, SQL generation, validation, execution, business summary.
  • Routing: Explicit conditional logic directs the path on validation/execution failure—back to generation up to a retry limit, or to terminal failure; successful execution routes to summarization.

This pattern is justified in audit-focused or regulated settings, or where non-terminal failures must be managed distinctly from fatal errors.

2. Agentic RAG with Evidence Gating

Use Case: Retrieval-Augmented Generation where decisions to answer, retry, or refuse are governed by explicit evidence quality thresholds.

  • Core State: Includes question, retrieved documents, evidence grade, answer, citations, retry count, and status.
  • Key Structure: Question analysis, targeted retrieval, evidence grading, answer generation, citation verification.
  • Routing: Weak evidence and failed verification propagate through retry or clarify loops; only sufficient support proceeds to finalized responses.

This enforces product behavior—such as non-hallucination, failure closure, or citation gating—as externalized workflow policy, not silent prompt heuristics.

3. Human-in-the-Loop (HITL) Policy Review

Use Case: High-risk, compliance-sensitive decision processes that require pausing for human input and resuming with preserved state and audit records.

  • Core State: Captures draft answer, risk assessment, identified policies, approval, reviewer feedback, and final decisions.
  • Key Structure: Drafting, risk scoring, conditional branching for human review, feedback application, and decision logging.
  • Routing: High risk, as determined by explicit state, triggers an interrupt and checkpoint; system execution halts and resumes only upon authenticated human feedback, enabling robust compliance and governance.

This paradigm is central where durable workflow boundaries, strong audit trails, and resumable human-in-the-loop operations are contractually mandated.

Design Patterns and Technical Implementation

Across all recipes, the framework generalizes several architectural patterns:

  • Shared typed state and deterministic tool nodes underpin explicit, inspectable workflow progression, facilitating debugging, reproducibility, and audit.
  • Conditional routing and branch labels make operational decisions explicit, allowing programmatic testability and regulatory traceability.
  • Retry loops and escalation are encoded in routing functions, not in implicit or foreign logic.
  • Checkpointing and interrupt primitives enable decoupled, asynchronous human review without loss of execution fidelity.
  • Audit traces are constructed as part of the state schema and transitions, not as after-the-fact log artifacts.

The paper provides concrete, minimal Python implementations of each workflow archetype, highlighting the separation of the orchestration layer (graph and routing functions) from the application/business logic (node implementations).

Trade-offs, Limitations, and Framework Selection

The authors underscore the added cognitive and engineering overhead of the graph abstraction: state schemas, explicit builders, routing and recovery logic, checkpointing, and invocation configuration, which are often unjustified for simple, linear, or stateless tasks. Overuse leads to unnecessary complexity; the decision process should be requirements-driven.

LangGraph is strongly preferred when:

  • The workflow must pause and resume (especially for human approval or long-running external dependencies),
  • Explicit state and branching are central to the operational semantics,
  • Tool failures require repeatable, testable, and auditable recovery paths,
  • Precise route tracing and compliance are product requirements,
  • Multiple heterogeneous components (tools, model calls, reviews) must operate over shared, durable state.

Alternative stacks (plain SDK, schema-first, DSPy) are preferred when workflows are linear, stateless, or focused on prompt/program optimization rather than orchestration.

Implications and Speculation on Future Directions

The formalization of controllable, auditable, and stateful workflows is increasingly important as LLM-driven systems move from isolated question-answering or tool-use demos into production business contexts, regulatory environments, and long-running operational processes. The agentic/graph-based approach provides a structured methodology for managing system state, human supervision, and explicit error handling.

As large-scale AI systems assume more complex roles in enterprise, compliance, and regulated domains, demand for transparent, auditable, and robust workflow orchestration will increase. Future developments are likely to merge the strengths of schema-driven validation and graph-based control planes, perhaps further abstracted for declarative specification while maintaining deterministic and auditable semantics. The pressure for seamless HITL integration, resilient checkpointing, and operational traceability will continue to drive architectural choices, with abstraction boundaries between orchestration and tool logic remaining a key site of engineering trade-off.

Conclusion

This paper advances the engineering utility of graph-based workflow orchestration in LLM-powered business applications, providing executable, auditable patterns for SQL analytics, evidence-gated RAG, and HITL policy review. The work clarifies when and why explicit graph orchestration is justified, codifies reusable design patterns, and positions LangGraph as a control-plane primitive for applications requiring persistent state, explicit branching, durable pause/resume, and traceable route history. The decision to employ such an abstraction must be governed tightly by workflow complexity, audit requirements, and the need for transparent operational contracts.

Whiteboard

Explain it Like I'm 14

A simple explanation of “Graph-Based Agentic AI with LangGraph: Workflow Pathways for Long-Running Stateful Business Processes”

Overview: What is this paper about?

This paper shows how to build AI assistants that follow clear, reliable steps—like a flowchart—especially for business tasks that can take a while, may fail and retry, or need a human to approve a decision. It uses a toolkit called LangGraph. LangGraph is not a LLM; it’s the “traffic controller” that organizes the steps an AI takes, remembers progress, and lets you pause and resume safely.

Think of it like this:

  • Graph = a flowchart of steps and decisions.
  • State = a backpack the AI carries from step to step, holding everything it has learned so far.
  • Checkpoint = a save point in a video game, so you can come back later without losing progress.
  • Interrupt = a planned pause where a human can look things over.

Goals: What questions does the paper answer?

In simple terms, the paper tries to answer:

  • When should you use LangGraph, and when is a simpler setup better?
  • How do you make AI workflows that can retry safely, show their work, pause for human review, and then continue?
  • What are solid “recipes” (step-by-step patterns) for three common business tasks using AI?

Approach: How did they do it?

Instead of running lots of accuracy tests, the authors give practical, runnable “recipes” that developers can copy and adapt. Each recipe is a small, working flowchart with:

  • Typed state (clearly labeled fields in the AI’s “backpack”)
  • Nodes (small steps like “get documents,” “check risk,” or “run a query”)
  • Conditional routes (traffic lights that choose the next step based on the current state)
  • Retries (controlled do-overs)
  • Interrupts and checkpoints (pauses for human approval, then resume from a save point)
  • Traces (a history of which path was taken and why, for auditing)

They also include a decision guide explaining when to pick LangGraph versus simpler tools:

  • Plain SDK/ReAct: good for short, simple tasks.
  • Schema-first tools: good for strict, structured outputs (like filling out reliable forms).
  • DSPy: good when the main job is tuning prompts/programs to be better.
  • LangGraph: best when you need branching steps, saved progress, retries, human approvals, and audit trails.

Main examples: What did they build and why does it matter?

The paper provides three clear recipes that show where a flowchart-style AI is worth it.

Here are the three recipes and what they teach:

  • SQL analytics with repair loops
    • What it does: Turns a question into a database query (SQL), checks if the query is valid, fixes it if it’s wrong, runs it, and then summarizes the result.
    • Why it matters: Errors aren’t just crashes—they become part of the workflow with controlled retries. This makes data questions more reliable and auditable.
  • Agentic RAG (Retrieval-Augmented Generation) with evidence gating
    • What it does: The AI first looks up information, grades the quality of what it found, and only answers if the evidence is strong enough. If not, it tries again, asks for clarification, or refuses to answer.
    • Why it matters: It stops the AI from guessing. Weak evidence triggers a detour or a polite “I don’t have enough proof,” which is safer and more honest.
  • Human-in-the-loop (HITL) policy review with interrupt and checkpoint
    • What it does: Drafts a decision, scores its risk, and if it’s high-risk, pauses the workflow so a human can review and approve or correct it. Then it resumes and records a proper decision log.
    • Why it matters: Some decisions must be reviewed by people. Checkpoints and interrupts make this safe, resumable, and trackable—great for compliance and accountability.

Key takeaways: What did they find?

In everyday language, here’s what the authors are saying:

  • Don’t overcomplicate simple tasks. If your AI just needs to answer a quick question or call one tool, a simple loop is enough.
  • Use LangGraph when the path matters. If your app needs to:
    • Branch based on risk, evidence quality, or validation results
    • Retry with a plan instead of crashing
    • Pause for human approval and then resume later
    • Keep a clean history of what happened and why
    • …then a flowchart-style design (graph) is the right tool.
  • Make decisions explicit. Don’t hide important rules inside a long prompt. Turn them into visible routes so you can test them, explain them, and audit them.

Why this matters: What’s the impact?

This approach helps teams build AI features that are:

  • More reliable: Clear repair paths and retries prevent silent failures.
  • Easier to govern: You can see which route was taken and why.
  • Safer for business and compliance: Human approvals and decision records are built in.
  • Practical to maintain: The flowchart makes debugging and updates simpler.

In short, if your AI work is more like a quick chat, keep it simple. But if it’s more like a business process with rules, approvals, and save points, LangGraph’s graph-based workflow makes that complexity manageable, transparent, and trustworthy.

View Paper Prompt  View All Prompts 

Knowledge Gaps

Below is a concise list of concrete knowledge gaps, limitations, and open questions that remain unresolved in the paper. These highlight where additional evidence, design guidance, or engineering patterns are needed for practitioners and researchers to advance the work.

  • No empirical evaluation of the three recipes (accuracy, latency, cost). Lacks quantitative comparisons against simpler baselines (plain SDK/ReAct), schema-first tools, or DSPy across representative tasks.
  • Missing decision thresholds and calibration methods for state fields such as evidence_grade and risk_level (e.g., how to set, validate, and monitor these thresholds; inter-annotator agreement; drift handling).
  • Unspecified reliability of evidence grading and citation verification (metrics, false-positive/false-negative rates, gold standards, adjudication protocols).
  • No guidance on selecting retry budgets or adaptive retry policies (e.g., budget sizing relative to error types, dynamic policies based on observed improvement, early-stopping criteria).
  • Absent cost models for repair loops and HITL interrupts (expected retries per task, time-to-resolution distributions, reviewer-time budgets, ROI).
  • Lacks performance characterization of LangGraph overhead (CPU/memory footprint, compilation/invocation cost, impact on tail latency vs plain loops).
  • No scalability patterns for distributed deployment (multi-worker execution, sharding large graphs, horizontal scaling, placement of nodes, queueing/backpressure).
  • Checkpointing semantics under load are unspecified (exactly-once vs at-least-once execution, idempotency strategies for nodes, deduplication on resume, failure injection tests).
  • Missing guidance for durable checkpointers in production (storage backends, schema design for checkpoints, compaction/retention, throughput limits, disaster recovery).
  • No migration strategy for evolving graphs and typed state (backward compatibility, schema evolution, in-flight thread upgrades and rollbacks).
  • Concurrency and race-condition handling are not addressed (simultaneous resumes, duplicate events, out-of-order messages, transactional boundaries with external systems).
  • Absent workflow-level transactional guarantees with side-effecting tools (compensation/SAGA patterns, rollback semantics, retries with non-idempotent tools).
  • No integration patterns with external workflow engines (e.g., Temporal) beyond conceptual mention (who owns retries/checkpoints, boundary definitions, observability integration).
  • Long-memory and cross-session state are out of scope (designs for episodic/semantic memory, forgetting policies, retrieval across cases, privacy-preserving memory).
  • Observability and monitoring gaps (standard trace schema, sampling strategies, dashboards, alerting on stuck interrupts, SLA/SLO definitions for pause durations).
  • Streaming UX and user-cancellation handling are not covered (mid-node streaming, cooperative cancellation, partial outputs consistency).
  • Security and privacy are unaddressed for stored state (PII handling, encryption at rest/in transit, RBAC for reviewers, least-privilege on tools, tamper-evident audit logs).
  • Compliance and governance questions remain (GDPR/CCPA deletion, retention schedules for decision_record, provenance/lineage tracking, policy-change auditability).
  • Bias, fairness, and accountability in risk scoring and review escalation are not analyzed (calibration by cohort, bias detection, appeal workflows, reviewer consistency).
  • No UI/UX design patterns for HITL review (work distribution, batching, avoiding rubber-stamping, reviewer load modeling, feedback quality controls).
  • Limited testing guidance beyond contract tests (property-based tests for routing, chaos testing for checkpoint/retry, flaky-LLM simulation, determinism under mocks vs live).
  • Lack of standardized metrics for “workflow quality” (repair success rate, escalation precision/recall, mean retries-to-success, time-to-decision, audit completeness).
  • Missing evaluation of model dependence and portability (do recipes hold across different LLMs and tool backends; sensitivity analyses).
  • Node boundary selection lacks operational criteria (how to quantify good boundaries; maintainability and debuggability metrics; anti-patterns).
  • No examples of composing schema-first validation or DSPy optimization inside graph nodes (end-to-end patterns, interface contracts, shared telemetry).
  • Unaddressed dataset leakage and safe RAG practices (privacy-preserving retrieval, citation attestation methods, anti-hallucination hard constraints).
  • No patterns for adaptive or learned routing (bandit/RL policies for route_after_* functions, offline policy optimization from logs).
  • Absence of backpressure and queue management strategies for long-running or bursty workloads (interrupt queues, prioritization, starvation avoidance).
  • Unclear guidance on provenance and reproducibility of decisions (hashing of inputs/outputs, artifact storage, deterministic tool configurations, seed management).
  • No real-world case studies with production telemetry (operational incidents, remediation patterns, cost/benefit analyses, stakeholder satisfaction).

These gaps suggest concrete avenues for future work: controlled experiments against baselines, operational runbooks for checkpointing and migrations, security/compliance blueprints, fairness audits for risk scoring, learned routing policies, and production case studies with end-to-end observability and cost models.

View Paper Prompt  View All Prompts 

Practical Applications

Immediate Applications

These applications can be built and deployed now using the paper’s recipes (SQL repair loop, agentic RAG with evidence gating, HITL interrupt/checkpoint) and the decision guidance on when to use LangGraph versus simpler alternatives.

  • SQL analytics copilot with controlled repair
    • Sectors: software/data platforms, finance, retail, telecom
    • What it is: Natural-language-to-SQL assistant that validates, retries, and summarizes results; routes generation/validation/execution errors to explicit repair paths rather than failing open
    • Tools/products/workflows: LangGraph StateGraph; SQL validator/executor against Postgres/Snowflake/BigQuery; business-summary node; retry budgets and audit fields (attempts, errors)
    • Assumptions/dependencies: Stable schema access; safe execution environment (read-only, row limits); clear retry budget; logs retained for audit; simpler SDK/ReAct path used when no repair/audit is needed
  • Evidence-gated enterprise RAG for policies and knowledge bases
    • Sectors: HR, legal, compliance, IT service desks, education
    • What it is: Retrieval assistant that grades evidence, refuses or retries on weak support, and verifies citations before finalizing
    • Tools/products/workflows: LangGraph nodes for question analysis, retrieval, grading, answer, citation verification; vector store/BM25; fail-closed finalize route; “retry_or_clarify” path
    • Assumptions/dependencies: Curated, up-to-date KB; well-tuned retriever; defined thresholds for evidence sufficiency; acceptance that some queries will terminate “unsupported” instead of fabricating answers
  • Human-in-the-loop (HITL) policy/compliance review with pause/resume
    • Sectors: HR, legal, finance (KYC/AML), healthcare compliance
    • What it is: Draft–risk score–interrupt–feedback–finalize process that pauses at high risk and resumes from a checkpoint after reviewer input, producing a decision record
    • Tools/products/workflows: LangGraph interrupt() with durable checkpointer (SQLite/Postgres); reviewer UI; risk scoring node; structured decision_record
    • Assumptions/dependencies: Human reviewers and SLAs; durable storage for threads; documented risk thresholds; access control and PII safeguards
  • Customer support triage and escalation with explicit routes
    • Sectors: customer service, telecom, SaaS
    • What it is: Classify → retrieve → draft → escalate if confidence/evidence is low; routes and audit trail make handoffs traceable
    • Tools/products/workflows: Conditional routing on confidence/evidence grade; human escalation interrupt; help-center retriever
    • Assumptions/dependencies: Labeled intents, escalation criteria, and SLAs; KB coverage; integration with ticketing (e.g., Zendesk/Jira)
  • Audit-ready AI decision trails (GRC)
    • Sectors: regulated industries (banking, insurance, healthcare, public sector)
    • What it is: Make branch decisions (risk level, retries, evidence grade) and state fields first-class for compliance and post-hoc review
    • Tools/products/workflows: Typed state with status/risk/attempts/evidence_grade; immutable logs; export of route history to GRC systems
    • Assumptions/dependencies: Data-retention policies; privacy controls; mapping of state fields to regulatory requirements
  • Contract or policy clause extraction with validation and review gates
    • Sectors: legal ops, procurement, vendor risk
    • What it is: Schema-first extraction inside nodes plus graph routes for validation failures and reviewer interrupts for high-risk clauses
    • Tools/products/workflows: Pydantic/JSON Schema for validation nodes; LangGraph for retry/escalation; reviewer UI; decision records
    • Assumptions/dependencies: Canonical schemas and policies; redaction for sensitive content; defined high-risk triggers
  • Clinical/technical guideline lookup with refusal on weak evidence
    • Sectors: healthcare, engineering QA
    • What it is: Evidence-gated RAG that either cites verified guidelines or declines; optional clinician/engineer review interrupt for ambiguous cases
    • Tools/products/workflows: Retrieval + grading + citation verification nodes; interrupt path for specialist review; fail-closed finalize
    • Assumptions/dependencies: Approved guideline repositories; strict refusal policy; governance for medical/engineering disclaimers
  • Engineering change-control assistant with approvals
    • Sectors: manufacturing, DevOps/SRE
    • What it is: Draft change plan, risk assess, interrupt for approvers, resume and log decision record
    • Tools/products/workflows: Risk scoring rules; interrupt/checkpointer; integration with CI/CD or PLM systems
    • Assumptions/dependencies: Approval hierarchy; rollback policies; traceability requirements
  • Course/pedagogy assistants that gate on weak citations
    • Sectors: education
    • What it is: RAG tutor that refuses unsupported claims and can pause to request teacher approval for sensitive topics
    • Tools/products/workflows: Evidence grading; teacher interrupt; classroom content repository
    • Assumptions/dependencies: School policy thresholds; age-appropriate content filters; privacy of student data
  • Route-behavior test harness for LLM apps
    • Sectors: software/LLMOps
    • What it is: Contract tests for routing and state transitions (e.g., “weak evidence → retry,” “high risk → interrupt”)
    • Tools/products/workflows: Pytest fixtures; mock mode; assertions on state fields and terminal statuses
    • Assumptions/dependencies: Clear route contracts; separation of graph routing from domain logic
  • Data-access governance for self-serve analytics
    • Sectors: data platforms, finance
    • What it is: SQL generation with validation/risk scoring before execution; optional review interrupt for sensitive queries
    • Tools/products/workflows: Static/dynamic SQL validation; PII/dataset sensitivity checks; interrupt for data steward approval
    • Assumptions/dependencies: Data classification metadata; RBAC/ABAC enforcement; safe query sandboxes
  • Public-records/FOIA request assistants with checkpointed handoffs
    • Sectors: public sector
    • What it is: Draft response, retrieve policy/evidence, interrupt for records officer review, resume to finalize
    • Tools/products/workflows: Evidence-gated retrieval; interrupt/resume with durable threads; decision logging
    • Assumptions/dependencies: Records systems integration; statutory deadlines; redaction tooling

Long-Term Applications

These require further research, integration, or scaling (e.g., multi-agent subgraphs, cross-system orchestration, policy standardization, or automated optimization).

  • Cross-session BPM orchestration for weeks-long processes
    • Sectors: insurance claims, mortgages, compliance investigations
    • What it is: Durable graphs spanning multiple pauses, external tasks, and human roles; restart-safe across deployments
    • Tools/products/workflows: LangGraph + durable checkpointers + workflow engines (e.g., Temporal); SLAs and compensations
    • Assumptions/dependencies: Enterprise-grade persistence; idempotent tooling; organizational adoption of graph-centric BPM
  • Supervisor–worker multi-agent subgraphs with persistent memory
    • Sectors: research, software engineering, complex ops
    • What it is: Structured subgraphs coordinating specialized agents with repair loops and HITL checkpoints
    • Tools/products/workflows: LangGraph subgraphs; role agents (AutoGen/CrewAI); memory stores across sessions
    • Assumptions/dependencies: Robust delegation/coordination policies; evaluation of multi-agent safety and cost
  • Sector-certified AI process engines with standard audit schemas
    • Sectors: finance, healthcare, government
    • What it is: Standardized route/state schemas mapped to regulations; push-button export for audits
    • Tools/products/workflows: Compliance profiles; schema registries for state fields; signed decision records
    • Assumptions/dependencies: Regulator-accepted formats; third-party certification; secure provenance of state/history
  • Auto-optimized workflow graphs (compile-time and run-time)
    • Sectors: software/LLMOps, R&D
    • What it is: DSPy-style optimization of prompts/programs combined with data-driven tuning of routes, thresholds, and retry budgets
    • Tools/products/workflows: Offline metric-guided compilation; online A/B; policy learning for routing
    • Assumptions/dependencies: High-quality telemetry; guardrailed exploration; stable objectives beyond answer quality (e.g., review load, SLA)
  • Adaptive retrieval/evidence policies across heterogeneous sources
    • Sectors: enterprise search, legal/e-discovery
    • What it is: Dynamic choice among retrievers/corpora per query; evidence thresholds that adapt to risk and source trust
    • Tools/products/workflows: Meta-retrieval nodes; per-source trust models; cost-aware routing
    • Assumptions/dependencies: Source metadata; unified identity and entitlements; monitoring for drift
  • Continuous compliance monitors with exception handling
    • Sectors: GRC, cybersecurity
    • What it is: Persistent agents scanning policy changes and controls; trigger HITL exceptions and route to remediation workflows
    • Tools/products/workflows: Scheduled graph runs; connectors to control evidence; exception queues and interrupts
    • Assumptions/dependencies: Access to control telemetry; change-detection reliability; playbooks for remediation
  • Personal AI process manager with household “approvals”
    • Sectors: daily life/consumer finance/admin
    • What it is: Multi-step assistants that pause for user confirmation (e.g., bill disputes, claims, bookings), resume with state intact
    • Tools/products/workflows: Mobile UIs for interrupts; calendar/banking integrations; durable local/cloud storage
    • Assumptions/dependencies: Secure OAuth/data permissions; consumer trust and safety; low-friction review UX
  • Safety-critical approval chains with formal checks
    • Sectors: healthcare orders, aviation MRO, energy operations
    • What it is: Graph nodes that combine LLM drafting with deterministic rule/constraint checks and mandatory human gates
    • Tools/products/workflows: Rule engines/formal methods inside validation nodes; redundant reviewer interrupts
    • Assumptions/dependencies: Verified rulesets; rigorous incident logging; certification pathways
  • Federated, privacy-preserving workflow state across org boundaries
    • Sectors: healthcare networks, supply chains, public sector consortia
    • What it is: Shared processes where only minimal, approved state crosses boundaries; interrupts at data-sharing gates
    • Tools/products/workflows: Data minimization and encryption; cross-tenant checkpointers; consent-aware routing
    • Assumptions/dependencies: Legal agreements; interoperable schemas; audit across multiple controllers/processors
  • Learned risk and evidence thresholds (instead of static heuristics)
    • Sectors: all regulated/high-stakes domains
    • What it is: Thresholds for “interrupt,” “retry,” or “fail closed” learned from outcomes and reviewer feedback over time
    • Tools/products/workflows: Continuous learning pipelines; feedback capture from interrupts; counterfactual evaluation
    • Assumptions/dependencies: High-quality labeled outcomes; bias and fairness monitoring; rollback on regressions

Notes on feasibility and dependencies

  • Model/tooling availability: Access to reliable LLMs, retrievers, and deterministic tools is assumed; simpler SDK/ReAct or schema-first stacks should be preferred for linear, validation-centric tasks.
  • Data governance: Many applications depend on curated corpora, data classification, and RBAC/ABAC to prevent overreach.
  • Human capacity and SLAs: HITL routes require reviewer availability; queue backlogs impact latency and user trust.
  • Persistence: Production HITL/background jobs require durable checkpointers (SQLite/Postgres/Temporal); in-memory is only for dev.
  • Policy definition: Clear thresholds and escalation rules are prerequisites; ambiguous policies degrade routing quality.
  • Cost/latency trade-offs: Retry budgets and verification add cost; must be tuned to business value and SLAs.
  • Security and privacy: Auditable traces must exclude sensitive data when exported; PII handling and logging policies are critical.
View Paper Prompt  View All Prompts 

Glossary

  • Agentic RAG: An agent-driven form of retrieval-augmented generation where retrieval quality controls the workflow route. "Agentic RAG requires conditional retrieval, evidence grading, and refusal or retry when support is weak"
  • Auditability: The capability to trace which branches and decisions were taken and why. "The team needs auditability for which branch ran and why"
  • AutoGen: A multi-agent conversation framework for coordinating LLM agents. "Multi-agent conversation frameworks such as AutoGen"
  • BEIR: A public benchmark suite for evaluating retrieval systems across many tasks. "Public retrieval benchmarks such as BEIR are useful references for the problem class"
  • BIRD: A text-to-SQL benchmark emphasizing more realistic database settings. "Semantic parsing benchmarks such as WikiSQL, Spider, and BIRD illustrate natural-language database querying under different schema and realism assumptions"
  • BPM (Business Process Management): The discipline of modeling and executing business processes, often long-running and stateful. "Long-running BPM"
  • Checkpointer: A persistence mechanism that saves workflow state to enable pause/resume across failures or reviews. "a checkpointer so an interrupted thread can resume later"
  • Citation verification: A step that checks whether generated answers are supported by cited sources. "evidence gating, citation verification, and retry semantics are not product requirements"
  • Conditional edges: Explicit routing links in a state graph that choose the next node based on current state. "Conditional edges make route decisions inspectable instead of hiding them in prompt text or nested application logic"
  • Control-plane primitives: Low-level orchestration constructs for managing workflow execution and state. "control-plane primitives—typed shared state, graph nodes and edges, conditional routing, durable checkpoints, and interrupt/resume"
  • CrewAI: A role-based agent orchestration framework for coordinating multiple LLM agents. "role-based crew orchestrators such as CrewAI"
  • Decision record: A structured, persistent account of the final decision and its rationale. "apply reviewer feedback, and leave a decision record"
  • DSPy: A framework for metric-guided optimization of prompts or programs rather than workflow control. "DSPy is often a better fit when the primary goal is metric-guided prompt or program optimization"
  • Durable execution: Execution that can survive failures, pauses, and restarts while preserving state. "make durable execution and governance first-class"
  • Durable workflow engines: Infrastructure that provides long-running execution, retries, and human tasks around LLM steps. "Durable workflow engines such as Temporal provide long-running execution, retries, and human tasks that can wrap LLM steps at the infrastructure layer"
  • Evidence gating: A policy that requires sufficient retrieved evidence before generating or finalizing an answer. "agentic retrieval-augmented generation with evidence gating"
  • Fail-closed terminations: Ending a workflow without producing an answer when support is insufficient. "Fail-closed terminations are preferable to unsupported answers when the workflow cannot justify a response"
  • Generative-agent architectures: Agent designs that incorporate memory, reflection, and long-horizon state. "generative-agent architectures likewise motivate memory, reflection, and long-horizon state as first-class concerns"
  • Graph orchestration: Managing LLM applications as explicit process graphs with state, routes, and checkpoints. "graph orchestration gives developers explicit places to store state, route failures, pause for review, resume after interruption, and explain what happened"
  • HITL (human-in-the-loop): A workflow pattern that pauses for human review and resumes afterward with preserved state. "HITL workflows are the clearest case for LangGraph"
  • InMemorySaver: An in-memory LangGraph checkpointer for local or test runs. "InMemorySaver() is appropriate for local tests"
  • Interrupt: A deliberate pause in the workflow to await external input (e.g., human review) before resuming. "The interrupt is not just a callback; it is a durable workflow boundary"
  • JSON Schema: A schema language for validating the structure of JSON outputs. "JSON Schema benchmarks measure schema compliance"
  • LangGraph: A low-level framework for building stateful, long-running LLM agent workflows as graphs. "LangGraph is a low-level orchestration framework for building, managing, and deploying long-running stateful agents and workflows"
  • LegalBench: A benchmark suite for legal reasoning tasks. "LegalBench provides a public multi-task benchmark for legal reasoning, including privacy-policy question answering"
  • Long-lived memory stores: Persistent stores for agent memory over extended sessions. "Separate long-lived memory stores are supported by the framework but are outside the three recipes exercised here"
  • MIPRO: A method for optimizing instructions and demonstrations across multi-stage LLM programs. "MIPRO optimizes instructions and demonstrations for multi-stage programs"
  • Node boundaries: Deliberate divisions between workflow steps that align with observable product events. "The most important implementation constraint is to keep node boundaries stable"
  • PydanticAI-style agents: Schema-first agents that emphasize typed validation of outputs. "PydanticAI-style agents emphasize validation"
  • ReAct-style loop: A lightweight reason-and-act prompting pattern interleaving thoughts and tool use. "A plain SDK or ReAct-style loop is often sufficient for basic tool use"
  • Reflexion: An approach where models reflect on prior outputs to iteratively improve performance. "Reflexion and generative-agent architectures likewise motivate memory, reflection, and long-horizon state as first-class concerns"
  • Retry budgets: Explicit limits on the number of times a failing step may be retried. "Retry budgets, error state, and repair nodes give failures a controlled path back into the workflow"
  • Route history: The recorded sequence of branches taken during workflow execution. "route history supports debugging and audit rather than leaving control flow only in model prose"
  • Schema-first systems: Approaches that prioritize structured output and validation using explicit schemas. "Schema-first systems are often a better fit for structured extraction and validation-heavy tasks"
  • StateGraph: The LangGraph abstraction that models a workflow as a typed, stateful graph. "LangGraph models long-running stateful workflows as StateGraph objects with nodes, edges, conditional routing, subgraphs, and optional checkpointers for durable execution"
  • Subgraphs: Nested graphs used to structure complex workflows or multi-agent patterns. "Supervisor--worker subgraphs"
  • Temporal: A production workflow engine for durable, long-running, human-in-the-loop processes. "Durable workflow engines such as Temporal provide long-running execution, retries, and human tasks that can wrap LLM steps at the infrastructure layer"
  • Toolformer-style tool use: A training/inference pattern where models learn to call tools during generation. "Toolformer-style tool use shows that lightweight tool-using loops can succeed without durable graph orchestration"
  • Typed state: A structured, schema-validated state object passed across workflow nodes. "Typed state is the durable record passed through the workflow"
  • Workflow orchestration: The coordination of multi-step LLM applications, including state, branching, and recovery. "When the main problem is prompt/program optimization rather than workflow orchestration"
View Paper Prompt  View All Prompts 

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 14 tweets with 856 likes about this paper.

Sign Up for Free