2000 character limit reached
Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence
Published 21 Aug 2026 in cs.IR, cs.AI, and cs.ET | (2608.21156v1)
Abstract: LLMs have evolved from language generators to autonomous agents capable of complex, long-horizon tasks. This evolution has produced paradigms including Prompt Engineering to elicit model capabilities, Context Engineering to manage information access, Harness Engineering to organize external tools and resources, and Loop Engineering to support continual reflection and self-improvement. Yet as tasks grow more complex, individual intelligence faces a fundamental limit: many tasks require heterogeneous expertise, interdependent subtasks, parallel execution, independent verification, and persistent state, exceeding any single agent's organizational capacity. Augmenting one agent's capabilities or context cannot resolve this architectural mismatch; intelligence must instead be distributed across specialized agents and organized at the system level. We call this System Intelligence: an agent system's ability to organize and coordinate multiple intelligent components into a coherent, adaptive whole pursuing a shared objective. Achieving it requires more than adding agents; it demands explicit structures to organize work, coordinate heterogeneous agents, and maintain evolving execution states. We introduce Graph Engineering, an emerging paradigm for next-generation agent systems. Unlike prior paradigms that mainly optimize individual interactions or agent-level behavior, Graph Engineering constructs explicit, dynamic, evolving graph structures representing tasks, agents, and system states. These abstractions provide a unified foundation for organizing complex objectives, orchestrating heterogeneous agents, modeling system dynamics, and enabling scalable agent evolution. We systematically review the principles, methodologies, and applications of Graph Engineering for LLM agents. Related papers, open-source data, and projects are collected at https://github.com/DEEP-JLU/Awesome-Graph-Engineering.
First 10 authors:
Whiteboard
Paper Prompts
Sign up for free to create and run prompts on this paper.
Top Community Prompts
Explain it Like I'm 14
1. What is the paper about?
This paper explains how artificial intelligence is changing from one smart LLM into a team of cooperating AI agents.
A LLM, such as ChatGPT, can answer questions and solve problems. However, difficult real-world tasksโlike writing large software projects, doing scientific research, or managing a business processโoften require many different jobs to happen at once.
The paper introduces Graph Engineering as a way to organize these AI agents. A graph is like a map made of points and connecting lines. In this paper:
- The points can represent tasks, agents, tools, or pieces of information.
- The connections show how these parts depend on or communicate with one another.
The authors argue that well-designed graphs can help many AI agents work together as one intelligent system.
2. What questions does the paper explore?
The paper mainly asks:
- How has AI developed from individual LLMs into autonomous agents?
- Why are individual AI agents not enough for very complicated tasks?
- How can multiple agents divide work and cooperate effectively?
- How can graphs organize tasks, communication, and information?
- What problems still need to be solved before AI systems can manage themselves reliably?
The paper describes three levels of intelligence:
| Level | Simple meaning |
|---|---|
| Model Intelligence | What one LLM knows and can reason about |
| Individual Intelligence | What one agent can do when connected to tools, memory, and an environment |
| System Intelligence | What a whole team of agents can accomplish by working together |
For example, Model Intelligence is like a studentโs knowledge. Individual Intelligence is like that student using books, a calculator, and the internet to complete a project. System Intelligence is like a whole project team dividing the work, sharing results, checking one another, and finishing together.
3. How did the researchers approach the topic?
This is mainly a survey and framework paper, rather than a paper reporting one new experiment.
A survey collects and organizes ideas from many earlier studies. The authors examine existing research, tools, benchmarks, and systems related to LLMs and AI agents. They then create a framework for understanding how these technologies fit together.
The paper follows the development of AI in stages.
Foundation models
First, LLMs are trained on huge amounts of text. This is called pre-training. It is similar to giving a student a giant library and allowing them to study from it.
After that, models often receive post-training, which teaches them to follow instructions, behave more safely, and solve certain kinds of problems more effectively.
Prompt and context engineering
The paper then discusses two ways to help a model use its abilities:
- Prompt Engineering means writing instructions carefully. It is like explaining the rules of a homework assignment clearly.
- Context Engineering means choosing and organizing the information the model needs. It is like giving the student the correct textbook pages, notes, and previous work.
Methods such as retrieval-augmented generation give the model useful information from outside sources instead of making it rely only on memory.
Harness and loop engineering
Next, the model can be connected to extra abilities through a harness. A harness is the software around the model that gives it access to:
- Tools, such as search engines, calculators, browsers, or coding programs
- Memory for remembering earlier information
- Reusable skills
- External environments, such as a computer or website
The model then works through an agent loop. This means it repeatedly:
- Observes the situation
- Thinks about what to do
- Takes an action
- Checks the result
- Updates its plan
This is similar to a person trying to repair a bicycle: inspect the problem, make a change, test it, and try again if needed.
Graph Engineering
Finally, the authors propose using graphs to organize entire teams of agents. They divide Graph Engineering into three major parts:
- Task Organization Breaking a large goal into smaller jobs and showing which jobs must happen first.
- Agent Coordination Assigning each job to the most suitable agent and helping agents communicate, share results, and resolve disagreements.
- Runtime State Management Keeping track of what has been completed, what is still running, what information has changed, and what went wrong.
The word runtime means what happens while the system is actively working.
4. What are the main findings?
Because this is a survey and framework paper, its main findings are ideas and conclusions rather than one set of experimental numbers.
Individual agents have important limits
A single agent may have strong reasoning abilities, but it can struggle when a task:
- Contains many separate subtasks
- Requires different kinds of expertise
- Needs several activities to happen at the same time
- Requires repeated checking by independent agents
- Takes a long time to complete
- Produces a large amount of information
Putting everything into one agentโs memory can make it confusing. Important information may be lost or mixed together, much like trying to manage an entire group project alone using one crowded notebook.
More agents do not automatically create a better system
The paper emphasizes that simply adding more agents is not enough. A team of agents can still fail if:
- Nobody knows who is responsible for each task
- Agents do not share information properly
- Two agents do conflicting work
- Results are not checked
- The system loses track of progress
- A failure causes all previous work to be lost
Therefore, a successful multi-agent system needs organization, communication rules, and reliable records.
Graphs can provide the needed structure
Graphs can show:
- Which tasks depend on other tasks
- Which tasks can happen in parallel
- Which agent is responsible for each task
- How information moves between agents
- Where verification is needed
- How the system should recover after a failure
For example, a research project might be organized like this:
1 2 3 4 5 6 7 8 9 |
Choose research question
|
Collect information
/ \
Analyze data Find earlier studies
\ /
Check results
|
Write final report |
This structure makes the work easier to understand and manage than a single long chain of instructions.
Several research areas are developing, but gaps remain
The paper surveys many existing systems for task planning, agent cooperation, memory, workflow management, and state tracking. It finds that these ideas are already being used in areas such as:
- Software engineering
- Scientific discovery
- Healthcare
- Business workflows
- Digital assistants
- Simulations of societies and economies
However, the authors identify major unfinished problems. Current systems often have difficulty with:
- Changing their own organization when conditions change
- Preserving a reliable history of who did what
- Recovering from partial failures
- Managing simultaneous actions safely
- Working across different software tools
- Measuring whether the whole systemโnot just one agentโworked correctly
5. Why are these findings important?
The paper suggests that the future of AI may depend less on making one model endlessly larger and more on designing good systems around models.
A very capable agent is useful, but a coordinated team could be much better at complex work. For instance:
- One agent could search for information.
- Another could analyze data.
- A third could write computer code.
- A fourth could check the results.
- A manager agent could organize the whole process.
Graph Engineering could help these agents work together in a clear and dependable way.
6. Simple conclusion and possible impact
The paperโs central message is:
The next major step in AI is not just creating smarter individual agents. It is learning how to organize many agents into reliable, cooperative systems.
If the ideas in the paper develop successfully, future AI systems could handle longer and more complicated tasks. They might support scientists, programmers, doctors, companies, and ordinary users.
However, these systems must be carefully designed. They need clear responsibilities, safe communication, accurate memory, progress tracking, and ways to correct mistakes. Without these features, a group of AI agents could become disorganized or produce unreliable results.
Overall, the paper presents Graph Engineering as a possible foundation for building AI systems that act less like one person answering a question and more like a well-organized team working toward a shared goal.
Knowledge Gaps
Knowledge Gaps, Limitations, and Open Questions
- The concept of โSystem Intelligenceโ lacks an operational definition. The paper describes it through task decomposition, coordination, and state management, but does not specify measurable criteria or thresholds for determining when a system possesses system-level intelligence.
- The proposed Graph Engineering paradigm is not formally grounded. The paper does not provide a unified mathematical formulation linking task graphs, agent graphs, runtime-state graphs, environments, and execution policies.
- The causal contribution of graph structures remains unverified. It is unclear whether graphs themselves improve performance, or whether observed benefits arise from better prompting, additional agents, explicit workflows, increased computation, or stronger orchestration infrastructure.
- There is no controlled empirical comparison between graph-based and non-graph-based systems. Future studies should compare equivalent systems using sequential workflows, flat multi-agent communication, hierarchical orchestration, and graph-based execution under matched model, tool, token, and latency budgets.
- The taxonomy is not validated against independent expert judgments or empirical data. The boundaries between Prompt Engineering, Context Engineering, Harness Engineering, Loop Engineering, and Graph Engineering may overlap, but the paper does not report a systematic classification protocol or inter-rater agreement.
- The distinction between Individual Intelligence and System Intelligence remains partly conceptual. The paper does not establish objective task-complexity, coordination, or state-persistence criteria for deciding when a problem exceeds the capabilities of a single agent.
- The paper does not identify which graph representations are appropriate for which task types. It remains unresolved when to use directed acyclic graphs, cyclic graphs, temporal graphs, hypergraphs, state-transition graphs, workflow graphs, or heterogeneous knowledge graphs.
- The semantics of graph nodes and edges are underspecified. The paper does not define standardized schemas for representing tasks, agents, capabilities, dependencies, messages, resources, verification constraints, failures, or state transitions.
- Graph construction is insufficiently addressed. The mechanisms by which a natural-language objective is converted into an executable graphโincluding decomposition accuracy, missing dependencies, ambiguous subtasks, and incorrect orderingโare not systematically analyzed.
- Graph correctness is not clearly distinguished from task correctness. A system may execute a structurally valid graph while producing an incorrect result, or produce a correct result through a structurally flawed graph; the paper does not define how these forms of correctness should be evaluated separately.
- Dynamic graph adaptation lacks formal guarantees. The paper identifies self-evolving graphs as a future direction but does not specify how systems should add, remove, reorder, or revise tasks and agents without destabilizing ongoing execution.
- The trade-offs of graph dynamism are unexplored. Adaptive restructuring may improve flexibility but could also increase planning overhead, introduce inconsistency, invalidate prior results, or make behavior difficult to audit.
- Concurrency and synchronization are not developed in sufficient technical detail. The paper identifies parallel execution and shared state as challenges but does not specify consistency models, synchronization protocols, locking strategies, transaction semantics, or methods for preventing race conditions.
- Failure recovery is not evaluated systematically. It remains unclear how graph-based systems should recover from agent failures, tool outages, corrupted state, invalid intermediate results, partial execution, or changes in the external environment.
- The paper does not provide guarantees for provenance and reproducibility. Although provenance is identified as important, the required granularity and format for recording prompts, model versions, tool calls, intermediate states, graph revisions, and agent decisions are not established.
- Conflict resolution among agents is underspecified. The paper does not explain how systems should reconcile contradictory outputs, competing plans, inconsistent shared memories, or disagreements between specialized agents and verifiers.
- Responsibility and accountability are unresolved. There is no clear method for attributing an erroneous or harmful system outcome to a particular agent, graph decision, communication event, tool, or state update.
- The role of human oversight is not systematically integrated. The paper mentions human support as a shared resource but does not define when intervention should occur, how authority is allocated, or how human decisions are represented in the execution graph.
- Resource allocation is not formalized. The paper does not provide methods for allocating agents, models, tools, memory, computation, or context length under cost, latency, availability, and reliability constraints.
- The relationship between agent specialization and coordination overhead remains unknown. Adding heterogeneous agents may improve expertise but also increase communication cost, duplicated work, routing errors, and integration failures; this trade-off is not quantified.
- Scalability has not been demonstrated. The paper does not establish how graph size, agent count, communication volume, state size, or execution duration affect accuracy, latency, monetary cost, and reliability.
- Long-horizon degradation is not empirically characterized. It remains uncertain how graph-based systems perform as errors accumulate across many dependent steps, especially when early mistakes contaminate later subtasks or shared state.
- Verification mechanisms are insufficiently specified. The paper refers to independent verification but does not clarify which subtasks require verification, how verifiers should be selected, or how verification quality and independence should be measured.
- Evaluation metrics for System Intelligence are incomplete. Accuracy alone is insufficient, but the paper does not define validated metrics for structural fidelity, coordination efficiency, state consistency, recovery quality, adaptability, resource use, or governance.
- Benchmark coverage may not reflect real-world system complexity. The discussion of benchmarks is described in the paperโs roadmap, but the provided text does not establish whether existing benchmarks contain long-running tasks, changing environments, partial observability, concurrent execution, realistic failures, and heterogeneous tools.
- Cross-domain generalization remains untested. The proposed framework is discussed across software engineering, science, healthcare, enterprise, and simulation, but it is unclear whether the same graph abstractions and coordination mechanisms transfer across these domains.
- Safety and security risks of graph-based orchestration are underdeveloped. Potential threats include malicious agents, prompt injection through shared state, tool misuse, privilege escalation, poisoned memories, compromised graph components, and unsafe self-modification.
- Self-evolving systems lack safety constraints. The paper does not specify how to prevent an agent system from modifying its own task graph, agent roles, tools, or memory policies in ways that violate objectives or governance requirements.
- Robustness to adversarial or unreliable agents is unexplored. The framework does not address Byzantine behavior, deceptive outputs, collusion, strategic manipulation, or agents that confidently produce invalid intermediate results.
- Interoperability standards are not concretely defined. Although the paper identifies interoperability as a gap, it does not propose common interfaces for graph exchange, state serialization, tool capabilities, agent contracts, event logs, or execution traces.
- The proposed ontology-engineering direction remains vague. The paper does not specify how task, capability, role, resource, state, and outcome ontologies should be constructed, aligned across domains, versioned, or automatically updated.
- The boundary between graph engineering and existing workflow, distributed-systems, and operating-system techniques is unclear. The paper does not sufficiently identify which contributions are novel to LLM-agent systems versus adaptations of established planning, scheduling, transaction, provenance, and fault-tolerance methods.
- The effects of model heterogeneity are not isolated. It remains unclear how differences in model architecture, scale, training, tool-use ability, reasoning reliability, and alignment affect graph execution and coordination.
- Agent communication is treated primarily structurally rather than informationally. The paper does not investigate how message compression, selective disclosure, communication protocols, or semantic misinterpretation affect system performance and state consistency.
- The economics of Graph Engineering are unresolved. No systematic analysis is provided of the cost-benefit relationship between additional agents, graph planning, verification, retries, persistent memory, and improved task outcomes.
- Persistent system evolution is asserted to be limited but not measured. The paper does not define what counts as system evolution, how frequently systems should evolve, or whether evolution improves performance across repeated tasks rather than merely changing system structure.
- There is no evidence that evolved graphs transfer across tasks or environments. Future research should test whether learned decompositions, agent assignments, skills, and coordination policies remain useful under distribution shifts and new objectives.
- The surveyโs evidence base may be difficult to reproduce. The excerpt presents a broad and rapidly changing set of methods, including many recent works, but does not state explicit inclusion criteria, search procedures, quality screening rules, or a reproducible literature-review protocol.
Practical Applications
Immediate Applications
- Graph-based orchestration for software engineering (software development; deployable now)
Teams can represent software tasks as dependency graphs: requirements analysis, architecture, implementation, testing, security review, documentation, and deployment. Specialized agents can be assigned to each node, while graph state records prerequisites, artifacts, approvals, failures, and retries.
Potential products/workflows: multi-agent issue-resolution systems, pull-request pipelines, code-review agents, CI/CD orchestration layers, and platforms built on workflow engines such as
LangGraph,AutoGen, or comparable stateful runtimes. Dependencies: reliable tool permissions, sandboxed code execution, human approval for production changes, provenance tracking, and integration with repositories and CI systems. LLM outputs still require deterministic testing and security validation. - Parallel research and evidence-synthesis workflows (academia and scientific discovery; deployable now for low-risk tasks) A research objective can be decomposed into literature retrieval, dataset discovery, method comparison, experiment design, statistical checking, citation verification, and report drafting. Agents can work concurrently and pass structured outputs through a directed acyclic graph or other task graph. Potential tools: literature-review assistants, systematic-review pipelines, experiment-planning systems, citation auditors, and graph-based research notebooks. Dependencies: access to scholarly databases, high-quality retrieval, reproducible computational environments, clear separation between generated hypotheses and verified findings, and researcher review before publication.
- Enterprise workflow automation (business operations, finance, legal services, and customer support; deployable now) Organizations can model processes such as procurement, invoice handling, contract review, employee onboarding, claims processing, and customer escalation as graphs with explicit handoffs and verification nodes. Different agents can extract information, check policy compliance, communicate with users, and update enterprise systems. Potential products: agentic workflow-management platforms, policy-aware document-processing systems, and orchestration middleware connecting CRM, ERP, ticketing, email, and knowledge-base systems. Dependencies: stable APIs, access-control policies, audit logs, data-quality checks, and exception handling. High-impact decisions should remain subject to human authorization and applicable regulations.
- Persistent digital assistants (daily life and general-purpose digital agents; deployable now with constrained permissions) A personal assistant can use a graph to organize calendar management, travel planning, reminders, shopping, household administration, and information retrieval. Runtime state can preserve unfinished tasks and resume them after interruptions instead of restarting from a single prompt. Potential products: assistants that maintain task boards, coordinate multiple services, remember user preferences, and request confirmation before external actions. Dependencies: explicit user consent, secure authentication, privacy-preserving memory, reliable service integrations, and safeguards against unauthorized purchases, messages, or account changes.
- Knowledge-intensive customer and employee support (enterprise IT, education, and public services; deployable now) A support request can be routed through retrieval, diagnosis, policy checking, response generation, and escalation nodes. Multiple agents can independently verify answers against organizational documentation before a response is sent. Potential workflows: IT-help-desk triage, benefits-information assistants, internal knowledge search, and educational administrative support. Dependencies: current and authoritative knowledge bases, source citations, monitoring for hallucinations, language accessibility, and escalation to human staff for ambiguous or sensitive cases.
- Healthcare administrative coordination (healthcare operations; deployable now for non-diagnostic tasks) Graph Engineering can coordinate appointment scheduling, referral tracking, prior-authorization documentation, discharge planning, medical-record summarization, and communication among departments. Runtime state can identify missing documents, pending approvals, and failed handoffs. Potential products: referral-management systems, clinical-documentation assistants, and care-coordination dashboards. Dependencies: HIPAA/GDPR-equivalent compliance, role-based access, auditable provenance, interoperability with electronic health records, and strict separation from autonomous diagnosis or treatment decisions.
- Multi-stage data and analytics pipelines (industry and academia; deployable now) Agents can perform schema inspection, data cleaning, feature construction, statistical analysis, visualization, and report generation as graph nodes. Independent validation agents can check data leakage, inconsistent assumptions, and reproducibility. Potential tools: natural-language data-analysis platforms, agent-managed notebooks, and provenance-aware analytics pipelines. Dependencies: deterministic execution where possible, validation against known datasets, protection of confidential data, and clear recording of code, parameters, and intermediate artifacts.
- Runtime monitoring, recovery, and audit for agent systems (software infrastructure and governance; deployable now) The paperโs emphasis on system state supports operational tooling that records task progress, agent availability, tool calls, intermediate results, conflicts, and failure records. Systems can retry failed nodes, resume partial work, roll back unsafe changes, or escalate to operators. Potential products: agent observability platforms, execution ledgers, graph visualizers, replay systems, and โcheckpoint-and-restartโ services. Dependencies: standardized event schemas, immutable logs, state consistency under concurrent updates, and safeguards against sensitive information being exposed in traces.
- Education and academic training (education; deployable now in supervised settings) A learning workflow can assign separate agents to explain concepts, generate exercises, assess answers, provide feedback, and monitor progress. Persistent state allows adaptation to a learnerโs prior errors and demonstrated mastery. Potential products: tutoring systems with explicit learning graphs, teacher dashboards, automated formative assessment, and research assistants for course development. Dependencies: pedagogical validation, age-appropriate safeguards, teacher oversight, privacy protection, and evaluation of learning outcomes rather than answer fluency alone.
- Policy and public-sector process mapping (government and public administration; deployable now for decision support) Agencies can represent eligibility checks, document requirements, appeals, approvals, and case escalation as auditable graphs. Agents can assist with document classification and procedural guidance without autonomously making legally binding decisions. Potential applications: permit processing, benefits administration, regulatory inquiry routing, and policy-impact analysis. Dependencies: transparency, nondiscrimination testing, statutory compliance, records retention, public accountability, and human review of adverse or discretionary outcomes.
Long-Term Applications
- Graph-native agent operating systems (software infrastructure; requires substantial development) The paper points toward an operating system in which tasks, agents, tools, memories, permissions, and runtime state are first-class graph entities. Such a system could schedule agents according to capabilities and cost, isolate execution contexts, synchronize concurrent work, and recover from partial failures across long-running tasks. Potential products: agent operating systems, graph-native schedulers, interoperable agent registries, and execution environments with built-in memory and provenance management. Dependencies: common ontologies, interoperable protocols, formal state semantics, secure capability delegation, fault-tolerant storage, and benchmarks for system-level reliability.
- Self-evolving organizational workflows (enterprise and software engineering; long-term) Systems could analyze execution histories and automatically revise task decompositions, agent roles, communication structures, and reusable skills. For example, a software organizationโs development graph might learn that security review should occur earlier or that a particular agent is better suited to a class of defects. Potential products: continuously optimizing workflow managers and organizational โprocess copilots.โ Dependencies: safe change-control mechanisms, causal evaluation of workflow modifications, protection against feedback loops, stable objectives, and human governance over structural changes.
- Autonomous scientific discovery teams (scientific research, pharmaceuticals, materials, and energy; long-term) Specialized agents could formulate hypotheses, search literature, design experiments, operate laboratory or simulation tools, analyze results, challenge interpretations, and update a persistent research graph. Independent verification agents could reproduce analyses and flag unsupported claims. Potential products: autonomous laboratory planners, computational drug-discovery teams, materials-design systems, and closed-loop simulation platforms. Dependencies: validated laboratory robotics, reliable experiment APIs, physical-world safety, reproducibility, expensive data and equipment, intellectual-property controls, and expert approval for experimental execution.
- Clinical decision-support and care orchestration (healthcare; long-term and high risk) A system could coordinate diagnostic evidence retrieval, guideline matching, specialist consultation, longitudinal patient history, treatment-option comparison, and follow-up monitoring through a structured clinical graph. Potential products: multidisciplinary clinical coordination systems and patient-specific care-pathway managers. Dependencies: prospective clinical trials, calibrated uncertainty estimates, complete and interoperable patient data, liability frameworks, regulatory approval, bias monitoring, and mandatory clinician control. The paperโs framework alone does not establish medical efficacy.
- Robotic and embodied multi-agent systems (robotics, manufacturing, logistics, and disaster response; long-term) Graphs could organize heterogeneous robots and software agents around shared objectives such as warehouse fulfillment, construction, infrastructure inspection, or search and rescue. Runtime state would track robot locations, resource constraints, environmental changes, task ownership, and recovery plans. Potential products: fleet-management systems, robot team coordinators, adaptive manufacturing cells, and disaster-response command platforms. Dependencies: reliable perception and localization, low-latency communication, formal safety guarantees, real-time planning, adversarial or uncertain environments, and hardware interoperability.
- Energy-grid and infrastructure management (energy and utilities; long-term) Agents could coordinate demand forecasting, renewable-energy scheduling, fault detection, maintenance planning, storage dispatch, and emergency response. A shared graph could represent dependencies among assets, operators, weather conditions, and grid constraints. Potential products: agentic grid-operations platforms and predictive-maintenance systems. Dependencies: real-time data quality, cybersecurity, deterministic control layers, regulatory compliance, fail-safe fallback systems, and rigorous validation before agents can influence physical infrastructure.
- Financial operations and market-risk management (finance; long-term for autonomous action) Multi-agent workflows could combine data ingestion, anomaly detection, portfolio analysis, regulatory checks, scenario generation, and independent risk review. Runtime provenance would help explain which data and agents contributed to a recommendation. Potential products: compliance orchestration systems, audit-ready research platforms, and risk-monitoring agents. Dependencies: trustworthy data, explainability, model-risk governance, anti-manipulation safeguards, strict permission boundaries, and human approval for trading or credit decisions. Autonomous financial execution is considerably less mature than analytical assistance.
- Digital twins and social/economic simulation (policy, urban planning, and economics; long-term) Graph-organized agents could represent households, firms, institutions, regulators, and infrastructure, allowing researchers to simulate policy interventions and interactions over time. Persistent system state could support evolving scenarios rather than isolated question answering. Potential products: policy sandbox platforms, urban-planning simulators, supply-chain stress-testing tools, and institutional strategy environments. Dependencies: empirical calibration, validation against real-world behavior, sensitivity analysis, protection against misleading forecasts, and careful distinction between simulation results and policy evidence.
- Cross-run organizational memory and institutional continuity (public administration, enterprises, and research organizations; long-term) A graph-based memory layer could preserve not only documents but also decisions, responsibilities, evidence, failed approaches, approvals, and causal links across projects and agent runs. This could reduce repeated work and support handover when staff or agents change. Potential products: institutional memory systems, decision ledgers, and provenance-aware project archives. Dependencies: reliable memory deletion and correction, access controls, data-retention policies, resistance to propagating obsolete or incorrect information, and mechanisms for contesting stored conclusions.
- Formal governance and certification of agent societies (policy, safety, and standards; long-term) The taxonomy of task organization, coordination, state management, and system evolution could support standards for evaluating whether an agent system has clear responsibilities, auditable communication, recoverable state, and safe adaptation. Potential outputs: certification protocols, compliance checklists, benchmark suites, and regulatory reporting formats for high-impact agent systems. Dependencies: agreed definitions and ontologies, representative benchmarks, measurable notions of structural fidelity and operational correctness, independent auditing, and international coordination.
- Everyday autonomous service ecosystems (daily life; long-term) Multiple specialized personal agents could coordinate health routines, finances, education, travel, home energy, and social communication through a user-controlled personal graph. The system could negotiate among competing goals and maintain continuity over years. Dependencies: interoperable service providers, strong identity and consent infrastructure, privacy-preserving learning, robust conflict resolution, affordable computation, and user-understandable controls. The central unresolved issue is ensuring that system-level autonomy remains aligned with changing human preferences rather than merely optimizing historical behavior.
Glossary
- Agent Harness: A runtime layer that connects an agent to tools, memory, skills, environments, and governance mechanisms. โIt consists primarily of a Foundation Model and an Agent Harness.โ
- Agent Loop: An iterative process in which an agent perceives, reasons, acts, processes feedback, and updates its state. โThese components are organized over time by an iterative Agent Loop, which repeatedly performs perception, reasoning, action, feedback processing, and state update.โ
- Agentic RL: Reinforcement learning in which models learn through interaction with tools, environments, or extended action trajectories. โThis paradigm has also expanded toward agentic RLโ
- Agent System: A collection of agents and supporting components that operate through shared resources, environments, and coordination mechanisms. โAn Agent System extends the Individual Agent abstraction to a collection of agents that operate through shared resources, external environments, and coordination mechanisms.โ
- Autonomous Agent: A computational entity capable of pursuing goals, taking actions, and adapting without continuous human control. โleading to the emergence of LLM-based agents.โ
- Chain-of-Thought: A prompting technique that elicits intermediate reasoning steps before producing an answer. โChain-of-Thought introduces intermediate reasoningโ
- Context Engineering: The acquisition, organization, processing, and maintenance of information supplied to a model during inference. โContext Engineering concerns what task-relevant information is provided and how it is organized and maintained.โ
- Context Management: The maintenance and updating of useful working information as an agent executes a task. โContext management maintains useful working information as execution progresses.โ
- Context Window: The bounded amount of text or information that a model can process in one inference operation. โtask-relevant information competes for contextual capacityโ
- Constitutional AI: An alignment approach that uses explicit principles to guide model behavior and generate feedback. โConstitutional AI and RLAIF extended alignment toward AI-generated principles and feedbackโ
- Coordination Mechanism: A system component that governs task assignment, information exchange, result integration, conflict resolution, and failure handling among agents. โMechanisms that determine how agents assign tasks, exchange information, integrate results, resolve conflicts, and handle failures.โ
- Dense Retrieval: Information retrieval in which queries and documents are represented as dense numerical vectors for similarity comparison. โContext acquisition has progressed from dense retrievalโ
- Execution Trajectory: The sequence of states, actions, observations, and outcomes generated during task execution. โGEPA uses execution trajectories and natural-language reflection to evolve prompts from task feedback.โ
- Foundation Model: A large, broadly trained model that provides general-purpose capabilities transferable across tasks. โThe Foundation Model serves as the cognitive coreโ
- Graph Engineering: An engineering paradigm that uses graph structures to organize tasks, agent coordination, and runtime-state evolution. โGraph Engineering, a novel engineering paradigm in which graph structures are used to organize and control task execution, agent coordination, and runtime state evolutionโ
- Graph of Thoughts: A reasoning framework that represents and explores interconnected thoughts as a graph rather than a single chain. โTree of Thoughts, Graph of Thoughts, and Self-Refine extend reasoning toward search and iterative refinement.โ
- Harness Engineering: The design and management of the runtime capabilities surrounding a LLM. โHarness Engineering provides and manages these capabilitiesโ
- Inference-Time Augmentation: The use of prompts, retrieved information, tools, or other mechanisms at inference time to expand or condition model capabilities. โInference-time Augmentationโ
- Iterative Retrieval: A retrieval process that repeatedly updates searches based on intermediate reasoning or newly acquired information. โHyDE, IRCoT, Self-RAG, and CoRAG progressively integrate query transformation, iterative retrieval, and reflectionโ
- Long-Horizon Execution: Task execution involving many sequential or parallel operations over an extended period. โMAGE further treats memory as execution-state management for long-horizon tasksโ
- Memory Consolidation: The process of organizing and integrating stored information or experiences for later reuse. โlater systems such as A-MEM, Mem0, Zep, and MemoryOS improve memory organization, consolidation, and reuse.โ
- Mixture-of-Experts: A neural architecture that activates selected expert subnetworks for different inputs, enabling large capacity with limited computation per input. โsparse mixture-of-experts models such as Switch Transformer, Mixtral, and DeepSeekMoEโ
- Model Intelligence: The ability of an individual model to use its learned knowledge and reasoning capabilities within a given context. โWe refer to this capability as Model Intelligenceโ
- Ontology Engineering: The systematic construction and maintenance of formal concepts, relationships, and categories for representing a domain. โfuture research directions toward ontology engineeringโ
- Parameter-Level Capability Development: The acquisition and modification of model abilities through changes to learned parameters during training. โWe refer to this process as parameter-level capability developmentโ
- Post-Training: Training performed after initial pre-training to improve controllability, alignment, instruction following, or specialized capabilities. โPost-training further updates model parametersโ
- Preference Alignment: The process of adapting model behavior to preferences expressed by humans or other models. โPreference Alignment introduced explicit human or model feedbackโ
- Prompt Engineering: The design of instructions, examples, constraints, and formats to influence a modelโs behavior. โPrompt Engineering primarily concerns how tasks and expected behaviors are specifiedโ
- Retrieval-Augmented Generation: A method that retrieves external information and supplies it to a generative model when producing an answer. โContext acquisition has progressed from dense retrieval and retrieval-augmented generationโ
- Reinforcement Learning from AI Feedback (RLAIF): Reinforcement learning in which an AI system supplies preference evaluations or feedback instead of human annotators. โConstitutional AI and RLAIF extended alignment toward AI-generated principles and feedbackโ
- Reinforcement Learning from Human Feedback (RLHF): Reinforcement learning that uses human preference judgments to optimize a modelโs behavior. โInstructGPT established the influential RLHF pipelineโ
- Runtime State: Information describing the current operational condition and progress of an agent or system during execution. โ denotes the runtime state of agent at time .โ
- Self-Consistency: A reasoning method that samples multiple reasoning paths and aggregates their resulting answers. โSelf-Consistency aggregates multiple reasoning pathsโ
- Skill Composition: The construction of complex reusable capabilities by combining procedural skills or workflows. โSubsequent methods such as SAGE, HASP, SkillComposer, and Skill-Use improve skill construction, composition, evolution, and invocation.โ
- System Evolution: The adaptation or restructuring of agents, workflows, skills, and graph topology as execution requirements or feedback change. โpersistent System Evolution remains limited.โ
- System Intelligence: The ability of an intelligent system to organize objectives, distribute responsibilities, coordinate agents, and maintain system-level state. โWe refer to this capability as System Intelligenceโ
- Task Decomposition: The division of a complex objective into smaller executable units with defined relationships and dependencies. โTask Organization determines how a global objective is decomposed into executable unitsโ
- Tool Invocation: The process by which an agent calls an external function, service, or computational resource. โThe Agent Harness extends these intrinsic capabilities through interfaces for perception and context construction, memory and knowledge access, tool invocationโ
- Verifiable Reward: A reward signal whose correctness can be checked against an objective criterion, execution result, or external verifier. โMore recently, RL has increasingly shifted from preference alignment toward direct capability development through verifiable rewards and environmental feedback.โ
- Zero-Shot Adaptation: Applying a model to a task without providing task-specific examples in the prompt. โTask specification uses instructions, demonstrations, constraints, and output formats to support zero-shot and few-shot adaptationโ
