LLM Agent: Iterative Reasoning & Action

• LLM agents are systems that combine perception, reasoning, memory, planning, and tool use to perform iterative decision-making.
• They implement structured feedback loops to update internal states and coordinate external actions in dynamic environments.
• Architectural variations—from single-agent loops to hierarchical multi-agent networks—enable specialized roles and adaptive learning without explicit weight updates.

A LLM agent is a goal-directed system in which a LLM is embedded in an agent loop with state, memory, planning, environment interaction, and often tool use, rather than being used as a single-turn text generator. In the surveyed and task-specific literature, such agents are variously formalized as policies over token histories, prompt-programmed modules that transform or route inputs, multi-role systems that coordinate specialized subagents, or LLM-guided controllers that shape downstream actions and learning dynamics (Barua, 2024, Wang et al., 2 Aug 2025, Liu et al., 2024). Across these formulations, the defining property is not merely language generation, but iterative perception, reasoning, action, and adaptation.

1. Conceptual foundations

A recurring formalization treats an LLM agent as a policy acting in a language-mediated environment. In the two-player-game view, a state is the token sequence generated so far, an action is the next token, and the LLM policy is written as $\pi^i(\cdot \mid s)$ over the available actions at state $s$; with the other player’s policy fixed, the setting induces an MDP for the agent (Liu et al., 2024). A related systems view expresses the agent as a loop with internal state $s_t$, observation $o_t$, and action $a_t$, with the LLM implementing the policy through a prompt-conditioned mapping $a_t \sim \pi(\cdot \mid o_t, s_t)$ (Barua, 2024).

This foundation differentiates an LLM agent from a plain chatbot. A plain LLM is used in a single-step text-in/text-out mode; an agent persists across turns, manages context and memory, invokes tools or external systems, and updates behavior based on environment feedback (Wang et al., 2 Aug 2025). In several papers, the agent’s output is not only a final answer but a structured trajectory. In ReAct-style settings, the trajectory explicitly alternates between internal reasoning and external actions, which can be flattened as

$$\tau' = [REASON]\; r_1 \cdots r_k\; [ACT]\; a_1 \cdots a_m$$

so that agent behavior is modeled as a sequence of thought and action spans rather than a monolithic response (2505.13820).

The literature also broadens the notion of “agent” beyond task-solving assistants. A prompt-configured persona with stable role constraints and retrieval access can be treated as an agent, as in the poetry-explanation setting where the underlying model is wrapped with RAG, a persistent conversational role, and prompted personality traits (Jayakumar et al., 27 Oct 2025). Conversely, highly operational agents may expose almost no conversational surface, acting primarily as controllers over tool chains, simulations, or classifiers.

2. Core architectural components

The survey literature organizes LLM agents around four core components: agent role, execution structure, external knowledge, and reflection (Wang et al., 2 Aug 2025). Role specifies what the agent is supposed to do; execution structure specifies how it decomposes and routes work; external knowledge grounds the agent beyond pretraining; reflection provides self-evaluation and correction.

Role can be extremely lightweight or highly specialized. In LLAMOS, the “Defense Agent” is defined entirely by a structured prompt that describes the task context, defense goal, and defense guidance; the agent is a prompt-programmed policy $g_{stl}$ that maps an adversarial text $x'$ to a purified text $\hat{x}$ before the target classifier processes it (Lin et al., 2024). In contrast, other systems instantiate multiple explicit roles. The digital-twin parametrization framework uses an observation agent, reasoning agent, decision agent, and summarization agent, each with its own prompt template and I/O contract, to interface with a cyclic simulation (Xia et al., 2024). The 6G communication system similarly decomposes responsibilities into secure, condensate, inference, planning, evaluation, reflexion, and refinement agents, with a shared knowledge base and memory (Jiang et al., 2023).

Execution structure ranges from single-agent loops to planner–executor and coder–reviewer decompositions, and further to hierarchical or dynamically generated agent networks (Wang et al., 2 Aug 2025). The planner–executor pattern is explicit in several domains: an LLM planner selects actions or subgoals, and specialized modules or downstream tools carry them out. The coder–reviewer pattern appears wherever generated code or structured outputs must be validated and repaired. More elaborate server–client or orchestrator architectures distribute work across specialized agents under a central controller.

External knowledge is usually supplied through retrieval or tool interfaces rather than weight updates. The trading survey reports agents that ingest prices, news, reports, social media, and charts, often converting numerical signals into textual summaries for LLM consumption (Ding et al., 2024). The data-science survey describes external databases, retrieval pipelines, and API-backed tools as standard mechanisms for grounding agent decisions (Wang et al., 2 Aug 2025). In domain-specific settings, these knowledge channels can be highly structured: digital twins expose a read-value and set-value interface (Xia et al., 2024), scientific workflows expose tool schemas backed by Parsl and HPC schedulers (Ma et al., 17 Feb 2025), and 6G agents query a vectorized communication knowledge base with secure filtering and MMR-style retrieval (Jiang et al., 2023).

Reflection closes the loop. It may take the form of explicit self-critique, reviewer feedback, tool-error handling, metric-driven search, or memory updates summarizing past successes and failures (Wang et al., 2 Aug 2025). In LaMDAgent, for example, the agent writes textual memories of previous pipeline trials and uses them to select the next post-training action, so reflection is not ancillary commentary but part of the policy state (Yano et al., 28 May 2025).

3. Agent topologies and interaction patterns

Single-agent systems remain common when the environment or task interface is narrow. LLM-Grounder is a single GPT-4-based agent that decomposes a 3D referring expression, calls target and landmark finders, and then reasons over returned bounding boxes, volumes, and distances to make a grounding decision (Yang et al., 2023). In the urban mobility framework, a single “virtual urban resident” carries a learned activity pattern, retrieved motivations, and a planning prompt that emits a daily trajectory (Wang et al., 2024). These systems are agentic because they operate over intermediate states and tool outputs, even without multiple conversational roles.

Two-agent systems often expose adversarial or cooperative structure. LLAMOS includes a defense agent and an attack agent that alternately perturb and purify text, producing iterative confrontations in which neither side completely defeats the other and repeated loops can emerge (Lin et al., 2024). Minecraft builder–architect settings assign planning to the architect and grounded execution to the builder, with clarification questions mediating ambiguity in natural-language instructions (Madge et al., 2024). Such cases make explicit that multi-agent interaction can be central to competence, not merely a modularization convenience.

Multi-agent systems use specialization to decompose heterogeneous technical workloads. The digital-twin parametrization system maps simulation control onto observation, reasoning, decision, and summarization roles (Xia et al., 2024). The 6G framework uses collaborative planning and evaluation/reflexion loops to design semantic communication systems under constraints such as BLEU and parameter budgets (Jiang et al., 2023). In control engineering, the LLM-Agent-Controller combines a central controller agent, specialized auxiliary agents for controller design and analysis, and a supervisor for workflow coordination; the reported system solved 83% of general tasks and individual agents achieved an average success rate of 87% (Zahedifar et al., 26 May 2025).

Dynamic topologies go further by letting agent structure evolve with the task. The data-science survey describes hierarchical and iterative agent-generation schemes in which parent agents spawn child agents or update other agents through feedback, extending static role assignment into runtime architectural adaptation (Wang et al., 2 Aug 2025). This suggests that “LLM agent” increasingly denotes not a fixed wrapper around one model, but a configurable computational organization.

4. Learning and adaptation paradigms

A notable feature of the literature is the diversity of learning regimes. Some agents are entirely prompt-only. LLAMOS explicitly emphasizes that its defense agent is a plug-and-play pre-processing module that operates without retraining the target LLM or learning from adversarial examples, with optional in-context learning used only to refine the prompt against specific failure modes such as extraneous trailing characters (Lin et al., 2024). LLM-Grounder is likewise zero-shot with respect to 3D grounding labels, using pretrained GPT-4 and visual grounding tools without task-specific 3D-text training (Yang et al., 2023).

Other systems depend heavily on in-context adaptation. The urban mobility framework uses self-consistency to select an activity pattern that best explains a user’s trajectories and retrieves motivations from recent or similar days before generation (Wang et al., 2024). The incident-response agent updates its conjecture of attack tactics by comparing LLM-simulated alerts with actual observations, thereby refining planning in context rather than through online weight updates (Gao et al., 13 Feb 2026). In the personality study, personality itself is prompted rather than learned, and the main research question is whether evaluation methods can detect the intended persona in a real conversational task (Jayakumar et al., 27 Oct 2025).

A third class uses explicit optimization over agent behavior. ML-Agent formulates autonomous ML engineering as an MDP with objective

$s$0

then trains a 7B Qwen-2.5 agent with exploration-enriched fine-tuning, step-wise PPO, and an agentic-ML reward module over nine ML tasks (Liu et al., 29 May 2025). Dynamic strategy adaptation in Overcooked-AI combines PPO with a frozen LLM evaluator whose “good” versus “bad” judgments shape reward in real time, yielding up to a 26% improvement in return over PPO baselines in high-noise environments while keeping latency under 1.05 milliseconds (Mallampati et al., 1 Jul 2025).

Efficiency-oriented work targets the agent architecture itself. Structured Agent Distillation compresses large ReAct-style teachers into smaller students by segmenting trajectories into reasoning and action spans and supervising them separately, preserving reasoning fidelity and action consistency better than flat token-level distillation (2505.13820). This suggests that when an LLM is treated as an agent, the object of compression is not only linguistic output but structured decision behavior.

5. Representative domains and functional roles

The literature demonstrates that “LLM agent” is a domain-general design pattern rather than a single application category. In security, LLAMOS acts as a defense agent for adversarial textual purification (Lin et al., 2024), while the incident-response system integrates perception, reasoning, planning, and action in a 14B model to process logs, infer attack state, simulate consequences, and produce recovery actions up to 23% faster than frontier LLM baselines on reported incidents (Gao et al., 13 Feb 2026). In scientific computing, an LLM agent can be a controller over tool calls and HPC resources, with Parsl-enabled functions queued concurrently on local or Polaris/ALCF resources (Ma et al., 17 Feb 2025).

In engineering and operations, OR-LLM-Agent translates natural-language optimization problems into mathematical models and Gurobi code, then uses OR-CodeAgent for sandbox execution and repair; on a benchmark of 83 real-world OR problems it achieved a 100% pass rate and 85% solution accuracy (Zhang et al., 13 Mar 2025). LaMDAgent treats post-training operations themselves as actions in an exploration space over SFT, TIES-Merging, datasets, and hyperparameters, improving tool-use accuracy by 9.0 points while preserving instruction-following performance (Yano et al., 28 May 2025). The control-engineering system extends this pattern to controller design and analysis via multi-agent specialization (Zahedifar et al., 26 May 2025).

In simulation and digital twins, LLM agents can function as “virtual experimenters.” The digital-twin parametrization framework autonomously searches over ball-addition sequences and shake counts in a mixing process, using observation, reasoning, decision, and summarization agents to convert natural-language goals into executable control calls (Xia et al., 2024). In urban mobility, LLMs serve as urban residents whose memories are activity patterns and whose plans are daily trajectories conditioned on retrieved motivations and optional scenario prompts such as pandemic constraints (Wang et al., 2024).

In perception and interaction, LLM-Grounder uses the LLM as a 3D grounding agent that decomposes natural-language queries, orchestrates OpenScene or LERF, and reasons over spatial and commonsense relations (Yang et al., 2023). The Minecraft builder and architect instantiate collaborative agents grounded in voxel actions and structured block-placement outputs (Madge et al., 2024). In tutoring, the poetry agents show that an LLM-based agent may be defined by role, RAG-backed domain knowledge, and personality constraints rather than by tool use alone (Jayakumar et al., 27 Oct 2025).

Surveys further show mature agent patterns in finance and data science. Financial trading agents ingest market, textual, visual, and simulated data; maintain memory and reflection; and either act directly as traders or indirectly as alpha miners (Ding et al., 2024). Data-science agents span data preprocessing, statistical computation, feature engineering, model training, evaluation, and visualization, with agent roles, execution structures, external knowledge, and reflection mechanisms mapped directly onto those workflow stages (Wang et al., 2 Aug 2025).

6. Evaluation, limitations, and contested points

Evaluation remains heterogeneous because agent behavior is multi-step, environment-dependent, and often tool-mediated. General-purpose benchmarks such as AgentBench, WebArena, and ToolLLM evaluate decision quality, tool use, and long-horizon web or environment interaction (Barua, 2024). Domain papers then add task-specific metrics: ASR and robust accuracy for adversarial defense (Lin et al., 2024), exact-match or composite task scores for post-training pipelines (Yano et al., 28 May 2025), AceBench and MT-Bench for tool use and instruction following (Yano et al., 28 May 2025), IoU-based grounding accuracy for 3D perception (Yang et al., 2023), pass rate and solution accuracy for OR automation (Zhang et al., 13 Mar 2025), and return, completion rate, idle steps, and latency for real-time multi-agent control (Mallampati et al., 1 Jul 2025).

Several recurring limitations qualify the apparent breadth of the paradigm. Hallucination, weak long-horizon reasoning, multimodal integration difficulty, alignment problems, and evaluation gaps are explicit concerns in the general survey literature (Barua, 2024). Prompted personality evaluation exhibits systematic bias in both LLM-as-judge and RoBERTa-based personality detectors, especially toward introversion, indicating that agent evaluation may fail even when human experts perceive consistent behavioral differences (Jayakumar et al., 27 Oct 2025). LLAMOS identifies a fundamental limit of purification-based defense: if an adversarial edit genuinely changes semantics, as by inserting “not,” the defender cannot restore the original label without hallucinating content (Lin et al., 2024). HPC-integrated agents inherit scheduler latency, data-movement constraints, and security boundaries (Ma et al., 17 Feb 2025). Trading agents face short backtest windows, weak treatment of transaction costs and liquidity, and ethical concerns such as simulated use of insider information or deceptive rationalization (Ding et al., 2024).

A common misconception is that an LLM agent must involve online weight updates or explicit reinforcement learning. The cited work shows otherwise: prompt-only defense agents, zero-shot grounding agents, role-prompted tutors, and retrieval-backed workflow orchestrators are all treated as agents because they operate over state, tools, and iterative decision loops (Lin et al., 2024, Yang et al., 2023, Jayakumar et al., 27 Oct 2025). A second misconception is that multi-agent organization automatically guarantees robustness. Several papers instead report convergence failures, infinite loops, brittle coordination, or strong dependence on prompt design and agent-role separation (Lin et al., 2024, Wang et al., 2 Aug 2025).

Taken together, the literature presents the LLM agent as a broad systems concept: a LLM, or a set of such models, embedded in a structured loop of observation, memory, reasoning, action, and reflection, with the surrounding architecture determining whether it behaves as a classifier front-end, planner, critic, controller, simulator, collaborator, or autonomous developer (Barua, 2024, Wang et al., 2 Aug 2025). This suggests a shift from viewing LLMs as static sequence models to treating them as programmable decision systems whose competence depends as much on role design, tool interfaces, feedback channels, and evaluation regimes as on model scale alone.