---
title: LLM-Based Agents in Software Engineering
url: https://www.emergentmind.com/topics/llm-based-agents-in-software-engineering
type: topic
---

# LLM-Based Agents in Software Engineering

Large Language Model (LLM)-based agents in software engineering are autonomous or semi-autonomous systems that integrate one or more LLMs with explicit modules for planning, memory, tool invocation, and perception, enabling them to perform complex, multi-step SE tasks across the software development lifecycle (SDLC). These agents extend the capabilities of standalone LLMs beyond mere prompt-response interactions by maintaining state, decomposing tasks, reasoning about tool outputs, invoking external resources, and—via modular or multi-agent orchestration—addressing requirements, implementation, testing, documentation, and maintenance. Their architectures range from simple single-agent pipelines to sophisticated multi-agent platforms aligned with agile roles, employing reflection, tool integration, hierarchical memory, and human-in-the-loop oversight. This article surveys foundational definitions, agent architectures, operational mechanisms, evaluation methodologies, practical deployments, and emerging research directions, providing a comprehensive technical reference on LLM-based agents in software engineering.

## 1. Core Definitions and Formal Models

An LLM-based agent in software engineering is formally defined as a tuple comprising at minimum: a perception module to encode environmental observations as LLM-ready context, a memory module (semantic, episodic, procedural) to persist relevant information and serve as intermediate context, and an action policy that may trigger both internal reasoning and external tool invocations to maximize a task-specific utility function (e.g., pass rate, requirement satisfaction) [2409.09030]. The agent’s policy is typically parameterized as $\pi_\theta(\mathcal{I} \times \mathcal{M} \rightarrow \mathcal{A})$, where $\mathcal{I}$ is the input embedding, $\mathcal{M}$ is memory, and $\mathcal{A}$ is the set of possible actions.

In multi-agent systems, additional orchestration logic governs role allocation, communication topology (centralized, decentralized, hierarchical, or nested), and collaborative/competitive paradigms [2404.04834, 2511.08475]. An LLM-based MAS can be abstracted as a set of such agent tuples organized under an orchestration platform that manages state transitions, message routing, and coordination, often employing reflection and “rethink” steps after every action.

**Agent types** can be grouped as:
- _Single-agent_: Monolithic pipelines (plan–generate–validate).
- _Multi-agent_: Modular pipelines with agents for planning, coding, testing, reviewing, etc.; typically assigned explicit SE roles.

LLM-based agents must satisfy autonomous criteria: internal planning, persistent memory, multi-turn interaction, tool integration, decision-making, option selection, and adaptive behavior [2408.02479].

## 2. Agent Architectures and Workflow Taxonomy

LLM-based agent architectures in SE range from elementary prompt-based systems to highly modular, multi-agent frameworks. The principal design patterns include:

- **Role-Based Cooperation**: Agents are assigned discipline-specific roles (e.g., product owner, architect, developer, tester, reviewer) and operate in pipelines or cooperative groups [2511.08475, 2510.03463, 2403.15852].
- **Self-Reflection and Cross-Reflection**: Agents revisit and introspect on actions/results to revise future steps or critique peer outputs, often reducing hallucination and improving solution quality [2511.03153, 2511.08475].
- **Tool-Augmented Loops**: Agents interact with the environment by issuing tool calls (file operations, build systems, static/dynamic analyzers, web search, etc.); tool APIs return artifacts or feedback consumed in the next agent iteration [2407.01489, 2507.23370, 2511.13998].
- **Memory-Oriented Design**: Agents deploy hierarchical or hybrid memory—short-term for current artifacts, long-term for prior interactions, and structural memory (project graphs, summaries) to alleviate context-window limits and maintain goal focus [2510.03463].
- **Human-in-the-Loop Integration**: Agents systematically incorporate human validation/feedback at requirements, planning, code review, or deployment gates to ensure trustworthiness, regulatory compliance, and correctness [2411.12924, 2505.04251].

The orchestration logic can be:
- **Pipelined** (e.g., Waterfall, Agile): Stages are executed in sequence, with artifacts handed off deterministically (requirements → design → code → test → review) [2403.15852, 2510.03463].
- **Debate/Consensus**: Parallel agents propose solutions, then coordinate via voting or critique to select the best outcome [2511.08475].
- **Layered/Hierarchical**: Agents are deployed or escalated dynamically based on task complexity or failure recovery requirements [2511.08475].

Representative frameworks include ChatDev, MetaGPT, ALMAS, HULA, FlowGen, and RefAgent, many of which implement role-based and self-reflective patterns [2404.04834, 2403.15852, 2411.12924, 2510.03463, 2511.03153].

## 3. Operational Mechanisms: Planning, Memory, Reasoning, and Tool Use

**Planning and Decomposition:** Agents decompose high-level goals into sub-tasks using chain-of-thought, structured prompts, or learned SOPs, then map sub-tasks to specialized agents (or self-steps in single-agent systems) [2409.09030, 2510.03463, 2403.15852]. Planning can be single-path (linear) or multi-path (branch, merge, filter).

**Memory Augmentation:** 
- _Semantic memory_ retrieves external documents, API specs, or repository summaries to enrich prompts at every step.
- _Episodic memory_ persists intermediate artifacts, localization steps, and full \((t,a,r)\) (thought–action–result) trajectories for subsequent access and feedback [2506.18824].
- _Procedural memory_ encodes role-specific heuristics, dynamic prompt templates, or skill-weights tuned via in-context learning or parameter-efficient fine-tuning [2409.09030, 2511.08475].

**Tool Use and Feedback:** LLM-based agents invoke tools via structured APIs (e.g., file edit, run tests, compilation, search) and process the feedback (test results, compiler output, log messages) to guide subsequent reasoning. Tool-invoking agents can operate in stateless or memoryful ReAct-style cycles [2511.13998, 2407.01489, 2507.23370].

**Self-Reflection and Verification:** Agents, especially in multi-agent systems, perform iterative self-evaluation (run/generate/refine) by integrating test failures, code review critiques, or formal verification outputs as feedback for plan updating or patch refinement [2511.03153, 2511.08475].

**Anti-pattern Detection:** Research on agent action trajectories reveals common failure modes—repetitive actions without follow-up, absence of intermediate test validation, and premature termination—which can be mitigated by architectural constraints, adaptive prompts, and explicit plan verification [2506.18824].

## 4. Application Domains and Benchmarking

LLM-based agents are deployed across a spectrum of SE tasks:

| SE Task                        | Example Agent System     | Key Evaluation Metric                      |
|------------------------------- |-------------------------|--------------------------------------------|
| Code Generation                | CodeAgent, ALMAS        | Pass@k, Correctness, Syntactic Validity    |
| Program Repair & Issue Fixing  | RepairAgent, Trae Agent | Pass@1, Correct Patch Rate, Test Coverage  |
| Refactoring                    | RefAgent                | Code Smell Reduction, Test Pass Rate       |
| Requirements Engineering       | MARE, HULA              | Precision, Recall, Task Fulfillment        |
| Full SDLC Automation           | ChatDev, MetaGPT, ALMAS | Fulfillment Rate, End-to-End Pass Rate     |
| Project Management/Agile       | CogniSim/CognitiveAgent | Backlog Reduction, Delivery Quality        |

Benchmarking frameworks include SWE-bench family (Lite, Verified), E2EDevBench, AGENTISSUE-BENCH, CodeAgentBench, LoCoBench-Agent, and various HumanEval/MBPP-derived datasets for function-level tasks [2407.01489, 2511.04064, 2505.20749, 2511.13998, 2403.15852].

**Advanced evaluation** involves not only test-pass rates and syntactic correctness but functional coverage across original and agent-created tests, LLM-based requirement verification, code-smell density via static analysis, and cross-session memory/consistency [2511.13998, 2511.03153, 2511.04064].

## 5. Empirical Results, Architectural Tradeoffs, and Observed Best Practices

Recent large-scale studies indicate:
- Multi-agent, role-divided pipelines outperform monolithic or single-turn approaches on complex SE tasks, yielding higher correctness (e.g., RefAgent's 90% Java unit-test pass rate and 52.5% code smell reduction [2511.03153]).
- Autonomous systems typically solve 30–60% of non-trivial GitHub issue tasks at repository scale; bottlenecks include requirement comprehension, error propagation from upstream planning, and insufficient self-verification [2511.04064, 2407.01489, 2507.23370].
- Simple, modular designs such as Agentless achieve comparable performance to stateful agents on benchmarks like SWE-bench Lite with far lower computational cost, when the task allows rigid decomposition [2407.01489].
- Long-context benchmarks (LoCoBench-Agent) demonstrate that agent comprehension degrades minimally at very large context windows, yet significant comprehension–efficiency tradeoffs and memory retention challenges persist [2511.13998].
- Best practices include separating planning from execution, explicit alignment between agent “thoughts” and actions, maintaining experience/trajactory buffers for meta-learning, embedding critique or reflection after each iteration, and, where possible, hybridizing tool use with retrieval augmented language model prompting [2506.18824, 2511.03153].

## 6. Challenges, Limitations, and Future Directions

Major unresolved technical challenges include:
- **Requirement grounding:** Failure rates are dominated by requirement omission and misinterpretation; enhanced requirement engineering modules, structured templates, and iterative coverage validation are needed [2511.04064].
- **Memory and context:** Context-window limits and weak multi-session retention hinder global reasoning and long-range dependency management. External memory, dynamic summarization, and hierarchical retrieval are open research areas [2510.03463, 2511.13998].
- **Agent coordination overhead:** Multi-agent communication can incur O($N^2$) scaling, introduce deadlocks, and drive up API and computation costs [2508.00083, 2511.08475].
- **Tool/Environment reliability:** SE agents are sensitive to environment nondeterminism, flaky LLM outputs, and fast-evolving toolchains/APIs [2505.20749].
- **Human–AI synergy and trust:** Human-in-the-loop is essential for compliance and high-precision settings, but remains labor-intensive without robust role allocation frameworks (e.g., RACI), audit traces, and explainability [2505.04251, 2411.12924].
- **Benchmarking gaps:** Lack of unified, high-fidelity, large-scale benchmarks that cover end-to-end SE processes, including artifact traceability, performance, maintainability, and developer revision effort [2511.04064, 2511.08475].

**Research opportunities** include developing dynamically adaptive agent topologies, integrating formal verification and safety constraints, scalable blackboard/tri-tier memory architectures, multi-modal perception (UI, diagram, code property graphs), and agent-centric qualification metrics that extend beyond simplistic pass@k measures [2508.00083, 2510.09721, 2411.12924].

## 7. Design Space, Best Practices, and Implications

Analyses of 94+ studies identify key architectural and workflow best practices [2511.08475]:
- Emphasize functional correctness and maintainability by rigorous specification adherence, role-based decomposition, and continuous validation.
- Prefer modularity via role-based cooperation, formalized agent interfaces, and hierarchical/adapter patterns for extensibility.
- Leverage self- and cross-reflection patterns to mitigate hallucinations and enforce semantic consistency.
- Manage resource–quality tradeoffs through incremental querying, dynamic agent deployment (hierarchical coordination), and judicious activation of human critique.
- Preserve adaptability via plug-and-play agent architectures, separated tool registries, and reuse of retrieval-augmented generation.
- Address trust, security, and regulatory concerns by enforcing human accountability in sensitive steps, audit trails, and formal resource management policies.

The emerging consensus is that LLM-based multi-agent systems, when architected with careful modularity, role alignment, and robust memory/tooling backbones, offer a promising solution space for automating large swaths of the SE workflow—contingent upon advances in requirement engineering, context management, and robust, multi-dimensional evaluation [2511.08475, 2404.04834].

Source: https://www.emergentmind.com/topics/llm-based-agents-in-software-engineering