LLM-Integrated Software Systems

• LLM-integrated software is a class of systems that embed large language models as first-class components to enable code generation, reasoning, and multi-agent coordination.
• These systems use multi-layered architectures with presentation, logic, integration, data management, and sidecar components to ensure scalability, maintainability, and security.
• LLM integration spans all SDLC phases, automating requirements, design, implementation, testing, deployment, and maintenance while addressing challenges like software aging and evolving security threats.

LLM-integrated software refers to applications or systems in which one or more LLMs are embedded as first-class software components, providing core functionalities such as code generation, reasoning, workflow orchestration, knowledge retrieval, or natural language interaction. Integration can occur across the entire software development lifecycle, enabling automation of requirements elicitation, design, implementation, testing, deployment, and maintenance, and transforming both the architecture and engineering workflow of contemporary software systems.

1. Definitional Scope and Taxonomy

LLM-integrated software extends beyond treating LLMs solely as back-end services or automation tools. Each LLM invocation point within an application—denoted as an “LLM component”—can be architecturally characterized by a 13-dimensional taxonomy, encompassing method of invocation (e.g., single or iterative), control manner (data calculation vs. control decisions), prompt construction and validation strategies, leveraged LLM skills (creation, reasoning, planning), output format (free text, code, structure), and the downstream consumer of the output (Weber, 2024). Applications are thus composed of multiple, heterogeneous LLM components, often leveraging different skills and fulfilling distinct roles.

The integration spectrum includes designs in which LLMs act as:

• Plug-in microservices, processing user queries or generating artifacts
• Process orchestrators with control over application logic
• Reasoning engines supporting high-level planning or decision support
• Sub-agents in multi-agent systems distributed across development roles

This taxonomy allows comparison, systematic analysis, and rigorous design and testing of LLM use within complex systems.

2. Reference Architectures and System Patterns

LLM-integrated systems are typically organized as multi-layered architectures, incorporating specific components and cross-cutting concerns for privacy, security, and monitoring (Bucaioni et al., 22 Jan 2025). A canonical architecture comprises:

• Presentation Layer: User interfaces, adapters for incoming requests (REST/gRPC), session validation, and authentication
• Application Logic Layer: Workflow orchestrators handling intent parsing, sub-task allocation, and pre/post-processing modules
• LLM Integration Layer: Prompt management (templating, retrieval-augmented context), LLM inference modules (via API/local deployment), post-processing (output parsing, format adherence)
• Data Management Layer: Model and adapter checkpoints, vector databases for embedding storage (RAG), session and historical memory, API integrations
• Sidecars:
  • Monitoring & Evaluation (continuous metrics, error flagging)
  • Guardrails (input/output filtering, content moderation, PII redaction, policy enforcement)

Microservice deployment, adapter patterns (for multi-provider/model compatibility), pipeline chaining, sidecar isolation, and event-driven orchestration recur as best practices for maintainability, scalability, and compliance.

3. Methodologies of Integration Across the SDLC

LLM integration now extends through all stages of the software development lifecycle (Vieira, 2024, He et al., 2024, Tawosi et al., 3 Oct 2025):

• Requirements Engineering: LLMs parse stakeholder transcripts and standards, extracting functional and non-functional requirements, often backed by retrieval-augmented generation (RAG) pipelines and structured extraction prompts.
• Design: LLM-driven architecture recommender systems select design patterns, perform trade-off analysis, and automate the creation of C4 or UML models via collaborative multi-agent dialogues among role-specialized “designers” (Szczepanik et al., 26 Oct 2025).
• Implementation: Prompt-engineered code generation, real-time code analysis (hybrid LLM and static tools), and LLM-based program repair services (with feedback loops) enable continuous code artifact production and automated remediation.
• Testing & Validation: Automated generation of unit, integration, load, and fuzz tests using LLMs; fault injection via LLM agents; metamorphic and higher-order testing frameworks address the oracle problem in domains like legal-critical software (Gogani-Khiabani et al., 16 Sep 2025).
• Deployment & Monitoring: LLM-powered infrastructure-as-code generation, anomaly detection on logs, issue triage bots, and ongoing trustworthiness assessment dashboards (Vieira, 2024).
• Maintenance: Automated bug triage, patch localization (LLM-enhanced recommenders), and code/documentation traceability using LLM-based entity recognition and feature/fault localization (Yu et al., 2024, Fuchß et al., 4 Nov 2025, Tan et al., 9 Jan 2026).

Multi-agent frameworks (e.g. ALMAS, FlowGen) instantiate agile pipelines, aligning LLM agents with roles such as Product Owner, Developer, Tester, Reviewer, with task assignment, context management, and self-refinement stages (Tawosi et al., 3 Oct 2025, Lin et al., 2024). Process selection (Waterfall, Agile, V-Model) strongly influences system adaptability, code quality, and resource efficiency (Ha et al., 17 Sep 2025).

4. Reliability, Safety, and Software Aging

Long-term reliability studies indicate that LLM-generated and LLM-integrated services are subject to software aging phenomena, including persistent memory leaks (rates measured up to 37.7 MB/hour), progressive latency growth, and performance instability under prolonged load (Santos et al., 28 Oct 2025). Quantitative modeling utilizes linear or power-law fits (e.g., M(t) = M₀ + α·t, R(t) = R₀ + β·t^γ) and non-parametric trend tests (Mann-Kendall, Sen’s slope) applied to time series of resource and performance metrics.

Different service archetypes (CPU-bound, I/O-bound, stateless) exhibit variable susceptibility to aging, with complex, stateful workloads at higher risk. Recommendations include:

• Periodic service rejuvenation (automated restarts below observed leak doubling time)
• Adaptive monitoring and container restarts based on trend-extrapolated resource usage
• Encouragement of explicit resource cleanup logic via prompt engineering and code generation policies

Reliability assessment mandates continuous, long-duration stress testing and explicit focus on non-functional degradation in CI/CD pipelines (Santos et al., 28 Oct 2025).

5. Security, Robustness, and Attack Surface

LLM-integrated software defines new attack surfaces at the interface between user queries, application-constructed prompts, downstream LLM calls, and resultant outputs (Jiang et al., 2023). Two primary threat vectors arise:

• Insider: Malicious modification of prompt construction, data fetching, or LLM response post-processing.
• Outsider: Data poisoning via manipulation of domain knowledge bases or databases feeding the prompt.

Attacks result in integrity and privacy violations, injection of bias, disinformation, or sensitive information leakage. Formal security properties required for mitigations include integrity, source identification, attack detectability, and utility preservation.

Defense strategies employ cryptographic signing of inputs/outputs, meta-prompt monitoring (using the LLM as a semantic guard), and shield APIs that implement tamper-evident logs and fallback logic (Jiang et al., 2023).

Further, defect localization in LLM-integrated systems necessitates multi-layer artifact analysis—encompassing code, prompts, configuration files, and orchestrator state—often employing knowledge graphs with LLM-aware annotations and counterfactual reasoning to distinguish root causes from symptoms (Tan et al., 9 Jan 2026).

6. Evaluation, Benchmarking, and Engineering Challenges

LLM-integrated software evaluation spans functional correctness, security, reliability, performance, bias, and explainability. Benchmark suites cover function-level, class-level, and repo-level tasks (HumanEval, MBPP, SWE-Bench, etc.), with problem paradigms mapped to solution types: prompt-based (for simple tasks), fine-tuned (complex translation/repair), and robust agent-based systems (multi-file, multi-step) (Guo et al., 10 Oct 2025).

Key evaluation metrics and formulas:

• pass@k, precision, recall, F₁ for correctness and traceability tasks (Fuchß et al., 4 Nov 2025, Ha et al., 17 Sep 2025)
• Latency models: L(n) = T₀ + n·T₁ (n = prompt+context tokens)
• Composite trustworthiness: T = w₁·R + w₂·P + w₃·(1–B) + w₄·(1–L/L_max)
• Probabilistic determinism and test flakiness measures, particularly relevant for promptware (2503.02400)

Formal software engineering of LLM prompts ("promptware engineering") is recognized as a distinct discipline, requiring methodologies for prompt requirements, design, implementation, testing, debugging, and evolution tailored to probabilistic and ambiguous NL-based runtimes (2503.02400).

Empirical analysis establishes that agentic and process-structured multi-agent systems (Scrum, Agile pipelines) yield superior code quality and robustness, albeit with resource and compute penalties (Lin et al., 2024, Ha et al., 17 Sep 2025).

7. Open Research Directions and Best Practices

Persistent challenges and emerging frontiers include:

• Multi-agent coordination protocols balancing specialization, redundancy, and communication overhead (He et al., 2024)
• Context and memory management for large-scale, multi-step or multi-modal applications (Tawosi et al., 3 Oct 2025)
• Domain specialization and compliance-aware adaptation in safety- or legal-critical domains (Petrovic et al., 5 Jan 2026, Gogani-Khiabani et al., 16 Sep 2025)
• Formal integration of LLM outputs with verification methods (OCL constraints, model-driven engineering) (Petrovic et al., 5 Jan 2026)
• Security auditing, adversarial prompt testing, and continuous vulnerability monitoring (Jiang et al., 2023)
• Scalable feedback loops for self-refining, production-ready agentic systems (Tawosi et al., 3 Oct 2025, Fuchß et al., 4 Nov 2025)
• Prompt engineering patterns, prompt versioning, and systematic testing of “promptware” modules (2503.02400)
• Modular architectures supporting plug-in adapters, monitoring, guardrails, and feedback channels (Bucaioni et al., 22 Jan 2025)

Best practices emphasize systematic layering, clear separation of concerns, sidecar isolation for monitoring and security, detailed prompt and output validation, and explicit metrics-driven feedback throughout the software lifecycle.

---

LLM-integrated software represents a paradigm shift in both application design and the practice of software engineering. Its success requires a rigorous, multi-faceted approach encompassing reference architectures, process-oriented agentic coordination, continuous security and reliability evaluation, and the development of systematic methodologies for promptware engineering and lifecycle management.