Persistent Cross-Attempt State Optimization for Repository-Level Code Generation

Abstract: LLMs have achieved substantial progress in repository-level code generation. However, solving the same repository-level task often requires multiple attempts, while existing methods still optimize each attempt in isolation and do not preserve or reuse task-specific state across attempts. In this paper, we propose LiveCoder, a novel framework for repository-level code generation based on cross-attempt knowledge optimization. LiveCoder maintains persistent task-specific state from prior attempts to guide subsequent generation. This state includes success knowledge, which captures reusable signals from previously strong repositories, failure knowledge, which records unsuccessful outcomes and their diagnostic signals, and a historical-best repository, which preserves the strongest result found so far and prevents regression. These components collectively transform repeated repository generation into a persistent, knowledge-driven optimization process. We evaluate LiveCoder using four frontier LLMs on two representative repository-level code generation benchmarks. Extensive experimental results demonstrate the effectiveness and efficiency of LiveCoder, improving the functional score by up to 22.94 percentage points, increasing repository reuse to 81.58%, and reducing cost by up to 53.63% on RAL-Bench while maintaining broadly stable non-functional quality.

• The paper introduces a persistent optimization framework (ive) that reuses success and failure knowledge to achieve non-regressive, cumulative repository improvements.
• It employs structured extraction of design decisions and failure patterns to guide iterative, feedback-informed multi-file code generation.
• Experimental results demonstrate significant functional score gains and cost reductions across various leading LLM models.

Persistent Cross-Attempt State Optimization for Repository-Level Code Generation

Introduction and Motivation

The persistent cross-attempt optimization challenge in repository-level code generation emerges as modern LLMs are tasked with synthesizing increasingly complex multi-file repositories from natural language specifications. While prior systems predominantly focus on single-shot or single-trajectory refinement, practical code generation for real-world repositories inherently involves repeated cycles of end-to-end generation, repair, and resubmission. Existing systems do not systematically reuse knowledge across independent attempts: each attempt discards accumulated success and failure information as well as historical-best solutions, leading to redundant exploration, lack of cumulative improvement, and the potential to overwrite previously superior repositories.

A motivating example Figure 1 highlights that independent retries on the same problem lead to regression, whereas a system that preserves and reuses task-specific state can guarantee non-regression, systematic reuse, and cumulative progress.

Figure 1: A motivating example of repeated repository-level code generation, demonstrating the inability of isolated attempts to preserve earlier strong progress, contrasted with a cross-attempt stateful framework.

ive Framework: Cross-Attempt Knowledge Optimization

The proposed framework, ive, formalizes repository-level code generation as an iterative process that maintains and evolves a persistent task-specific state across attempts. This state comprises three critical components:

• Success Knowledge ($K^+$): Structured signals extracted from high-performing repositories, encoding repository-level design, interface choices, dependency strategies, and validated implementation patterns.
• Failure Knowledge ($K^-$): Systematic extraction of execution and repository failures, including interface contract violations, missing components, and behavioral deficits, to guide subsequent attempts away from known failure modes.
• Historical-Best Repository ($R^*$): Monotonically updated artifact representing the best functional outcome achieved, guaranteed to be non-regressive across attempts.

At each attempt, the generation module receives both $K^+$ and $K^-$, as well as feedback regarding $R^*$. The framework updates these with structured LLM-based extraction and only allows progression or non-regressive fallback, handling both exploration and exploitation in a unified optimization protocol.

Figure 2: Framework overview. ive maintains persistent task-specific state: Success Knowledge, Failure Knowledge, and the historical-best repository, guiding all subsequent attempts.

Success and Failure Knowledge Extraction and Usage

Success Knowledge captures every positive, repository-level design decision validated by prior repository executions. Each entry includes source context (e.g., attempt index, score), implementation strategies, and modular structure. This allows for powerful inheritance and transfer of repository-level priors across attempts.

Figure 3: Example Success Knowledge entry illustrating structured extraction of validated multi-file architecture and usage patterns for a Stegano task.

Failure Knowledge records structured negative experience, capturing failures at all abstraction levels: interfaces, inter-module contracts, incomplete paths, and unsuccessful repair attempts. During subsequent generations, this constrains the exploration space and eliminates repeats of known failure trajectories.

Figure 4: Example Failure Knowledge entry for Stegano, summarizing actionable failure evidence and carry-over constraints for repository generation.

Both knowledge forms are encoded as textual schemas, automatically extracted and semantically matched to task requirements prior to each attempt. This modular knowledge embedding is directly injected into LLM prompting, enabling experience-grounded generation, not simply retry-based sampling.

Preservation and Selection of Historical-Best Repositories

A core tenet is that repository-level optimization should be monotonic: the framework always retains the highest-scoring repository observed so far. If a new attempt does not improve on $R^*$, the final output defaults to the historical best, preventing any regression. Direct reuse is triggered when $R^*$ achieves full score, and after all attempts, the best repository is returned for evaluation.

Experimental Evaluation

Extensive experiments are conducted on RAL-Bench and NL2Repo-Bench, two canonical and challenging benchmarks for repository-level generation. Evaluation covers functional correctness (test pass rate) and non-functional metrics (maintainability, security, robustness, efficiency, resource usage) under four frontier LLMs: GPT-5-2025-08-07, DeepSeek-V3-0324, Claude-Sonnet-4.5-20250929, and Gemini-3-Pro-Preview.

Quantitative Results

ive achieves strong numerical gains relative to all baselines—Direct, Self-Reflection, SE-Agent, AlphaEvolve, CSE, and Live-SWE-Agent—across multiple axes (Tables reported in the paper):

Functional Score Improvements on RAL-Bench (selected models):

• GPT-5: 19.70 points over Direct
• DeepSeek-V3: 10.00 points
• Claude-Sonnet-4.5: 27.68 points
• Gemini-3-Pro-Preview: 25.30 points

Functional improvement is monotonic across repeated attempts, and repository reuse rate rises up to 81.58% (Gemini-3-Pro-Preview), reflecting systematic exploitation of accumulated knowledge.

Cost decreases with knowledge evolution: up to 53.63% cost reduction (Claude-Sonnet-4.5) from Attempt 1 to Attempt 4, as reusable knowledge prunes redundant generation and repair.

Non-functional attributes (ISO/IEC 25010) remain broadly stable, indicating that aggressive functional optimization does not degrade maintainability or robustness.

Error Analysis and Knowledge Evolution

Residual failures concentrate in integration bottlenecks—packaging, incomplete functionality, contract violations—reflecting genuine limits of LLM reasoning rather than lack of persistent experience or insufficient optimization cycles.

Figure 5: Residual functional failures after the final knowledge-evolved attempt, stratified by failure category, underscoring persistent LLM limitations in deep integration and contract preservation.

Ablation Analysis

All three state components are critical. Removing any one—Success Knowledge, Failure Knowledge, or Historical-Best Repository—substantially degrades performance, and the dominant effect is model-dependent. For example, Claude-Sonnet-4.5 degrades by 36.5 points without Success Knowledge; GPT-5 is especially sensitive to Failure Knowledge removal.

Practical and Theoretical Implications

ive establishes that repository-level code generation should be framed as a knowledge-driven, persistent optimization process, not as independent retries or pure in-attempt search. The formalization and operationalization of Success/Failure Knowledge and non-regressive historical-best selection enable cumulative improvement, cost efficiency, and reduced task-level regression. This aligns more closely with human engineering practice and supports persistent, experience-based AI programming.

Further, ive's modular protocol naturally generalizes to domains with long-horizon, multi-attempt generation and can be seen as a blueprint for stateful agentic LLMs that must accumulate and preserve execution-grounded knowledge.

Future Directions

• Meta-Knowledge Structuring: Advancing beyond task-local knowledge to enable cross-task or domain-general re-use.
• Backbone-Adapter Design: Tailoring knowledge extraction and injection strategies to match architectural or tokenizer constraints of target LLM backbones.
• Domain-General Applications: Extending persistent experience optimization to scientific discovery, design automation, or agentic planning.

Conclusion

This work formalizes and operationalizes persistent cross-attempt optimization for repository-level code generation, showing that systematically preserved and injected task-specific knowledge yields significant improvements in effectiveness, stability, and cost over previous baselines. The framework’s modular state maintenance—encompassing Success Knowledge, Failure Knowledge, and the historical-best repository—enables non-regressive, cumulative repository synthesis, setting a new standard for experience-based agentic code generation.