Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution

Abstract: Code LLMs need repository-level context to resolve imports, APIs, and project conventions. Existing methods inject this knowledge as long inputs (retrieved through RAG or dependency analysis) or through per-repository fine-tuning and LoRA -- costly at repository scale and brittle to evolving codebases. We introduce Code2LoRA, a hypernetwork framework that generates repository-specific LoRA adapters, effectively injecting repository knowledge with zero inference-time token overhead. Code2LoRA supports two usage scenarios: Code2LoRA-Static converts a single repository snapshot into an adapter, suitable for comprehension of stable codebases; while Code2LoRA-Evo maintains an adapter backed by a GRU hidden state updated per code diff, suitable for active development of evolving codebases. To evaluate Code2LoRA against parameter-efficient fine-tuning baselines, we build RepoPeftBench, a benchmark of 604 Python repositories with two tracks: a static track with 40K training and 12K test assertion-completion tasks, and an evolution track with 215K commit-derived training and 87K commit-derived test tasks. On the static track, Code2LoRA-Static achieves 63.8% cross-repo and 66.2% in-repo exact match, matching the per-repository LoRA upper bound; on the evolution track, Code2LoRA-Evo achieves 60.3% cross-repo exact match (+5.2 pp over a single shared LoRA). Code2LoRA's code can be found at https://anonymous.4open.science/r/code2lora-6857; the model checkpoints and RepoPeftBench datasets can be found at https://huggingface.co/code2lora.

• The paper introduces Code2LoRA as a hypernetwork-based approach that generates repository-specific LoRA adapters for frozen code LMs, achieving per-repo performance with zero inference token cost.
• Empirical results on RepoPeftBench show that both static and evolution modes outperform traditional context-injection and fine-tuning methods across multiple metrics.
• The paper demonstrates that GRU-based recurrent aggregation maintains robust performance over long commit histories, ensuring scalable adaptation to evolving codebases.

Authoritative Summary of "Code2LoRA: Hypernetwork-Generated Adapters for Code LLMs under Software Evolution"

Motivation and Framework

Code LLMs (CLMs) such as Qwen2.5-Coder require repository-level context to resolve imports, API usages, and project-specific conventions for code completion and assertion tasks. Traditional methods inject this context via long input sequences (using RAG or dependency analysis), which incur prohibitive token costs and degrade performance as repository size grows (Figure 1). Adapter-based approaches, e.g., LoRA/QLoRA, push knowledge into parameters but are brittle to codebase evolution, requiring retraining after each commit.

Code2LoRA introduces a hypernetwork-based method generating repository-specific LoRA adapters for a frozen CLM. The architecture is divided into two operational modes:

• Static Mode: Generates a LoRA adapter from a single repository snapshot, suitable for stable codebases.
• Evolution Mode: Maintains a repository adapter via GRU aggregation over sequential code diffs, suitable for actively evolving repositories.

This design eliminates inference-time token overhead and supports adaptation to unseen codebases without retraining, addressing both the scaling and brittleness limitations of prior approaches.

Figure 1: Token length distributions for prefix-only (left) and DRC+prefix (right) input formats across all splits. Vertical dashed lines mark common context window sizes. Prefix-only inputs are compact (median 224 tokens), while DRC+prefix inputs have a heavy right tail requiring larger context windows.

Methodology

The Code2LoRA framework consists of three components:

• Repository Encoder: Uses Qwen3-Embedding-0.6B to map codebase snapshots or diffs into dense embeddings via a two-step process (file-level mean pooling, followed by weighted aggregation across files).
• Hypernetwork: Projects repository embeddings into LoRA adapters via a shared MLP trunk and per-module heads (generating decomposed rank-16 weight deltas for all 7 transformer projection types). In evolution mode, a GRU aggregates diff embeddings, and its hidden state conditions the LoRA-generation head.
• Frozen Code LM: Receives the adapter and performs inference; only the hypernetwork is trained.

Static and evolution modes differ in how knowledge is refreshed: static mode uses a single embedding, while evolution mode employs a recurrent hidden state updated per commit.

Benchmarking: RepoPeftBench

RepoPeftBench is constructed for rigorous evaluation of repository-level parameter-efficient fine-tuning (PEFT). It comprises 604 MIT-licensed Python repositories, divided into in-distribution and post-scrape out-of-distribution splits. Two tracks are established:

• Static Track: Assertion-completion tasks from a single repository snapshot.
• Evolution Track: Tasks indexed by commit history, with assertions extracted after each commit modifying test code.

Evaluation metrics include exact match (EM), EditSim, and CodeBLEU. Both in-repo (IR) and cross-repo (CR) splits are used, with per-repo LoRA adapters serving as upper bounds on in-repo adaptation. The benchmark exposes the cost and limitations of context-injection baselines, adapter finetuning, and contextually generated LoRA approaches.

Figure 2: Bursty commit pattern, illustrated using randomly selected 5 repositories out of the 604. Test-touching commits arrive irregularly; the median repository accumulates over 100 such commits, motivating per-commit adaptation under software evolution.

Empirical Results

Static Track

Code2LoRA (static) achieves 63.8% EM on CR and 66.2% EM on IR, matching the per-repo LoRA upper bound for in-repo adaptation and outperforming FFT, RAG, DRC, sLoRA, and a strengthened Text2LoRA baseline (input-matched, head-matched). The results are robust across EditSim and CodeBLEU, with consistent gains even under varying repository sizes.

Evolution Track

Adapting via GRU aggregation, Code2LoRA (evolution) scores 60.3% EM on CR in the commit-derived scenario—outperforming static snapshot methods, sLoRA, and per-repo LoRA, with a gain of +5.2 pp over a shared LoRA. The performance advantage persists throughout long commit histories (Figure 3), with minimal downward drift, demonstrating resilience to staleness and efficient knowledge refresh.

Figure 3: CR-test exact-match vs. normalized commit position (51 held-out repositories, commit-derived prefixes). Each repository's timeline is scaled to 0–100%; points are qna-weighted means per 5% bin.

Out-of-Distribution Evaluation

On repositories created post-scrape, Code2LoRA maintains strong performance (74.1% EM), outperforming all baselines and demonstrating robust generalization to unseen repository types. The narrower OOD margin (relative to evolution-track CR) is attributed to shorter assertion targets in the OOD split (Appendix), not loss of adaptation capacity.

Adapter Analysis

The manifold of generated LoRA adapters is diverse, semantically meaningful, and organized—repositories with similar codebases cluster together, and clusters align with coherent EM ranges (Figure 4). Generated adapters exhibit module-specific adaptation, sharply contrasting with FFT+DRC's uniform deltas (Figure 5). This structural bias explains the superior transfer and generalization.

Figure 4: t-SNE of generated LoRA adapters for 52 CR-test repositories (PCA pre-reduction to 50 dims, then t-SNE). Color indicates per-repo Exact Match (%). Repositories with similar codebases tend to cluster together, and clusters show coherent EM ranges, demonstrating that the hypernetwork learns a smooth, semantically meaningful adapter manifold.

Figure 5: Comparison of per-module weight norms. Top: Code2LoRA generates repo-specific LoRA adapters with varying weight distributions across module types. Bottom: FFT+DRC applies a uniform weight delta. Code2LoRA's structured, repo-specific adaptations explain its stronger cross-repo performance.

Numerical Highlights and Contradictory Claims

• Context injection methods are consistently outperformed: RAG and DRC baseline results fall below those of parameter-injected approaches, even when provided with import-resolved context.
• Code2LoRA achieves per-repo LoRA upper bound without per-repo training: Cross-repo transfer via the hypernetwork is more effective than fitting a standalone adapter on the IR data budget.
• GRU-based recurrent hypernetwork provides a significant advantage in evolving codebases: Snapshot methods degrade as commit history unfolds, while recurrent aggregation maintains stability and performance.

Implications and Future Directions

Practically, Code2LoRA enables scalable repository-adaptation for CLMs with negligible inference cost (zero extra tokens, rapid adapter generation), supporting deployment across massive and frequently changing codebases. Theoretically, it demonstrates that parameter-efficient adaptation via hypernetwork-generated LoRA can distill repository-level conventions, structural knowledge, and evolutionary history much more effectively than input-based context injection or static per-repo fine-tuning.

The methodology is language- and task-agnostic in principle, opening avenues for cross-language repository adaptation and downstream task expansion beyond assertion completion. Future work might explore scaling to larger backbones, more aggressive adapter compression, and further refinement of evolutionary adaptation mechanisms.

Conclusion

Code2LoRA provides a robust, scalable framework for repository-level adaptation in code LMs, leveraging hypernetworks to generate semantic, task-specific LoRA adapters with zero inference-time overhead. Empirical evidence on RepoPeftBench demonstrates consistent superiority over both context-injection and classic adapter methods, especially when tracking active codebase evolution. The structural properties of the generated adapters further underscore the capacity for semantically coherent, cross-repository generalization. The approach positions itself as a foundational building block for future AI code assistants that are both cost-efficient and customizable to dynamic repository contexts.