Code2LoRA: Repository-Conditioned Adapters

• Code2LoRA is a framework that generates repository-specific LoRA adapters by mapping dense repository embeddings via a hypernetwork to efficiently adapt frozen code language models.
• Its design avoids inference-time token overhead by pre-computing adapter weights from full repository snapshots or diffs, providing a scalable alternative to retrieval-based methods.
• The system supports both static and evolutionary settings with shared parameter mapping, achieving significant gains in assertion completion tasks on Python repositories.

Searching arXiv for the named work and closely related code-LoRA papers to ground the article. Code2LoRA is a repository-conditioned adaptation framework for code LLMs in which repository knowledge is injected parametrically, as generated LoRA weights, rather than appended as retrieved text. In "Code2LoRA: Hypernetwork-Generated Adapters for Code LLMs under Software Evolution" (Hotsko et al., 4 Jun 2026), a frozen repository encoder compresses either a repository snapshot or a sequence of code diffs into a dense representation, and a trainable hypernetwork maps that representation into repository-specific LoRA adapters for a frozen base code model. The method is designed for software-engineering settings in which imports, internal APIs, helper utilities, naming conventions, and other project-specific regularities are distributed across many files, and it is explicitly formulated to avoid inference-time prompt expansion for repository context.

1. Concept and problem setting

Code2LoRA addresses repository-level conditioning for code LMs under two deployment regimes: stable codebases and evolving repositories. Its motivating task is assertion completion from test files, where the correct target may depend on definitions, conventions, or APIs located elsewhere in the repository. The framework is positioned against three prior families of approaches: retrieval-augmented long-context methods, dependency-analysis-based context injection, and per-repository fine-tuning or LoRA. The paper argues that these baselines are costly, brittle, or stale at repository scale, especially when repositories evolve commit by commit (Hotsko et al., 4 Jun 2026).

A central design objective is zero inference-time token overhead. In this usage, repository knowledge is converted into LoRA parameters before task inference, so the base LLM does not require extra repository-context tokens in its prompt. This directly contrasts with RAG- or DRC-style methods, which prepend repository material to each query and therefore pay a per-query context cost. The motivation is sharpened by repository-size statistics in RepoPeftBench: repository token counts have mean 284,509, median 165,376, and max 2,994,853, making direct context injection structurally expensive for many repositories.

The framework is also explicitly cross-repository. Rather than training a separate adapter for each repository, Code2LoRA learns a shared mapping from repository representation to adapter parameters. This shared mapping is intended to generalize to unseen repositories in cross-repo evaluation, while its recurrent evolutionary variant is intended to mitigate the staleness of static repository-specialized adapters under ongoing development.

2. Repository encoding and LoRA generation

All reported experiments use Qwen2.5-Coder-1.5B as the frozen base LLM and Qwen3-Embedding-0.6B as the frozen repository encoder (Hotsko et al., 4 Jun 2026). Repository context is encoded in two stages. Each file $f_i$ or diff $\Delta f_i$ is chunked into 4096-token chunks with 512-token overlap; chunk embeddings are mean-pooled to form a file vector $\mathbf{f}_i \in \mathbb{R}^{1024}$. For a full repository snapshot, file vectors are then aggregated with importance weights $w_i$, based on content distinctiveness, file size, and path importance, into

$$\mathbf{e} = \big[\textstyle\sum_i w_i \mathbf{f}_i \,;\, \max_i \mathbf{f}_i\big] \in \mathbb{R}^{2048}.$$

The generated adapters use standard LoRA-style low-rank updates,

$$\mathbf{W}' = \mathbf{W} + \tfrac{\alpha}{r}\mathbf{B}_m\mathbf{A}_m,$$

with rank $r=16$ and LoRA alpha $\alpha=32$. Unlike narrower Q/V-only designs, Code2LoRA adapts all seven attention and MLP projection types,

$m \in \{q,k,v,o,gate,up,down\},$

and the same $(\mathbf{A}_m,\mathbf{B}_m)$ for each module type is shared across all 28 transformer layers of Qwen2.5-Coder-1.5B. This means adapter generation is organized by module type rather than by separate per-layer LoRA weights.

For the static variant, the hypernetwork first computes

$\Delta f_i$0

where the shared trunk is a 2-layer MLP with GELU and $\Delta f_i$1. Per-module heads then generate

$\Delta f_i$2

with learnable log-scales $\Delta f_i$3 initialized to $\Delta f_i$4. The reported trainable size of this static hypernetwork is about 720M parameters. The design combines bounded outputs via $\Delta f_i$5 with learned module-specific magnitudes via $\Delta f_i$6, which the paper presents as a stabilization mechanism for adapter generation.

3. Static and evolutionary variants

Code2LoRA is split into two named variants. Code2LoRA-Static converts a single repository snapshot into an adapter and is intended for comprehension of stable codebases. Code2LoRA-Evo maintains an adapter backed by a recurrent hidden state updated from chronological code diffs and is intended for active development settings (Hotsko et al., 4 Jun 2026).

For the static path, the procedure is straightforward: encode a snapshot into $\Delta f_i$7, generate $\Delta f_i$8 for each module type, inject those weights into the frozen backbone, and run assertion completion with no repository-token augmentation. For the evolutionary path, the repository state evolves as

$\Delta f_i$9

where $\mathbf{f}_i \in \mathbb{R}^{1024}$0 is the embedding of the production-code diff at commit $\mathbf{f}_i \in \mathbb{R}^{1024}$1. The initial state $\mathbf{f}_i \in \mathbb{R}^{1024}$2 is projected from the initial repository embedding using Linear $\mathbf{f}_i \in \mathbb{R}^{1024}$3 GELU $\mathbf{f}_i \in \mathbb{R}^{1024}$4 LayerNorm. Adapter generation then reuses the same projection heads as the static model, with $\mathbf{f}_i \in \mathbb{R}^{1024}$5 substituted for the snapshot embedding.

The reported additional recurrent machinery is modest relative to the hypernetwork trunk but not negligible in absolute terms: the GRU + initializer add about 25M parameters, bringing total trainable parameters to about 745M. The appendix specifies a 1-layer GRU, hidden size 2048, and truncated BPTT with detach every $\mathbf{f}_i \in \mathbb{R}^{1024}$6 commits. The paper’s temporal modeling choice is to encode diffs offline and update the hidden state incrementally, thereby avoiding full re-encoding of the repository after every commit.

Both variants are trained end-to-end with cross-entropy on assertion completion,

$\mathbf{f}_i \in \mathbb{R}^{1024}$7

where $\mathbf{f}_i \in \mathbb{R}^{1024}$8 for Static and $\mathbf{f}_i \in \mathbb{R}^{1024}$9 for Evo. Only the hypernetwork is trained; both the base LLM and the repository encoder remain frozen.

4. Benchmark design and empirical performance

To evaluate repository-level PEFT under both static and evolving codebases, the paper introduces RepoPeftBench, a benchmark of 604 Python repositories from GitHub with two tracks: a static snapshot track and an evolution track derived from commit histories (Hotsko et al., 4 Jun 2026). The repositories are selected under requirements that they be Python, use pytest or unittest, have a permissive license, and show recent activity, with a temporal cutoff at 2025-04-01. The benchmark contains an in-distribution set of 512 repositories and an OOD temporal holdout of 92 repositories.

The central task is assertion completion. Inputs include all imports, the enclosing class definition if any, helper methods such as fixtures or setUp / tearDown, and the test function body up to the assertion cut point. The target is the expected value in the assertion, either the right-hand side of a binary comparison or the last argument of an assertion call. The primary metric is Exact Match (EM), computed after whitespace collapsing, trailing-punctuation removal, and a relaxed matching rule that tolerates overgeneration; Edit Similarity and CodeBLEU are reported as additional metrics.

The static track contains 62,294 tasks, split into 39,612 train, 11,046 validation, and 11,636 test. The evolution track contains 399,649 commit-derived tasks, split into 215,129 train, 97,727 validation, and 86,793 test. Cross-repo evaluation holds out whole repositories, whereas in-repo evaluation splits tasks within the training repositories.

Setting	Code2LoRA result	Comparison highlighted in the paper
Static CR Test EM	63.8	FFT + RAG: 53.9
Static IR Test EM	66.2	pLoRA: 64.0
Evolution CR Test EM	60.3	sLoRA: 55.1
Evolution IR Test EM	64.5	pLoRA: 64.2
OOD Test EM	74.1	sLoRA: 72.3

On the static track, Code2LoRA-Static achieves 63.8 cross-repo EM and 66.2 in-repo EM. The cross-repo result exceeds the strongest reported baseline, FFT + RAG at 53.9, by 9.9 percentage points. In-repo, the abstract describes Code2LoRA-Static as matching the per-repository LoRA upper bound, while the table shows a small numerical edge over pLoRA: 64.0.

On the evolution track, Code2LoRA-Evo reaches 60.3 cross-repo EM and 64.5 in-repo EM. The headline gain is +5.2 pp over a single shared LoRA in cross-repo evaluation. The paper also reports that static adapters go stale: Code2LoRA-Static drops from 63.8 cross-repo EM on static snapshots to 55.7 on commit-derived inputs, whereas Evo recovers part of that loss through recurrent updating.

Several analyses in the paper clarify what the reported gains mean. A repository-count scaling study shows that Static reaches 57.7 CR Test EM with only 10 repositories and scales roughly log-linearly up to about 200 repositories, after which performance saturates. Adapter diversity analysis shows pairwise cosine similarities spanning nearly the full $w_i$0 range, with mean 0.01 and std 0.94, which the authors interpret as evidence against collapse to a single mean adapter. The per-repository variance analysis also indicates that cross-repository hypernetwork generation is more stable than per-repository LoRA when data is sparse.

5. Position within code-specific LoRA research

Code2LoRA belongs to a broader line of work on code-specialized parameter-efficient adaptation, but its operating point is distinct. A useful precursor is "LoRACode: LoRA Adapters for Code Embeddings" (Chaturvedi et al., 7 Mar 2025), which studies LoRA-based parameter-efficient fine-tuning for Code2Code and Text2Code retrieval over base encoders such as CodeBERT, GraphCodeBERT, and UniXcoder. In that setting, the backbone is frozen, LoRA is inserted into Query and Value projections only, and only about 1.83%–1.85% of base parameters are trainable. LoRACode’s best results are retrieval-specific: up to 9.1% MRR gain for Code2Code search on XLCoST and up to 86.69% MRR gain for language-specific Text2Code adapters on CodeSearchNet. The practical lesson drawn there is adapter specialization by task and language; Code2LoRA instead specializes by repository identity and repository evolution.

A second adjacent but methodologically different line concerns weight-space interpretation of adapters. "W2T: LoRA Weights Already Know What They Can Do" (Han et al., 16 Mar 2026) studies what can be inferred from saved LoRA weights alone, without running the base model or accessing training data. It canonicalizes LoRA updates via QR decomposition followed by SVD and then embeds those canonicalized updates with a Transformer for attribute classification, performance prediction, and adapter retrieval. In relation to Code2LoRA, this suggests a plausible downstream tooling layer: repository-conditioned adapters generated by Code2LoRA could themselves be indexed, compared, or retrieved in weight space once factorization ambiguity is removed.

These neighboring works delineate three different meanings of code-oriented LoRA specialization. LoRACode focuses on retrieval-specialized compact adapters; W2T focuses on reading adapter behavior from checkpoint weights; Code2LoRA focuses on generating repository-specific adapters from repository representations. The shared substrate is LoRA, but the objects being optimized are different: retrieval geometry, weight-space metadata, and repository-conditioned task behavior, respectively.

6. Limitations, misconceptions, and interpretation

Several misconceptions are addressed directly by the reported evidence. First, Code2LoRA is not a retrieval-augmented prompting method. Its claim of zero inference-time token overhead applies specifically to repository-context injection: once the adapter is generated, no additional repository tokens are prepended to the prompt. Second, it is not equivalent to per-repository LoRA. Per-repo LoRA remains an in-repo upper-bound baseline in the paper, but Code2LoRA is designed to produce adapters for unseen repositories without per-repository optimization (Hotsko et al., 4 Jun 2026).

The paper also places clear limits on the method’s scope. Empirically, evaluation is restricted to Python repositories, a single backbone (Qwen2.5-Coder-1.5B), and one downstream task (assertion completion). The reported metrics—EM, Edit Similarity, and CodeBLEU—do not guarantee functional equivalence. The authors mention a pytest-based execution probe on a runnable slice, but not a full execution-based benchmark across all cases. The OOD holdout also has a target-length artifact: OOD targets are shorter, with median 7 chars versus 12–13 in-distribution, inflating EM for all methods.

A further practical limitation is scale. Although Code2LoRA avoids storing one trained adapter per repository, the hypernetwork is large in absolute terms: about 720M parameters for Static and 745M for Evo. The efficiency claim is therefore primarily about shared storage and adaptation topology, not about a universally small trainable model. The paper’s efficiency table makes this explicit: Code2LoRA-Static is reported with 0 extra tokens, <10 ms/repo, and +679 MB shared, whereas Code2LoRA-Evo adds 0 extra tokens, <10 ms + GRU update, and +65 MB extra storage. This suggests strong storage scaling with repository count relative to per-repo LoRA, but not negligible standalone model size.

Finally, the name can be confused with unrelated literatures. Code2LoRA concerns LoRA adapters for code LLMs, not LoRa wireless communication. This distinction matters because the supplied corpus also includes LoRa communication papers such as "Design and Analysis of Chirp-Layered Superposition Coding for LoRa" (Huang et al., 7 Apr 2026) and "Network-Coded Cooperative LoRa Network with D2D Communication" (Alves et al., 2021), which address spectral efficiency, coding, and outage behavior in chirp-based wireless systems rather than parameter-efficient adaptation for code LMs. Within code-ML research, however, the central significance of Code2LoRA is precise: it reframes repository conditioning as adapter generation and software evolution as recurrent adapter updating over diffs, rather than as prompt-time context injection or repeated repository-specific fine-tuning.