NARRepair: Non-Autoregressive Program Repair

• NARRepair is a non-autoregressive code repair model integrating a repair action predictor, inter-token dependency extractor, and two-stage decoder to address APR challenges.
• It achieves significant speed gains, being 1.4–6.4 times faster than autoregressive systems, while maintaining competitive accuracy on benchmarks like Defects4J and QuixBugs.
• The design tackles code-specific issues such as over-correction, loss of inter-token dependencies, and missing contextual feedback through specialized architectural modules.

Searching arXiv for the cited NARRepair papers to ground the article in the current literature. NARRepair is a non-autoregressive code generation model for automatic program repair (APR) that adapts parallel decoding to bug fixing through three coordinated mechanisms: a repair action predictor, an inter-token dependency extractor, and a two-stage decoder (Yang et al., 2024, Yang et al., 2 Oct 2025). It was introduced as the first customized non-autoregressive model for APR, with the explicit objective of reducing the inference latency associated with autoregressive repair while maintaining high repair accuracy (Yang et al., 2024). In later reporting, it is characterized as achieving state-of-the-art comprehensive performance in terms of repair speed and accuracy, especially under limited repair time (Yang et al., 2 Oct 2025).

1. Concept and research setting

NARRepair emerged from a line of APR research in which deep learning-based systems predominantly generated patches in an autoregressive manner. In that setting, future tokens are predicted from past tokens, and code is generated token by token. The reported motivation for NARRepair is that this mode of generation creates a substantial time delay, which becomes particularly serious for APR models with a large number of parameters (Yang et al., 2024, Yang et al., 2 Oct 2025).

The model is therefore positioned against a specific bottleneck in neural APR: inference latency rather than only patch quality. In the 2024 report, this motivation is framed in terms of “real-time program repair,” especially for domains such as embedded systems, robotics, and large-scale software development (Yang et al., 2024). The 2025 report retains the same core motivation but emphasizes “best performance within the limited repair time” and compares NARRepair directly against both ML-based autoregressive APR models and LLM-based repair systems (Yang et al., 2 Oct 2025).

A central point in the NARRepair literature is that APR is not treated as generic sequence transduction. The model is presented as a customized non-autoregressive repair architecture rather than a direct transfer of non-autoregressive machine translation methods to code repair (Yang et al., 2024, Yang et al., 2 Oct 2025). This distinction is important because the reported failure modes of naïve non-autoregressive repair are code-specific and repair-specific rather than merely sequence-modeling artifacts.

2. Design rationale and problem formulation

The formal contrast between autoregressive and non-autoregressive generation is stated as follows in the 2024 description: an autoregressive model predicts according to $y_t = f(y_{1:t-1}, x)$, whereas a non-autoregressive model predicts all positions in parallel as $y_t = f(x)$ (Yang et al., 2024). NARRepair adopts the latter regime in order to avoid the sequential dependency that slows conventional APR.

The model is built around three limitations that arise when non-autoregressive generation is applied naïvely to APR. The first is over-correction: unlike machine translation, where all source tokens are typically transformed, APR usually requires keeping most program tokens unchanged. The second is loss of inter-token dependency information: parallel decoding weakens the ability to represent structural and semantic dependencies that are crucial for source code. The third is loss of contextual information: without autoregressive feedback, prediction at a position lacks access to previously generated output tokens (Yang et al., 2024, Yang et al., 2 Oct 2025).

These three deficits define the architecture. The repair action predictor addresses over-correction by explicitly modeling edit operations; the inter-token dependency extractor addresses missing structural dependence through AST-derived relations; and the two-stage decoder addresses missing context by re-decoding uncertain positions after an initial parallel pass (Yang et al., 2024, Yang et al., 2 Oct 2025). A common misconception is therefore that NARRepair is simply “parallel patch generation.” The model is instead described as a repair-specific non-autoregressive system whose auxiliary modules are intended to recover information that would otherwise be lost in parallel decoding (Yang et al., 2 Oct 2025).

3. Architecture and optimization

NARRepair follows a Transformer encoder-decoder architecture optimized for parallel code generation (Yang et al., 2 Oct 2025). The code encoder maps buggy code tokens to hidden representations, reported in one formulation as

$E_{i:n} = Encoder(W_{i:n} + W_{pos}).$

Its three principal modules can be summarized as follows.

Component	Reported purpose	Reported mechanism
Repair action predictor	Alleviate over-correction	Predicts keep, insert, delete, replace and repair length
Inter-token dependency extractor	Recover lost dependency information	Uses AST relationships and an attention mechanism akin to biaffine parsing
Two-stage decoder	Reintroduce contextual information	First-pass parallel generation followed by masked infilling of low-confidence positions

The repair action predictor is implemented as a convolutional neural network with classification layers over token embeddings (Yang et al., 2 Oct 2025). In the 2024 description, it is framed as reducing the output prediction space from vocabulary size to four actions, namely keep, replace, insert, and delete, together with repair length prediction (Yang et al., 2024). The corresponding losses are reported as

$$L_{length} = -\sum_{i}^{n} \log p_{length}(l_i \mid X, M),$$

$$L_{act} = -\sum_{i}^{n} \log p_{act}(a_i \mid X, N).$$

The inter-token dependency extractor derives pairwise token dependencies from the abstract syntax tree by identifying the nearest common AST parent for a token pair (Yang et al., 2024, Yang et al., 2 Oct 2025). The 2025 report gives the core projections as

$$Q = W^q \cdot D_{1:m}, \qquad K = W^k \cdot D_{1:m},$$

followed by

$M_{dependency} = Linear3(Q) \cdot K^T.$

This component is explicitly described as injecting AST-level dependency information back into the token representations used by the decoder (Yang et al., 2 Oct 2025).

The two-stage decoder first generates output tokens for all positions in parallel, then retains high-confidence outputs and masks low-confidence positions for a second decoding pass (Yang et al., 2024, Yang et al., 2 Oct 2025). The stated purpose is to recover context for uncertain positions “in spirit similar to BERT-style masked language modeling” in the 2024 description, and to approximate autoregressive context sensitivity in the 2025 description (Yang et al., 2024, Yang et al., 2 Oct 2025).

The overall objective is reported as

$$L = L_{dec} + \alpha (L_{act} + L_{length}) + \lambda L_{depend},$$

with $\alpha$ and $\lambda$ set to $y_t = f(x)$0 in the 2025 report (Yang et al., 2 Oct 2025). This loss structure makes the repair-action and dependency objectives auxiliary to the decoder objective rather than separate post hoc heuristics.

4. Training data and evaluation protocol

NARRepair is evaluated on three APR benchmarks widely used in the literature: Defects4J v1.2, Defects4J v2.0, and QuixBugs (Yang et al., 2024, Yang et al., 2 Oct 2025). The 2024 report lists Defects4J v1.2 with 395 bugs, Defects4J v2.0 with 420 bugs, and QuixBugs with 40 bugs, and states that training uses about 1.52 million examples in total from a mixture of self-supervised code error/fix pairs and real code (Yang et al., 2024). The 2025 report lists Defects4J v1.2 with 395 bugs, Defects4J v2.0 with 438 additional bugs, QuixBugs with 40 Java bugs, and SelfAPR data with approximately 1.5 million buggy/fixed code pairs for pretraining (Yang et al., 2 Oct 2025).

The later evaluation further reports knowledge distillation, using both original and distilled data, with the teacher generated via CodeT5-large (Yang et al., 2 Oct 2025). Patch validation is performed with ExpressAPR, described as accelerated test-based validation (Yang et al., 2 Oct 2025).

The baseline set also evolves across reports. The 2024 study compares against strong autoregressive APR models such as SequenceR, CoCoNuT, RewardRepair, Recoder, AlphaRepair, and Tare, and against non-autoregressive baselines such as ReorderNAT, SNAT, Fully-NAT, OAXE-NAT, and DePA (Yang et al., 2024). The 2025 study expands this to include ML-based autoregressive systems such as SequenceR, CoCoNuT, Recoder, RewardRepair, AlphaRepair, and TENURE, together with LLM-based models including Incoder-1B, CodeGen2-1B, ChatGPT, and ThinkRepair (Yang et al., 2 Oct 2025).

This protocol indicates that NARRepair is evaluated not only as a faster alternative to earlier seq2seq APR, but also as a compact repair model competing with larger autoregressive and LLM-based systems under explicit time constraints. A plausible implication is that the authors treat latency as a first-class evaluation dimension rather than a secondary engineering concern.

5. Reported empirical performance

In the 2024 report, under perfect fault localization, NARRepair is reported to correctly repair 69 bugs on Defects4J v1.2, 41 bugs on Defects4J v2.0, and 23 bugs on QuixBugs (Yang et al., 2024). These results are described as 90% of Tare on Defects4J v1.2, 95% of RewardRepair on Defects4J v2.0, and 82% of AlphaRepair on QuixBugs (Yang et al., 2024). The same report states that NARRepair substantially outperforms other non-autoregressive baselines and that, without perfect fault localization, it still outperforms other non-autoregressive approaches by more than 150% (Yang et al., 2024).

The 2025 report presents a time-constrained evaluation. Under perfect fault localization and a time limit of less than 10 minutes, the reported number of correctly repaired bugs is 74 on Defects4J v1.2, 66 on Defects4J v2.0, and 34 on QuixBugs for NARRepair, compared with 67, 61, and 31 for ThinkRepair-ChatGPT (Yang et al., 2 Oct 2025). Under unlimited time, the same table reports 78, 69, and 36 for NARRepair, and 98, 107, and 39 for ThinkRepair-ChatGPT (Yang et al., 2 Oct 2025). The speed-oriented claim is thus strongest in the limited-time regime rather than as an unconditional accuracy lead.

Latency results are reported relative to SequenceR. In the 2025 report, SequenceR is set to 1.0x, NARRepair to 0.71x, TENURE to 3.8x, CodeGen2-1B to 2.8x, InCoder-1B to 3.4x, ChatGPT to 5.8x, and ThinkRepair-ChatGPT to 6.4x (Yang et al., 2 Oct 2025). This is the basis for the claim that NARRepair is 1.4–6.4 times faster than autoregressive methods in the GPU environment (Yang et al., 2 Oct 2025). The 2024 report gives a similar but slightly stronger speed characterization for patch generation, reporting 5.4x acceleration on CPU and 6.2x on GPU relative to SequenceR, and 13–18x relative to slower but more accurate autoregressive models (Yang et al., 2024).

Ablation results identify all three architectural components as necessary. In the 2025 report, removing the repair action predictor reduces fixed bugs by 9, removing the inter-token dependency extractor reduces them by 7, and removing the two-stage decoder reduces them by 17 (Yang et al., 2 Oct 2025). The same report gives an over-correction analysis in which full NARRepair averages 2.2 wrongly modified correct tokens, compared with 3.1 without the repair action predictor and 5.3 without the two-stage decoder (Yang et al., 2 Oct 2025). The action and length predictors are reported to reach 82.6% and 79.8% accuracy, respectively (Yang et al., 2 Oct 2025). In the 2024 report, the repair action predictor is also described as the component whose removal causes the largest decline, dropping correct fixes on Defects4J v1.2 from 69 to 34 (Yang et al., 2024).

The later paper additionally reports that NARRepair has 520M parameters, smaller than LLM-based baselines in the 1B–7B range, while outperforming them under time-constrained conditions (Yang et al., 2 Oct 2025). This suggests that the reported speed gains are not reducible to trivial model downsizing alone.

6. Development, interpretation, and terminological scope

The published record presents NARRepair in at least two closely related forms. The 2024 paper, titled “NARRepair: Non-Autoregressive Code Generation Model for Automatic Program Repair” (Yang et al., 2024), introduces the architecture and emphasizes the feasibility of competitive non-autoregressive APR. The 2025 paper, “Towards Speeding up Program Repair with Non-Autoregressive Model” (Yang et al., 2 Oct 2025), retains the same core design and reframes the contribution around broader comparisons with autoregressive and LLM-based systems, knowledge distillation, and limited-time repair. Read together, these reports portray NARRepair as a stable model family rather than a one-off experimental prototype.

One interpretive point follows directly from the evidence: NARRepair should not be understood as claiming that non-autoregressive APR is universally superior to autoregressive APR. The reported literature instead argues that non-autoregressive repair can become practically competitive when it is specialized for edit locality, code structure, and uncertainty handling (Yang et al., 2024, Yang et al., 2 Oct 2025). Its strongest claims concern the speed–accuracy tradeoff and performance under limited repair time.

The term also requires disambiguation. In the 2025 paper “Neural Variable Name Repair: Learning to Rename Identifiers for Readability”, the phrase “neural variable name repair (NARRepair) problem” is used as a shorthand for identifier renaming in C++ functions (Yousuf et al., 30 Nov 2025). That usage refers to a distinct task involving masked local or parameter names, LoRA tuning on Llama 3.1–8B, and a dual-encoder reranker (Yousuf et al., 30 Nov 2025). In the APR literature, however, NARRepair denotes the non-autoregressive code generation model for automatic program repair introduced in 2024 and extended in 2025 (Yang et al., 2024, Yang et al., 2 Oct 2025).

Within APR research, NARRepair therefore occupies a specific place: it is the first customized non-autoregressive repair model, defined by action-guided editing, AST-conditioned dependency modeling, and two-stage contextual decoding, and evaluated primarily as a latency-conscious alternative to autoregressive repair systems (Yang et al., 2024, Yang et al., 2 Oct 2025).