---
title: 'NARRepair: Non-Autoregressive Program Repair'
url: https://www.emergentmind.com/topics/narrepair
type: topic
---

# NARRepair: Non-Autoregressive Program Repair

Searching arXiv for the cited NARRepair papers to ground the article in the current literature.
NARRepair is a non-autoregressive code generation model for automatic program repair (APR) that adapts parallel decoding to bug fixing through three coordinated mechanisms: a repair action predictor, an inter-token dependency extractor, and a two-stage decoder [2406.16526][2510.01825]. It was introduced as the first customized non-autoregressive model for APR, with the explicit objective of reducing the inference latency associated with autoregressive repair while maintaining high repair accuracy [2406.16526]. In later reporting, it is characterized as achieving state-of-the-art comprehensive performance in terms of repair speed and accuracy, especially under limited repair time [2510.01825].

## 1. Concept and research setting

NARRepair emerged from a line of APR research in which deep learning-based systems predominantly generated patches in an autoregressive manner. In that setting, future tokens are predicted from past tokens, and code is generated token by token. The reported motivation for NARRepair is that this mode of generation creates a substantial time delay, which becomes particularly serious for APR models with a large number of parameters [2406.16526][2510.01825].

The model is therefore positioned against a specific bottleneck in neural APR: inference latency rather than only patch quality. In the 2024 report, this motivation is framed in terms of “real-time program repair,” especially for domains such as embedded systems, robotics, and large-scale software development [2406.16526]. The 2025 report retains the same core motivation but emphasizes “best performance within the limited repair time” and compares NARRepair directly against both ML-based autoregressive APR models and LLM-based repair systems [2510.01825].

A central point in the NARRepair literature is that APR is not treated as generic sequence transduction. The model is presented as a customized non-autoregressive repair architecture rather than a direct transfer of non-autoregressive machine translation methods to code repair [2406.16526][2510.01825]. This distinction is important because the reported failure modes of naïve non-autoregressive repair are code-specific and repair-specific rather than merely sequence-modeling artifacts.

## 2. Design rationale and problem formulation

The formal contrast between autoregressive and non-autoregressive generation is stated as follows in the 2024 description: an autoregressive model predicts according to $y_t = f(y_{1:t-1}, x)$, whereas a non-autoregressive model predicts all positions in parallel as $y_t = f(x)$ [2406.16526]. NARRepair adopts the latter regime in order to avoid the sequential dependency that slows conventional APR.

The model is built around three limitations that arise when non-autoregressive generation is applied naïvely to APR. The first is **over-correction**: unlike machine translation, where all source tokens are typically transformed, APR usually requires keeping most program tokens unchanged. The second is **loss of inter-token dependency information**: parallel decoding weakens the ability to represent structural and semantic dependencies that are crucial for source code. The third is **loss of contextual information**: without autoregressive feedback, prediction at a position lacks access to previously generated output tokens [2406.16526][2510.01825].

These three deficits define the architecture. The repair action predictor addresses over-correction by explicitly modeling edit operations; the inter-token dependency extractor addresses missing structural dependence through AST-derived relations; and the two-stage decoder addresses missing context by re-decoding uncertain positions after an initial parallel pass [2406.16526][2510.01825]. A common misconception is therefore that NARRepair is simply “parallel patch generation.” The model is instead described as a repair-specific non-autoregressive system whose auxiliary modules are intended to recover information that would otherwise be lost in parallel decoding [2510.01825].

## 3. Architecture and optimization

NARRepair follows a Transformer encoder-decoder architecture optimized for parallel code generation [2510.01825]. The code encoder maps buggy code tokens to hidden representations, reported in one formulation as
$$
E_{i:n} = Encoder(W_{i:n} + W_{pos}).
$$

Its three principal modules can be summarized as follows.

| Component | Reported purpose | Reported mechanism |
|---|---|---|
| Repair action predictor | Alleviate over-correction | Predicts `keep`, `insert`, `delete`, `replace` and repair length |
| Inter-token dependency extractor | Recover lost dependency information | Uses AST relationships and an attention mechanism akin to biaffine parsing |
| Two-stage decoder | Reintroduce contextual information | First-pass parallel generation followed by masked infilling of low-confidence positions |

The **repair action predictor** is implemented as a convolutional neural network with classification layers over token embeddings [2510.01825]. In the 2024 description, it is framed as reducing the output prediction space from vocabulary size to four actions, namely `keep`, `replace`, `insert`, and `delete`, together with repair length prediction [2406.16526]. The corresponding losses are reported as
$$
L_{length} = -\sum_{i}^{n} \log p_{length}(l_i \mid X, M),
$$
$$
L_{act} = -\sum_{i}^{n} \log p_{act}(a_i \mid X, N).
$$

The **inter-token dependency extractor** derives pairwise token dependencies from the abstract syntax tree by identifying the nearest common AST parent for a token pair [2406.16526][2510.01825]. The 2025 report gives the core projections as
$$
Q = W^q \cdot D_{1:m}, \qquad K = W^k \cdot D_{1:m},
$$
followed by
$$
M_{dependency} = Linear3(Q) \cdot K^T.
$$
This component is explicitly described as injecting AST-level dependency information back into the token representations used by the decoder [2510.01825].

The **two-stage decoder** first generates output tokens for all positions in parallel, then retains high-confidence outputs and masks low-confidence positions for a second decoding pass [2406.16526][2510.01825]. The stated purpose is to recover context for uncertain positions “in spirit similar to BERT-style masked language modeling” in the 2024 description, and to approximate autoregressive context sensitivity in the 2025 description [2406.16526][2510.01825].

The overall objective is reported as
$$
L = L_{dec} + \alpha (L_{act} + L_{length}) + \lambda L_{depend},
$$
with $\alpha$ and $\lambda$ set to $0.1$ in the 2025 report [2510.01825]. This loss structure makes the repair-action and dependency objectives auxiliary to the decoder objective rather than separate post hoc heuristics.

## 4. Training data and evaluation protocol

NARRepair is evaluated on three APR benchmarks widely used in the literature: **Defects4J v1.2**, **Defects4J v2.0**, and **QuixBugs** [2406.16526][2510.01825]. The 2024 report lists Defects4J v1.2 with 395 bugs, Defects4J v2.0 with 420 bugs, and QuixBugs with 40 bugs, and states that training uses about 1.52 million examples in total from a mixture of self-supervised code error/fix pairs and real code [2406.16526]. The 2025 report lists Defects4J v1.2 with 395 bugs, Defects4J v2.0 with 438 additional bugs, QuixBugs with 40 Java bugs, and **SelfAPR data** with approximately 1.5 million buggy/fixed code pairs for pretraining [2510.01825].

The later evaluation further reports knowledge distillation, using both original and distilled data, with the teacher generated via CodeT5-large [2510.01825]. Patch validation is performed with **ExpressAPR**, described as accelerated test-based validation [2510.01825].

The baseline set also evolves across reports. The 2024 study compares against strong autoregressive APR models such as SequenceR, CoCoNuT, RewardRepair, Recoder, AlphaRepair, and Tare, and against non-autoregressive baselines such as ReorderNAT, SNAT, Fully-NAT, OAXE-NAT, and DePA [2406.16526]. The 2025 study expands this to include ML-based autoregressive systems such as SequenceR, CoCoNuT, Recoder, RewardRepair, AlphaRepair, and TENURE, together with LLM-based models including Incoder-1B, CodeGen2-1B, ChatGPT, and ThinkRepair [2510.01825].

This protocol indicates that NARRepair is evaluated not only as a faster alternative to earlier seq2seq APR, but also as a compact repair model competing with larger autoregressive and LLM-based systems under explicit time constraints. A plausible implication is that the authors treat latency as a first-class evaluation dimension rather than a secondary engineering concern.

## 5. Reported empirical performance

In the 2024 report, under perfect fault localization, NARRepair is reported to correctly repair 69 bugs on Defects4J v1.2, 41 bugs on Defects4J v2.0, and 23 bugs on QuixBugs [2406.16526]. These results are described as 90% of Tare on Defects4J v1.2, 95% of RewardRepair on Defects4J v2.0, and 82% of AlphaRepair on QuixBugs [2406.16526]. The same report states that NARRepair substantially outperforms other non-autoregressive baselines and that, without perfect fault localization, it still outperforms other non-autoregressive approaches by more than 150% [2406.16526].

The 2025 report presents a time-constrained evaluation. Under perfect fault localization and a time limit of **less than 10 minutes**, the reported number of correctly repaired bugs is **74** on Defects4J v1.2, **66** on Defects4J v2.0, and **34** on QuixBugs for NARRepair, compared with **67**, **61**, and **31** for ThinkRepair-ChatGPT [2510.01825]. Under **unlimited** time, the same table reports **78**, **69**, and **36** for NARRepair, and **98**, **107**, and **39** for ThinkRepair-ChatGPT [2510.01825]. The speed-oriented claim is thus strongest in the limited-time regime rather than as an unconditional accuracy lead.

Latency results are reported relative to **SequenceR**. In the 2025 report, SequenceR is set to **1.0x**, NARRepair to **0.71x**, TENURE to **3.8x**, CodeGen2-1B to **2.8x**, InCoder-1B to **3.4x**, ChatGPT to **5.8x**, and ThinkRepair-ChatGPT to **6.4x** [2510.01825]. This is the basis for the claim that NARRepair is **1.4–6.4 times faster** than autoregressive methods in the GPU environment [2510.01825]. The 2024 report gives a similar but slightly stronger speed characterization for patch generation, reporting **5.4x** acceleration on CPU and **6.2x** on GPU relative to SequenceR, and **13–18x** relative to slower but more accurate autoregressive models [2406.16526].

Ablation results identify all three architectural components as necessary. In the 2025 report, removing the repair action predictor reduces fixed bugs by **9**, removing the inter-token dependency extractor reduces them by **7**, and removing the two-stage decoder reduces them by **17** [2510.01825]. The same report gives an over-correction analysis in which full NARRepair averages **2.2** wrongly modified correct tokens, compared with **3.1** without the repair action predictor and **5.3** without the two-stage decoder [2510.01825]. The action and length predictors are reported to reach **82.6%** and **79.8%** accuracy, respectively [2510.01825]. In the 2024 report, the repair action predictor is also described as the component whose removal causes the largest decline, dropping correct fixes on Defects4J v1.2 from **69** to **34** [2406.16526].

The later paper additionally reports that NARRepair has **520M parameters**, smaller than LLM-based baselines in the **1B–7B** range, while outperforming them under time-constrained conditions [2510.01825]. This suggests that the reported speed gains are not reducible to trivial model downsizing alone.

## 6. Development, interpretation, and terminological scope

The published record presents NARRepair in at least two closely related forms. The 2024 paper, titled **“NARRepair: Non-Autoregressive Code Generation Model for Automatic Program Repair”** [2406.16526], introduces the architecture and emphasizes the feasibility of competitive non-autoregressive APR. The 2025 paper, **“Towards Speeding up Program Repair with Non-Autoregressive Model”** [2510.01825], retains the same core design and reframes the contribution around broader comparisons with autoregressive and LLM-based systems, knowledge distillation, and limited-time repair. Read together, these reports portray NARRepair as a stable model family rather than a one-off experimental prototype.

One interpretive point follows directly from the evidence: NARRepair should not be understood as claiming that non-autoregressive APR is universally superior to autoregressive APR. The reported literature instead argues that non-autoregressive repair can become practically competitive when it is specialized for edit locality, code structure, and uncertainty handling [2406.16526][2510.01825]. Its strongest claims concern the speed–accuracy tradeoff and performance under limited repair time.

The term also requires disambiguation. In the 2025 paper **“Neural Variable Name Repair: Learning to Rename Identifiers for Readability”**, the phrase “neural variable name repair (NARRepair) problem” is used as a shorthand for identifier renaming in C++ functions [2512.01141]. That usage refers to a distinct task involving masked local or parameter names, LoRA tuning on Llama 3.1–8B, and a dual-encoder reranker [2512.01141]. In the APR literature, however, **NARRepair** denotes the non-autoregressive code generation model for automatic program repair introduced in 2024 and extended in 2025 [2406.16526][2510.01825].

Within APR research, NARRepair therefore occupies a specific place: it is the first customized non-autoregressive repair model, defined by action-guided editing, AST-conditioned dependency modeling, and two-stage contextual decoding, and evaluated primarily as a latency-conscious alternative to autoregressive repair systems [2406.16526][2510.01825].

Source: https://www.emergentmind.com/topics/narrepair