Multi-task LLM for Bug Localization (MLC)

• The paper introduces a two-channel framework that combines a frozen causal language model with an auxiliary line-wise BUG/NO BUG classifier for precise bug ranking.
• It leverages an efficient token–line alignment algorithm and parallel scoring to handle full-file context with sub-second inference latency.
• Empirical results demonstrate state-of-the-art line-level performance and significant speedups over iterative, agentic debugging approaches.

Searching arXiv for the focal and related bug-localization papers to ground the article with current citations. arxiv_search(query="(Rozanov, 8 Jun 2026)", max_results=5) arxiv_search(query="(Rozanov, 8 Jun 2026) Multi-task LLMs for Bug Classification: Efficient Inference with Auxiliary Decoding Heads", max_results=5) Multi-task LLM for Bug Localization (MLC) is a line-level fault-localization framework that adapts a frozen code LLM into a two-channel system combining the original causal language-model backbone with an auxiliary line-wise BUG/NO BUG classifier. The method is designed for full-file context, precise line-level ranking, and low-latency inference: it predicts bug probabilities for all lines in parallel in a single forward pass and requires only one generated token per file at test time. Its central technical components are a shallow decoding head over per-line aggregated hidden states, a two-pointer token–line alignment algorithm that maps source lines to tokenizer spans, and a weighted line-wise binary cross-entropy objective trained over entire files (Rozanov, 8 Jun 2026).

1. Conceptual scope and problem setting

MLC addresses bug localization at line granularity rather than at file or function granularity. In the formulation reported for the method, the input is an entire source file tokenized up to a maximum sequence length, and the output is a ranking of source-code lines by predicted probability of being buggy. This design explicitly targets a setting in which coarse function-level localization is considered insufficient for precise debugging, while agentic approaches are considered expensive because they require iterative reasoning, long latency, and many generated tokens per file (Rozanov, 8 Jun 2026).

The framework is described as a multi-task LLM because it preserves the original next-token language-model channel while adding a second task channel for line-wise classification. In practice, however, the backbone and original LM head remain frozen, so optimization is concentrated on the classification pathway. The resulting architecture is therefore not a generative debugging agent in the style of multi-step project exploration, but a discriminative classifier layered on top of a causal code model (Rozanov, 8 Jun 2026).

This suggests a specific interpretation of “multi-task” in MLC: the term refers less to multiple externally orchestrated subtasks and more to shared hidden representations supporting both language modeling and line-level bug discrimination. A plausible implication is that MLC occupies a different design point from agentic project-level systems and hierarchical coarse-to-fine systems: it trades explicit search procedures for direct per-line scoring under full-file context.

2. Architecture and auxiliary decoding head

MLC starts from a frozen causal-LM code model and appends an optional LoRA adapter together with a shallow classification head. The paper lists candidate backbones including Qwen-2.5-Coder-7B, Qwen3-1.7B/4B/8B, and CodeGen-2/6/16B. Let the tokenized file be $X = [x_1,\dots,x_n]$. The pretrained backbone maps this sequence to hidden states

$H = [h_1,\dots,h_n] \in \mathbb{R}^{n \times d}.$

All backbone parameters are frozen, while optional LoRA adapters of rank $r=32$ permit lightweight adaptation (Rozanov, 8 Jun 2026).

The core architectural step is aggregation from token-level hidden states to line-level representations. If $\mathcal{I}_\ell$ denotes the set of token indices aligned to line $\ell$, MLC evaluates three aggregation operators:

$$h_\ell^{\text{agg}} = \sum_{i \in \mathcal{I}_\ell} h_i,$$

$$h_\ell^{\text{agg}} = \frac{1}{|\mathcal{I}_\ell|}\sum_{i \in \mathcal{I}_\ell} h_i,$$

and

$h_\ell^{\text{agg}} = h_{\max(\mathcal{I}_\ell)}.$

The reported experiments state that “mean” and “last-token” both work well (Rozanov, 8 Jun 2026).

Each line embedding is then passed through a feed-forward layer of size $d \rightarrow d$ with ReLU activation, followed by a final linear projection to two logits and a sigmoid or softmax:

$$s_\ell = \mathrm{ReLU}(W_1 h_\ell^{\text{agg}} + b_1),$$

$H = [h_1,\dots,h_n] \in \mathbb{R}^{n \times d}.$0

$H = [h_1,\dots,h_n] \in \mathbb{R}^{n \times d}.$1

Here $H = [h_1,\dots,h_n] \in \mathbb{R}^{n \times d}.$2, with $H = [h_1,\dots,h_n] \in \mathbb{R}^{n \times d}.$3 and $H = [h_1,\dots,h_n] \in \mathbb{R}^{n \times d}.$4. The added parameter count is approximately $H = [h_1,\dots,h_n] \in \mathbb{R}^{n \times d}.$5, plus the optional LoRA adapter of approximately $H = [h_1,\dots,h_n] \in \mathbb{R}^{n \times d}.$6 (Rozanov, 8 Jun 2026).

The architectural significance of this design is that the auxiliary head reuses the backbone’s contextual states instead of invoking iterative reasoning over candidate snippets. By predicting all lines in parallel from shared full-file representations, MLC converts bug localization into a dense classification problem over line embeddings.

3. Token–line alignment and the multi-task objective

A technical obstacle for line-level classification is that standard LLM tokenizers do not align newline characters to distinct tokens. MLC therefore introduces a token–line alignment procedure based on tokenizer offset mappings. If $H = [h_1,\dots,h_n] \in \mathbb{R}^{n \times d}.$7 gives the character span of token $H = [h_1,\dots,h_n] \in \mathbb{R}^{n \times d}.$8, and if line-start indices are $H = [h_1,\dots,h_n] \in \mathbb{R}^{n \times d}.$9, then the goal is to construct

$r=32$0

The paper’s two-pointer algorithm runs in $r=32$1 time and assigns each token to exactly one line (Rozanov, 8 Jun 2026).

Operationally, the algorithm maintains a token pointer and scans the token spans against consecutive line boundaries. This is the mechanism that permits the model to train and predict at line granularity while still using an ordinary subword tokenizer over the entire file. The method is reported to handle multi-token lines and overlapping spans (Rozanov, 8 Jun 2026).

The training objective formalizes MLC as a two-task system:

• Task $r=32$2: original next-token LM, frozen and not updated.
• Task $r=32$3: line-wise BUG/NO BUG classification, trained through the auxiliary head.

Let $r=32$4 be the ground-truth label for line $r=32$5, and let $r=32$6 denote the predicted probability that the line is buggy. The classification objective is a weighted binary cross-entropy over all $r=32$7 lines in a file:

$r=32$8

The selected class weights are $r=32$9 and $\mathcal{I}_\ell$0 in order to counter line imbalance (Rozanov, 8 Jun 2026).

The general multi-task objective is written as

$\mathcal{I}_\ell$1

but the reported implementation sets $\mathcal{I}_\ell$2 and $\mathcal{I}_\ell$3 because the backbone and original LM head are frozen (Rozanov, 8 Jun 2026). This makes the practical training regime a single-task optimization over a multi-task architecture.

4. Training configuration and inference characteristics

The reported training datasets and splits are:

• Defects4J (Java, v1.2): 1,499 train / 187 val / 188 test files.
• PypiBugs (Python): 288 / 19 / 78.
• BugsEval (Python OOD): 67 files, held out for generalization (Rozanov, 8 Jun 2026).

Preprocessing uses full-file context up to 8,000 tokens, or 2,048 for the CodeGen family, followed by token–line alignment. No additional data augmentation is used beyond class weighting. The selected hyperparameters are AdamW with learning rate $\mathcal{I}_\ell$4, batch size 1 file, one decoding layer, mean aggregation, sigmoid activation, and LoRA rank/$\mathcal{I}_\ell$5 when LoRA is enabled. Training runs on 48 GB L40S GPUs in BF16, or 4-bit for LoRA (Rozanov, 8 Jun 2026).

At inference time, MLC tokenizes a source file, computes hidden states with the frozen LLM, aligns tokens to lines, aggregates per-line hidden states, applies the classification head, and sorts lines by predicted bug probability. The procedure is explicitly non-iterative: no per-line generation and no function-by-function reasoning are performed (Rozanov, 8 Jun 2026).

The reported latency is approximately a single forward pass plus one token-generation overhead, under $\mathcal{I}_\ell$6 seconds on an L40S, with a more specific value of approximately $\mathcal{I}_\ell$7 seconds per file in the experiments. This is contrasted with agentic pipelines reported at 60–120 seconds per file (Rozanov, 8 Jun 2026). The efficiency claim is central to MLC’s identity: its design objective is not only localization accuracy, but also drastic reduction in inference-time overhead.

5. Empirical performance and ablations

The experimental results are reported at line level for Defects4J and PypiBugs, and on an out-of-domain Python evaluation using BugsEval. On Defects4J with full-file 8K context, MLC Qwen3-1.7B + LoRA achieves Top-1 16.3%, Top-3 23.3%, and Top-5 39.5%. On the same benchmark, MLC Qwen2.5-Coder-7B reports 9.3%, 18.6%, and 27.9%, while MLC Qwen3-8B reports 7.0%, 20.1%, and 34.9% (Rozanov, 8 Jun 2026).

On PypiBugs, MLC Qwen3-8B reports Top-1 10.3%, Top-3 26.3%, and Top-5 36.6%, while MLC Qwen3-1.7B + LoRA reports 8.6%, 27.4%, and 37.1%. The paper contrasts these results with prompt-based supervised fine-tuning of Qwen2.5, which reports 2.0%, 10.0%, and 14.0% on PypiBugs, and 0.0%, 7.4%, and 14.3% on Defects4J (Rozanov, 8 Jun 2026).

For shorter-context comparisons on Defects4J under function-level or 2K short-context conditions, the paper reports:

• AgentFL (gpt-3.5): Top-1 39.7%, Top-5 47.3%.
• FlexFL (gpt-4): Top-1 49.9%, Top-5 66.9%.
• MemFL (gpt-4.1): Top-1 50.9%, Top-5 73.9%.
• MLC CG-16B (2K): Top-1 28.6%, Top-5 71.4% (Rozanov, 8 Jun 2026).

These figures indicate that MLC is evaluated against both line-level peers and coarser-grained agentic systems. The paper characterizes MLC as achieving state-of-the-art line-level performance among similar full-file, long-context setups while reaching comparable performance to agentic approaches on Defects4J and PypiBugs with orders-of-magnitude lower latency (Rozanov, 8 Jun 2026).

The out-of-domain evaluation trains on PypiBugs and tests on BugsEval. Reported Top-1 scores are 13.8% for MLC Qwen2.5, 13.8% for MLC Qwen3-4B, 13.8% for MLC Qwen3-8B, and 9.2% for MLC Qwen3-1.7B + LoRA, with Top-5 values ranging from 24.6% to 29.2% (Rozanov, 8 Jun 2026). This is presented as evidence of strong out-of-domain generalization.

The ablation results on Qwen3-1.7B + PEFT on Defects4J report that one decoding layer improves Top-5 by 4.6% relative to zero layers, that last-token and mean aggregation are approximately best while sum is weaker, and that sigmoid outperforms softmax by 5 percentage points on Top-5 (Rozanov, 8 Jun 2026). The paper interprets the activation result as suggesting that independent-probability modeling handles line ambiguity better.

6. Position within bug-localization research

MLC belongs to a broader line of LLM-based bug-localization systems, but its operating assumptions differ substantially from project-level multi-agent search and hierarchical multi-granularity pruning. AgentFL models project-level fault localization as a three-step process—fault comprehension, codebase navigation, and fault confirmation—implemented by multiple specialized agents using strategies such as Test Behavior Tracking, Document-Guided Search, and Multi-Round Dialogue. On Defects4J-V1.2.0, AgentFL localizes 157 out of 395 bugs within Top-1 and reports an average cost of 0.074 dollars and 97 seconds per bug (Qin et al., 2024). In contrast, MLC avoids agentic decomposition and replaces multi-round reasoning with one-pass line-wise scoring (Rozanov, 8 Jun 2026).

BLAZE represents another contrasting direction. It addresses cross-language and cross-project bug localization through dynamic chunking, hard example learning, and hybrid retrieval over a GPT-style dual encoder, evaluated on the BEETLEBOX dataset spanning 26,321 bugs across 29 projects and five languages. Its focus is retrieval and ranking of code chunks or files under cross-project and cross-language settings rather than line-wise full-file classification (Chakraborty et al., 2024). A plausible implication is that MLC and BLAZE target different failure modes of LLM-based localization: one emphasizes precise line ranking under full-file context, while the other emphasizes retrieval robustness under corpus scale and heterogeneity.

BugCerberus offers a hierarchical comparison point. It organizes localization into file, function, and statement levels using three separately fine-tuned Llama-3 models and static-analysis-derived contexts such as sub-directory trees, call chains, and program slices. Evaluated on SWE-bench-lite, it reports statement-level Top-1 0.268, Top-3 0.373, Top-5 0.401, and Top-10 0.453, and improves the fix count of Agentless from 35 to 42 out of 300 issues (Chang et al., 21 Feb 2025). Relative to BugCerberus, MLC does not perform coarse-to-fine pruning; instead, it scores lines directly from full-file hidden states (Rozanov, 8 Jun 2026).

The main limitations stated for MLC are that it addresses only binary per-line classification, that function-level or project-level localization would require extending the alignment algorithm to coarser code constructs, and that larger backbones would likely yield higher accuracy at greater cost (Rozanov, 8 Jun 2026). These limitations define its current research position: MLC is an efficient long-context line localizer rather than a complete project-level diagnostic system or an end-to-end repair framework.

7. Significance and prospective extensions

The principal significance of MLC lies in the combination of line-level precision, full-file context, and extremely low inference overhead. By sharing backbone representations and applying an auxiliary decoding head over aligned line spans, it converts bug localization into parallel classification over source lines, thereby avoiding iterative chain-of-thought generation and candidate-by-candidate analysis (Rozanov, 8 Jun 2026).

Several extensions are explicitly suggested in the source. The paper notes that multi-span localization or repair could be appended as further decoding heads; that function-level or project-level localization would require extending the alignment algorithm to coarser constructs; and that larger backbones would likely improve accuracy at increased cost (Rozanov, 8 Jun 2026). This suggests a broader research program in which MLC’s decoding-head architecture serves as a lightweight substrate for richer debugging tasks, potentially complementing retrieval systems, hierarchical pruning systems, or repair pipelines rather than replacing them outright.