---
title: 'AlignGuard: Silent Correctness Bug Detection'
url: https://www.emergentmind.com/topics/alignguard
type: topic
---

# AlignGuard: Silent Correctness Bug Detection

AlignGuard is a proof-of-concept testing technique for detecting **silent correctness bugs** in PyTorch’s compiler, `torch.compile`, where compiled deep learning models produce incorrect outputs without crashes, exceptions, or warnings [2604.08720]. It was introduced after an empirical study of `torch.compile` bug reports found that correctness bugs were both prevalent and undercharacterized, and it operationalizes that study through **LLM-based test mutation** over existing compiler tests. In the literature, the exact string **“AlignGuard”** also appears as a source of naming confusion with the distinct multimodal safety method **GuardAlign**; the latter paper explicitly states that its method is called **GuardAlign**, not AlignGuard [2602.24027].

## 1. Definition and problem scope

AlignGuard targets a specific failure mode of compiler-enabled deep learning systems: the compiled model executes normally yet produces **incorrect outputs**. The paper distinguishes these failures from crash bugs, exception bugs, and performance bugs, and treats them as especially dangerous because execution appears successful while downstream training or inference may already be corrupted [2604.08720].

The motivating context is contemporary AI infrastructure. The paper notes that `torch.compile` is a core optimization layer for deep learning models, including LLMs, and cites its use in settings such as **vLLM** and by industrial users such as **AWS**. It also gives an example in which an **OLMo LLM** failed to converge during training because of a `torch.compile` correctness bug. In the PyTorch issue data examined by the paper, **46.6% (138/296)** of high-priority issues collected between **April 19, 2024 and April 19, 2025** concerned `torch.compile`, and **41.3% (57/138)** of those were correctness bugs. In the paper’s summary framing, correctness bugs account for **19.2%** of all high-priority issues, the **second-most-common** category, behind crashes at **19.57%** [2604.08720].

This problem framing makes AlignGuard distinct from runtime LLM safety guardrails. Its object of protection is not dialogue behavior or agent action selection, but the **semantic correctness of compiled model execution**.

## 2. Empirical foundation and taxonomy of correctness bugs

The method is grounded in a two-year issue-study of `torch.compile`. From **2,542** candidate GitHub issues between **April 19, 2023 and April 19, 2025**, the paper filtered to **116 confirmed correctness bugs**. These were then organized into three main categories—**graph-related**, **operator-related**, and **memory-related**—with six subcategories and a small residual bucket for precision, configuration, and external-library bugs [2604.08720].

The taxonomy is central because AlignGuard does not mutate tests arbitrarily; it mutates them to match the bug-triggering patterns distilled from this study.

| Subcategory | Count | Primary bug-triggering pattern |
|---|---:|---|
| Graph Semantic Capturing Bugs | 18/116 | Non-computational operations; execution context mutation; corner case scenario |
| Graph Caching Bugs | 5/116 | Repeated execution with varying internal state or input properties |
| Operator Transformation Bugs | 21/116 | Non-default parameters; edge-case inputs; optimization-triggering operator sequence |
| Low-Level Code Generation Bugs | 23/116 | Extreme values for numerical sensitive operators; boundary scenarios |
| In-Place Operation Handling Bugs | 25/116 | In-place operations on tensor aliases/views |
| Memory Layout Conflicts | 14/116 | Optimization-driven or operator-induced layout transformations |

Graph-related bugs arise during graph capture or graph reuse. **Graph Semantic Capturing Bugs** include incomplete symbolic tracing for custom logic, aggressive optimization, and missing fallback handling for unsupported APIs. **Graph Caching Bugs** are tied to incorrect guard conditions that cause cached graphs to be reused across semantically different executions, especially in repeated runs with mutable internal state.

Operator-related bugs split into **Operator Transformation Bugs** and **Low-Level Code Generation Bugs**. The former involve semantic mismatches during decomposition or fusion; the latter reflect faulty generated C++ or Triton code, often under extreme numerical values or unhandled boundary conditions.

Memory-related bugs are dominated by **In-Place Operation Handling Bugs**, the largest subcategory at **25/116 (21.6%)**, and by **Memory Layout Conflicts**. A recurring pattern is that merely exercising an in-place operator is insufficient; the critical trigger is often an alias- or view-producing structure followed by mutation, or a layout-sensitive operator sequence [2604.08720].

## 3. AlignGuard workflow

AlignGuard begins from **existing PyTorch compiler tests** rather than generating programs from scratch. The seeds were extracted from PyTorch **2.8.0**, specifically `test/inductor/test_torchinductor.py`, where **475** raw tests were reduced to **341 valid seed test cases** after filtering. The paper also verified that these seeds did not already trigger correctness bugs in the latest PyTorch version [2604.08720].

The method standardizes each seed test into four components:

1. **Model Structure**
2. **Input Data**
3. **Compilation Arguments**
4. **Testing Pipeline**

From there, AlignGuard proceeds in three stages. First, it performs **bug-triggering pattern extraction** from historical issue reports and fixing commits, using the bug type, triggering-pattern category, and root-cause category as structured guidance. Second, it performs **mutation rule synthesis**, generating rules in a `<condition, action>` format. Third, it applies **LLM-based test mutation** to the seed tests under constraints intended to preserve executability and avoid unrelated syntax or semantic failures.

The paper uses **`gpt-4o`** for pattern extraction and mutation-rule synthesis, and **`gpt-4o-mini`** for test-case mutation, with temperature **0.2**. It describes the extracted pattern representation in terms of fields such as `"Model Structures"`, `"Input Features"`, `"Compilation Arguments"`, `"Testing Pipeline"`, and `"Pattern Rationale"` [2604.08720].

Bug detection itself relies on the assertions already present in the mutated tests, including standard checks such as `torch.testing.assert_close`. For the historical benchmark, the paper defines detection using patch-based validation:

\[
\exists~S_F \in S_F^{all}: \text{Error}(S_F,V_B) \land \neg \text{Error}(S_F,V_F)
\]

where \(V_B\) is the buggy version, \(V_F\) is the fixed version, and \(S_F^{all}\) is the set of bug-revealing tests produced by technique \(F\). This criterion is necessary because silent correctness bugs do not reliably produce overt failure signals [2604.08720].

## 4. Comparison with prior testing techniques

To evaluate existing approaches, the paper built a reproducible benchmark of **77** correctness bugs from the original **116** after excluding issues without reproducible code, clear patches, release-version reproducibility, or standard CPU/GPU feasibility. On this benchmark, five prior techniques collectively detected only **26/77 = 33.8%** of the bugs [2604.08720].

| Technique | Bugs detected |
|---|---:|
| NNSmith | 4 |
| NeuRI | 12 |
| Opera | 4 |
| WhiteFox | 12 |
| DeepConstr | 6 |

A central empirical result is that these techniques detected **0 graph-related bugs**. Among the **26** historical bugs they did detect, **17** were operator-related and **6** were memory-related. The paper attributes this weakness to several factors: single-run differential testing misses repeated-execution graph bugs; generated programs emphasize computational operators but neglect non-computational APIs and execution-context mutation; and existing generators underproduce the specific alias-heavy, layout-sensitive, or optimization-triggering patterns that characterize many `torch.compile` correctness failures [2604.08720].

AlignGuard is explicitly designed around these failure modes. Rather than treating operator coverage as sufficient, it tries to reproduce the semantic structures that actually triggered past bugs: non-computational operations, repeated-execution pipelines, non-default parameters, extreme values, alias-producing views, and layout-transforming sequences.

## 5. Reported discoveries and representative bugs

Applied to recent `torch.compile`, AlignGuard detected **23 new correctness bugs**. According to the paper, **23/23** were confirmed by the PyTorch development team, **10/23** were fixed at the time of writing, and **14/23** were marked high-priority. The breakdown was **12 memory-related**, **7 operator-related**, **3 graph-related**, and **1 precision** bug [2604.08720].

The paper presents representative cases. In one **memory-related** bug, a generated test created a view with `torch.cat([x, y], dim=1)` and then performed an in-place indexed assignment, causing incorrect behavior because aliasing relationships were lost during slicing and `index_put`. In one **operator-related** bug, broadcast addition between a complex tensor and a real tensor triggered a constant-folding optimization that mishandled the complex dtype corner case, yielding values such as `2.+1.j` instead of `2.+0.j`. In one **graph-related** bug, the configuration `torch._inductor.config.fallback_random = True` combined with `torch.manual_seed(0); return torch.randn_like(x)` caused inconsistent RNG behavior between eager and compiled execution for a non-contiguous tensor of size at least 16 [2604.08720].

These examples illustrate the paper’s main thesis: the bug-triggering structures are often semantically narrow and difficult to reach with generic graph generation or untargeted mutation.

## 6. Limitations and significance

The paper explicitly notes several limitations. The empirical study required manual filtering and categorization, so subjectivity in issue analysis remains a validity threat. The dataset is limited to the **116** correctness bugs collected from the PyTorch issue tracker, and the benchmark covers only correctness bugs in `torch.compile`, not crash bugs or broader compiler quality. The paper also does not report ablation studies isolating the contribution of empirical labels, LLM mutation, seed selection, or oracle design. In the provided text, open-source availability of AlignGuard itself is not stated [2604.08720].

Even with those limits, the method’s significance lies in the combination of **systematic bug characterization** and **knowledge-guided test mutation**. AlignGuard treats correctness-bug detection not as unconstrained fuzzing, but as a search problem informed by an empirical model of how `torch.compile` actually fails. This suggests a broader methodological lesson: for silent compiler miscompilation, bug discovery can benefit from explicitly encoding **historical trigger patterns** rather than optimizing only for generic operator diversity or syntactic validity.

In that sense, AlignGuard occupies a specific place in the literature. It is not a general compiler verifier, nor a runtime model-safety guardrail, but a targeted testing technique for one of the most operationally consequential failure modes in modern deep learning infrastructure: **incorrect outputs without visible failure** [2604.08720].

Source: https://www.emergentmind.com/topics/alignguard