Compiler-as-a-Review Module (CRM)
- CRM is a compiler-backed review module that validates CAD parametric sequences by compiling them and returning precise error diagnostics.
- It operates as the review node in an agentic generation-review loop, ensuring only valid outputs through iterative refinement.
- CRM enhances synthesis efficiency by reducing the Invalidity Ratio and improving geometric and syntactic fidelity with minimal computational overhead.
Compiler-as-a-Review Module (CRM) denotes a compiler-backed review component introduced in CAD-Judge for efficient, automated verification of LLM-generated CAD outputs at test/inference time (Zhou et al., 6 Aug 2025). In text-to-CAD systems, LLMs generate sequences of parametric CAD commands, but these sequences often contain syntactic or geometric errors that cause compilation failures and yield invalid or unusable CAD models. CRM addresses this by compiling the generated sequence with a local CAD compiler, accepting compilable outputs and returning compiler diagnostics when compilation fails, thereby supporting iterative refinement without relying on rendering-heavy or VLM-based review (Zhou et al., 6 Aug 2025).
1. Conceptual role and motivation
Within CAD-Judge, CRM functions as the review node in an agentic generation-review loop, whereas the Compiler-as-a-Judge Module (CJM) is adopted as a fast, direct reward signal for alignment during training (Zhou et al., 6 Aug 2025). The motivation is operational rather than purely architectural: rendering CAD models can be slow, and deploying VLMs to review CAD models can be expensive and may introduce reward hacking that degrades the systems. CRM therefore re-centers review around compilation, using the local CAD toolchain itself as the validator.
The module is defined by its inference-time role. It does not depend on ground-truth data or user reference models at test time; it uses only the generated sequence and the compiler’s response (Zhou et al., 6 Aug 2025). This property is central to its positioning in CAD-Judge: CRM is not a dataset annotator, a reward model for training, or a human-in-the-loop adjudicator, but a lightweight verifier that enforces hard syntactic and geometric constraints during deployment.
A plausible implication is that CRM formalizes a narrow but practically important notion of “review”: rather than ranking outputs by subjective quality, it checks whether an output is compilable and, if not, exposes the compiler’s failure mode in a form that can be reused for correction. In CAD-Judge, this makes review synonymous with verifiable executability.
2. Placement in the agentic generation-review loop
CRM operates inside what CAD-Judge calls agentic CAD generation (Zhou et al., 6 Aug 2025). The workflow is iterative and bounded.
Generation: the LLM, acting as the generation node, produces a candidate CAD parametric sequence in response to a user prompt.
Review/validation: CRM, acting as the review node, receives the generated sequence and attempts to compile it into a 3D CAD model using a local CAD compiler.
Error feedback: if the sequence successfully compiles, the model is accepted and returned as output. If compilation fails, CRM collects the specific error diagnostics from the CAD compiler.
Iterative refinement: the error information is appended to, or incorporated with, the original prompt and resubmitted to the LLM for further refinement. The loop repeats until a valid sequence is produced or a preset maximum number of iterations is reached (Zhou et al., 6 Aug 2025).
This placement gives CRM a dual function. First, it is a gatekeeper: only compilable outputs are admitted. Second, it is a feedback generator: failed outputs are not merely rejected, but converted into structured compiler-derived evidence that conditions the next sample. In CAD-Judge, the default configuration typically uses only one repair attempt because additional rerolls exhibit diminishing returns (Zhou et al., 6 Aug 2025).
3. Validation substrate and error classes
Technically, CRM parses the generated sequence into a CAD modeling script, typically in a sketch-and-extrude parametric format, and invokes a local CAD compiler such as pythonOCC to attempt model construction (Zhou et al., 6 Aug 2025). If compilation succeeds, the resulting model can be tessellated and processed further. If compilation fails, CRM records the exact error message and returns that diagnostic to the LLM for the next attempt.
The error space covered by CRM spans both syntax and geometry.
| Aspect | CRM behavior |
|---|---|
| Syntactic errors | Detects invalid formatting, missing terminators such as <end>, insufficient or extra tokens, and tokenization errors |
| Geometric/structural errors | Detects unclosable loops, inconsistent parametric values, failed boolean operations, and violated geometric constraints |
| General compilation failures | Captures compiler-reported exceptions during model building or mesh tessellation |
This verification regime is more stringent than surface-form checking. An output may be token-level well formed yet still fail because a sketch does not form a closed loop, an extrusion height falls outside model bounds, or a boolean command sequence is invalid (Zhou et al., 6 Aug 2025). CRM therefore reviews not only textual conformance but also whether the parametric program induces a buildable geometric object.
Because the review signal is emitted by the CAD compiler, the feedback is intrinsically tied to the semantics of the downstream toolchain. This suggests that CRM belongs to a broader class of verifiable review mechanisms in which the compiler serves as both checker and diagnostic oracle.
4. Efficiency, measured effects, and iteration behavior
CAD-Judge reports that CRM enables real-time, per-sample validation without the need for heavy rendering or VLM-based reward assessment, and that each CRM validation step is “negligible” in computational overhead (Zhou et al., 6 Aug 2025). This is the efficiency argument for the module: repeated validation is feasible because the review operation is delegated to a fast local compiler rather than to a visual or multimodal evaluator.
The measured effect is reported primarily through Invalidity Ratio (IR). In ablation studies, adding agentic generation reduced IR from 3.07% to 2.52%, and when combined with two-stage finetuning, reduced IR to 1.54% on the DeepCAD L2 task (Zhou et al., 6 Aug 2025). The paper also reports that CRM improves geometric and syntactic fidelity, increases the rate of valid, synthesizable CAD programs, and contributes to higher F1 and lower Chamfer Distance in experiments.
Iteration studies refine this picture. Table 5 in the paper shows that each additional CRM-guided generation-review iteration slightly lowers IR, while Chamfer Distance plateaus quickly, indicating diminishing returns after the first review (Zhou et al., 6 Aug 2025). Accordingly, a single CRM iteration is reported as sufficient in practice for strong gains. This establishes an important operational boundary: CRM is effective as a lightweight repair loop, but not presented as a deeply recursive search procedure.
At the system level, CAD-Judge states that the overall method achieves state-of-the-art performance while maintaining superior efficiency (Zhou et al., 6 Aug 2025). Within that claim, CRM is the principal inference-time mechanism for preventing non-compilable outputs from propagating to the final result.
5. Relation to automated review systems for code and software artifacts
Although CRM is introduced in the context of text-to-CAD, related work on automated code review exhibits several adjacent design patterns. BitsAI-CR, deployed at ByteDance, uses a two-stage architecture in which RuleChecker identifies issues from code diffs and ReviewFilter validates candidate comments to remove hallucinations, false positives, and superfluous suggestions; it reports 75.0% precision in review comment generation, a Go Outdated Rate of 26.7%, and deployment at over 12,000 weekly active users (Sun et al., 25 Jan 2025). Its workflow also includes context preparation, comment aggregation via embedding-based clustering, and a structured feedback loop. This suggests a recurrent pattern across review systems: proposal generation followed by precision-oriented validation.
Benchmarking work sharpens the evaluation side of that pattern. ContextCRBench was constructed from 153.7K issues and pull requests and filtered to 67,910 context-enriched entries, with three evaluation scenarios aligned with review workflow: hunk-level quality assessment, line-level defect localization, and line-level review comment generation (Hu et al., 10 Nov 2025). The study reports that textual context yields greater performance gains than code context alone and that, when deployed at ByteDance, the benchmark drove a self-evolving code review system with a 61.98% relative improvement. For CRM-like systems, this is relevant because it frames review quality not only as syntactic correctness but also as context-sensitive judgment.
Agent-based review architectures provide another adjacent line. RevAgent decomposes automated review comment generation into five category-specific commentator agents and a critic agent that selects the most appropriate issue-comment pair, reporting improvements of 12.90% on BLEU, 10.87% on ROUGE-L, 6.32% on METEOR, and 8.57% on SBERT over the best competitors (Li et al., 1 Nov 2025). In parallel, CRScore++ combines verifiable feedback from static analyzers with LLM-based subjective feedback, extending CRScore into a reinforcement-learning framework for code review comment generation (Kapadnis et al., 30 May 2025). Together, these systems show that “review” in current research spans several evidence sources: compilers, static analyzers, taxonomies, critic agents, and context-rich benchmarks.
A plausible implication is that CRM occupies the most verifier-centric end of this spectrum. Its feedback is narrower than human-like code review, but it is correspondingly more direct and tool-grounded.
6. Evidence-first review and its extension to compiler engineering
The most direct extension of CRM’s underlying philosophy appears in automated review for compiler optimizations. Archer, designed for LLVM, constrains agentic review through semantic obligations and a deterministic validation guard that admits only findings backed by executable evidence (Ni et al., 2 Jul 2026). On 70 open PRs and 328 closed PRs from the last two months, Archer reports that 21% of open PRs and 11% of closed PRs are buggy, with 51 semantic bugs discovered overall, including 34 miscompilations and 17 compiler crashes (Ni et al., 2 Jul 2026).
Archer is not labeled a CRM, but its validation principle is closely aligned with compiler-backed review. A hypothesized flaw is materialized into a validation strategy and reported only if a semantic discrepancy is proven and uniquely triggered by the patch, using tools such as Alive2 and LLUBI (Ni et al., 2 Jul 2026). In CAD-Judge, the compiler diagnosis is used to refine a generated CAD program; in Archer, executable evidence is used to justify reporting a bug in compiler code itself. This suggests a broader methodological convergence around evidence-first review, especially where semantic correctness is difficult to assess from text alone.
The significance of CRM therefore extends beyond CAD generation. It exemplifies a review paradigm in which the artifact under review is coupled to a deterministic toolchain, the review signal is extracted from that toolchain, and iterative correction is driven by compiler or verifier diagnostics rather than by free-form preference alone. In domains where outputs must be compilable, synthesizable, or semantically validated, that coupling is the central technical property rather than an implementation detail.