The Alignment Problem in Constrained Code Generation

Abstract: LLMs have demonstrated strong capabilities in code generation, but their outputs frequently contain syntax or type errors that result in compilation failures. Constrained decoding has been proposed as a solution to mitigate compilation errors by construction, improving functional correctness as a byproduct. However, previous works overlook a critical aspect of constrained decoding: the alignment between constrainer (e.g., types), LLM and the target specification language (e.g., TypeScript). Misalignment is caused by the constrainer being incomplete--rejecting programs that belong to the target--or unsound--allowing programs that are not part of the target. The bias created by incompleteness distorts the LLM distribution, and can be detrimental for code generation. We evaluate this hypothesis using seven LLMs, two target languages, two constrainers, enforcing types and syntax during decoding, and we study how LLMs react to varying levels of incompleteness. On three benchmarks, when the constrainer is incomplete, unconstrained decoding significantly outperforms constrained decoding in terms of functional correctness. Incompleteness pushes the model into low-probability regions of the program space, causing the generation to frequently time out, and reducing functional correctness by up to 97%. These contributions make the community aware of the negative effects of misalignment in constrained decoding, and provide quantitative insights on how to design constrainers that are beneficial for code generation systems with formal guarantees.

• The paper shows that misalignment in constrained decoding can sharply decrease functional correctness, with drops up to 97% in certain scenarios.
• It establishes a formal framework using Jaccard similarity to assess the alignment among model outputs, target languages, and constrainer implementations.
• Empirical results indicate that finetuning and designing complete constrainers can partially mitigate the negative effects of incomplete constraints.

The Alignment Problem in Constrained Code Generation

Introduction

"The Alignment Problem in Constrained Code Generation" (2606.21619) systematically investigates the interplay between LLMs, formal decoders (constrainers), and target programming specifications in code generation tasks. Although constrained decoding has been posited as a means to guarantee type- and syntax-correct code, empirical observations indicate that it often degrades functional correctness, especially under practical (i.e., incomplete or unsound) constrainer implementations. This work crafts a formal and experimental framework to precisely characterize and quantify the detrimental effects of misalignment among the LLM, the constrainer, and the target language, and establishes the conditions under which constrained decoding can be beneficial.

Alignment Framework for Constrained Code Generation

The paper formalizes three interrelated "languages" inherent to constrained code generation:

• Model Language ($L_M$): The set of programs that the LLM is likely to generate under a given prompt and sampling distribution.
• Target Language ($L_T$): The set of programs sanctioned by the specification of the target programming language (e.g., full TypeScript semantics).
• Constrained Language ($L_C$): The set of programs allowed by the constrainer, operationalized as incremental syntactic and semantic validators enforced during decoding.

The crux of the paper's thesis is that beneficial constrained decoding requires strong alignment among $L_M$, $L_T$, and $L_C$. Misalignment, particularly incompleteness in $L_C$, can distort the sampling distribution of the LLM, forcing it to generate statistically unlikely or degenerate code and sharply reducing functional correctness.

The alignment is formally measured using Jaccard similarity among the three sets, and the framework predicts several real-world pathological regimes: incomplete constrainers (constraining more than the target), unsound constrainers (constraining less), and ill-posed combinations resulting in diminutive intersection among the sets.

Empirical Analysis of Constrained vs. Unconstrained Decoding

The empirical strategy includes extensive benchmarking with six LLMs (2B–34B parameters, including Gemma-2 and CodeLlama), using two code generation datasets (HUMANEVAL and MBPP for TypeScript) and a further syntax-focused setting using TOML. The main metric is functional correctness (percentage of generated programs passing all test cases), supplemented by type/syntax correctness, and timeout rate (frequency of exceeding budget via degenerate generation).

Key Findings:

• Unconstrained decoding outperforms constrained decoding in functional correctness across all models and hyperparameters on TypeScript, with constrained decoding's negative impact most pronounced for small models and at higher temperatures.
• Constrained decoding yields higher type-syntax correctness (measured by compilation success) for less capable models, but this does not translate into superior functional correctness, as timeouts and text degeneration dominate.
• For incomplete constrainers (e.g., those lacking forward reference support in TypeScript), functional correctness drops by up to 97% relative to a complete constrainer baseline.
• The cost of misalignment is quantifiable: Constrained decoding forces the LLM distribution into low-probability regions, verified by elevated negative log-likelihood (NLL) statistics and increased timeout rates.

Mitigating Incompleteness Bias: Finetuning and Constrainer Design

To probe remedies, the paper explores two mitigation strategies:

1. LLM Alignment via Finetuning: By finetuning LLMs with reinforcement learning methods (e.g., group relative policy optimization) on examples conforming to the constrainer, the discrepancy between the LLM’s generative capabilities and constrainer’s acceptance is reduced. Empirically, this narrows but does not close the gap in functional correctness between constrained and unconstrained decoding. While type/syntax correctness improves and timeouts decrease by 53–73%, residual incompleteness bias remains.
2. Constrainer Completeness: In TOML, where a complete syntax-based constrainer is practical, constrained decoding substantially outperforms the unconstrained strategy (e.g., by 53% for the 2B model and 2.5% for a 32B model). However, even minimal incompleteness (forbidding optional whitespace) can devastate performance, again up to 97% drops. Perturbation experiments reveal that model sensitivity to incompleteness extends even to "aesthetic" features like spacing, demonstrating that statistical misalignment is extremely brittle to constrainer coverage.

Theoretical and Practical Implications

This work contradicts the common assumption that constrained decoding is always preferable for code generation by showing that, unless strict alignment is achieved, imposing constraints often distorts the learned model distribution, ultimately producing less useful code. The practical implication is that constrained decoding frameworks must either:

• engineer highly complete—and ideally sound—constrainer implementations closely matching target semantics, or
• co-train or extensively finetune LLMs to the actual distribution induced by the constrainer.

Otherwise, especially for languages with complex semantics (e.g., TypeScript's type system), the risk is that constrained decoding sacrifices functional quality for formal token-level guarantees.

In smaller models, the negative impact of constrainer incompleteness is even more severe, suggesting that constrained decoding as a crutch for low-capacity LLMs is only justified under very accurate constraint modeling.

Prospective Research Directions

The alignment framework introduced in this paper generalizes beyond code generation and is applicable to any structured generative LM application employing formal constraints (e.g., data serialization, program synthesis, symbolic reasoning). Further research may seek:

• automated constrainer completeness analysis tools,
• data-driven constrainer synthesis informed by LLM error statistics,
• joint training protocols that harmonize the LLM and constrainer distributions,
• more nuanced trade-offs between formal correctness, sample diversity, and practical utility in real-world generation workloads.

Conclusion

This work rigorously demonstrates that strictly complete constrainers are a necessary (but not always feasible) prerequisite for beneficial constrained decoding in LLM-based code generation. Misalignment introduces an "incompleteness bias" that can devastate functional correctness, especially for smaller models or complex languages. Finetuning can partially restore alignment but does not fully resolve the challenge. The alignment perspective offers an actionable diagnostic and design principle for future research into controllable and reliable code generative systems (2606.21619).