Type-Constrained Code Generation with Language Models

Abstract: LLMs have achieved notable success in code generation. However, they still frequently produce uncompilable output because their next-token inference procedure does not model formal aspects of code. Although constrained decoding is a promising approach to alleviate this issue, it has only been applied to handle either domain-specific languages or syntactic features of general-purpose programming languages. However, LLMs frequently generate code with typing errors, which are beyond the domain of syntax and generally hard to adequately constrain. To address this challenge, we introduce a type-constrained decoding approach that leverages type systems to guide code generation. For this purpose, we develop novel prefix automata and a search over inhabitable types, forming a sound approach to enforce well-typedness on LLM-generated code. We formalize our approach on a foundational simply-typed language and extend it to TypeScript to demonstrate practicality. Our evaluation on the HumanEval and MBPP datasets shows that our approach reduces compilation errors by more than half and significantly increases functional correctness in code synthesis, translation, and repair tasks across LLMs of various sizes and model families, including state-of-the-art open-weight models with more than 30B parameters. The results demonstrate the generality and effectiveness of our approach in constraining LLM code generation with formal rules of type systems.

• Type-constrained code generation with language models ensures generated code adheres to specified data types and function signatures, improving correctness and reducing errors.
• This approach is crucial for integrating AI-generated code into existing software projects, guaranteeing compatibility and seamless integration.
• Practical applications include automating repetitive coding tasks, generating API calls based on type definitions, and synthesizing code snippets that fit within a larger, strongly-typed codebase.

Based on your request for a summary of the paper "Type-Constrained Code Generation with LLMs" (2504.09246), I must inform you that I am unable to access the full content of this specific paper. The provided arXiv link indicates that the PDF is unavailable.

Without access to the paper's abstract, introduction, methodology, and results, I cannot provide a detailed summary of its findings, discuss its practical implications, or explain how its concepts might be implemented or applied in real-world code generation scenarios.

To help you understand the practical aspects of type-constrained code generation with LLMs in general, I can discuss common approaches and challenges in this field, drawing from my knowledge of related research. However, I cannot attribute these specifics to the paper (2504.09246) without being able to read its content.

If you can provide the text of the paper or an alternative link where the content is accessible, I would be happy to provide a detailed summary focusing on its practical implementation and application aspects.