Internalizing Geometric Law: Learning from Solver Residuals for Precision-Critical Generation

Abstract: LLMs frequently hallucinate in precision-critical domains such as technical diagramming and mechanical design, where outputs must satisfy strict geometric constraints. We study open-ended geometric synthesis from natural language: translating free-form descriptions into precise constructions whose entities must simultaneously satisfy dozens of interacting constraints. To make this tractable, we release PyGeoX, a programmable geometric DSL that compiles declarative constraints into a differentiable loss, and PyGeoX-Bench, a stratified suite of 300 problems with per-constraint verifiable rewards. Using PyGeoX as a verifier, we identify a failure mode we call Outlier Gradient Masking: under global-norm rewards (any scheme that aggregates residuals through a single norm, for example, $\exp(-\mathrm{MSE})$), a single outlier constraint can nullify the learning signal across all others. To address this, we propose Saturating Additive Rewards (SAR), which decompose the reward into bounded per-constraint terms, preserving partial progress and ensuring consistent gradients even under severe violations. Against MSE-based rewards, the natural baseline for geometry solvers, SAR improves the hard-tier solving rate by $2.3\times$, and the resulting 8B model is competitive with much larger frontier systems on this benchmark. We release the engine, benchmark, and data at https://github.com/Huawei-AI4Math/PyGeoX.

• The paper introduces the Saturating Additive Rewards (SAR) framework that decomposes rewards into dense, local signals, effectively mitigating outlier gradient masking.
• It presents PyGeoX, a differentiable geometry engine that translates natural language diagram specifications into exact metric coordinates using symbolic and numeric constraints.
• Experimental results demonstrate that SAR-based RL agents achieve 2.3× higher hard-tier solving rates and robust out-of-distribution performance in complex geometric constraint tasks.

Internalizing Geometric Law: Learning from Solver Residuals for Precision-Critical Generation

Problem Formulation and Motivation

This work systematically addresses the problem of geometric constraint satisfaction in generative models, notably LLMs, where solutions must simultaneously satisfy high-order, interdependent, and precision-critical geometric constraints. Prior approaches in technical design and diagramming typically delegate constraint satisfaction to external solvers, granting models only the capacity to translate problems into solver-specific representations, thus preventing direct internalization of geometric law. These limitations result in "precision hallucination": outputs that may be syntactically plausible but violate geometric or engineering feasibility. This study formalizes end-to-end Geometric Constraint Solving (GCS) as an RL-driven LLM alignment task, requiring models to directly emit exact metric coordinates grounded in the underlying spatial law.

Figure 1: The GCS agent emits Python code specifying exact geometric entities—coordinates, radii, etc.—corresponding to precise solutions of natural language diagram specifications.

PyGeoX: Differentiable Geometry Environment

The core technical infrastructure introduced is PyGeoX, a domain-specific language (DSL) and programmable geometry engine facilitating both symbolic and numeric constraint handling. PyGeoX allows the declarative specification of geometric objects (35 types) and relational constraints (38 relationships, including both equalities and inequalities). The system translates these into symbolic residuals and aggregates them into a differentiable global error function, enabling efficient procedural data generation and per-constraint verifiable RL reward computation.

Figure 3: PyGeoX maps geometric descriptions into systems of equality and inequality constraints, dynamically compiling all residuals into a differentiable composite error.

PyGeoX's differentiable engine enables not only robust verification of generated coordinate tuples but also supports expressive constraint types—critical for real-world technical design problems, which may require relationships not limited to classic geometric predicates.

Outlier Gradient Masking in Reward Design

The central technical contribution is the identification and analysis of Outlier Gradient Masking in standard RL reward aggregation for GCS. Traditional schemes (e.g., global-norms such as $\exp(-\|\mathbf{r}\|_2^2)$ or sparse binary signal) collapse the reward landscape in high dimensions: a single severe constraint violation can reduce the reward to near-zero, eradicating learning signal for all satisfied constraints and thus suppressing policy gradient updates. The authors provide theoretical analysis, showing the exponential decay in reward signal volume under global norms as the number of constraints grows, while a per-constraint decomposed approach sustains dense feedback throughout the search space.

Figure 2: (Left) Global-norm rewards collapse with partial correctness, producing no useful gradient; SAR preserves dense signal even under partial violation. (Right) SAR enables higher reward with fewer generated tokens.

Saturating Additive Rewards (SAR): Dense and Local Signal

To circumvent Outlier Gradient Masking, the SAR framework decomposes the total reward into a sum of bounded, local terms—one for each constraint—with domain-specific kernel selection (e.g., Boltzmann, Cauchy, sigmoidal) to ensure that progress on a single constraint is not vetoed by errors elsewhere. This design is shown to guarantee both dense initial feedback (saturating the effective reward volume in high dimensions) and local gradient informativeness, enabling RL optimization even on complex, highly constrained geometric synthesis tasks.

SAR is further combined with (1) a sparse bonus to ensure precise convergence, incentivizing satisfaction of all constraints (comparable to the harsh correctness demands of technical domains), and (2) a degeneracy penalty to avoid trivial or collapsed solutions (e.g., all points coincident), yielding a compositional reward suitable for neuro-symbolic RL.

Data Generation and Benchmarking

A fully-automated pipeline seeds up to $10^5$ diverse geometric problems using PyGeoX's generative engine, stratified by constraint and object count. The PyGeoX-Bench assessment suite samples 300 evaluation problems across three difficulty tiers, and the PyGeoX-Wild OOD diagnostic evaluates generalization to diverse linguistic forms and unseen constraint types.

Figure 4: Data pipeline for synthetic geometric construction, validation, and stratification into training and evaluation splits.

Experimental Results and Analysis

The authors perform controlled ablation across reward schemes, showing that pure global-norm loss (e.g., MSE-based) yields collapse on hard problems, suppressed partial-credit signal for complex constraints, and convergence instability. In contrast, SAR+Sparse+Degeneracy penalties consistently outperform both field-standard sparse RLVR and pure dense alternatives—achieving 2.3× higher hard-tier solving rates relative to global-norm rewards and outperforming several much larger proprietary models in zero-shot evaluations.

Figure 6: PyGeoX ground truth and RL-generated diagram for an easy task (reward=10); SAR enables precise constructive synthesis.

Figure 5: Medium-tier task; the model approaches but does not perfectly match ground truth, yielding partial reward of 3.0.

Architecturally, SAR-based RL agents exhibit both shorter reasoning traces (∼23% token reduction on average) and robust OOD transfer, attaining leading solve rates on the PyGeoX-Wild benchmark despite three-way domain shift and combinatorially infeasible memorization.

Theoretical and Practical Implications

This work situates RL for GCS within the broader neuro-symbolic reasoning and physics-informed learning literature, offering a formal bridge between residual-based supervision in continuous domains and dense RL reward design. The challenge and remedy outlined here—local reward decomposition for high-dimensional constraint satisfaction—extends naturally to domains such as physics simulation, kinematic mechanism design, robotic manipulation, and other settings where verifiable solver residuals define ground truth.

The PyGeoX infrastructure, data, and model checkpoints are released to accelerate follow-on research; the technical design also offers a blueprint for extending RL with dense, local, differentiable supervision in any setting where constraints admit compositional residuals.

Conclusion

By operationalizing GCS as an RL alignment task and introducing SAR as a robust, domain-native reward, this work demonstrates that LLMs can be aligned to internalize geometric law without reliance on external symbolic engines. The result is a scalable, extensible framework for precision-critical code and diagram generation, achieving order-of-magnitude improvements in hard problem success rates. The research motivates future work in hybrid reward composition for RL, generalized to high-dimensional, compositional verification tasks in mathematical, physical, and engineering AI.

• "Internalizing Geometric Law: Learning from Solver Residuals for Precision-Critical Generation" (2606.09278)