GenCAD-3D: Multimodal CAD Program Generation

• GenCAD-3D is a multimodal generative framework that synthesizes parametric CAD programs from 3D geometric data through latent space alignment.
• It employs a CAD autoencoder, modality-specific encoders, and a conditional latent diffusion model to translate point clouds and meshes into command sequences.
• The framework integrates SynthBal, a synthetic dataset balancing strategy, to enhance performance on complex, long-featured CAD programs.

GenCAD-3D is a multimodal generative framework for CAD program generation, retrieval, and reconstruction from 3D geometric data, specifically point clouds and meshes. It targets parametric CAD programs structured as sequences of commands that compile into precise 3D geometries, and it addresses two limitations identified in prior deep generative CAD systems: the difficulty of aligning nonparametric 3D observations with parametric program spaces, and the imbalance of available CAD datasets toward simple programs rather than complex feature trees. Its formulation combines a CAD program autoencoder, modality-specific 3D encoders aligned to the CAD latent space through contrastive learning, a conditional latent diffusion model for sequence generation, and a synthetic data augmentation strategy called SynthBal that balances representation across CAD program sequence lengths (Yu et al., 17 Sep 2025).

1. Conceptual position within generative CAD

GenCAD-3D operates in the domain of automated CAD program synthesis rather than mesh-only or point-cloud-only generation. In this setting, the output is not merely a geometric surface approximation but a parametric program, represented as a feature tree or CAD command sequence, that remains editable and suitable for downstream engineering workflows. This design objective places GenCAD-3D in continuity with earlier efforts that framed CAD generation as sequence modeling, such as "GenCAD: Image-Conditioned Computer-Aided Design Generation with Transformer-Based Contrastive Representation and Diffusion Priors" (Alam et al., 2024), while extending the conditioning modalities from images to point clouds and meshes and placing stronger emphasis on sequence complexity and dataset balancing (Yu et al., 17 Sep 2025).

A central problem addressed by GenCAD-3D is that current deep generative models for CAD generation are significantly limited by imbalanced and insufficiently large datasets, particularly those lacking representation for complex CAD programs. The framework therefore couples representation learning with dataset engineering. This suggests that, in the authors’ formulation, model architecture alone is insufficient unless the training distribution adequately covers long and structurally complex command sequences (Yu et al., 17 Sep 2025).

The broader research landscape clarifies this positioning. GenCAD established an image-conditioned pipeline based on autoregressive transformers, contrastive CAD-image representation alignment, and a latent diffusion prior for CAD sequence generation (Alam et al., 2024). CAD-GPT moved toward single-image or text-conditioned synthesis using a spatial reasoning-enhanced multimodal LLM and a 3D Modeling Spatial Localization Mechanism (Wang et al., 2024). GeoCAD shifted the focus to local geometry-controllable modification of CAD parts under user-provided geometric instructions (Zhang et al., 12 Jun 2025). GenCAD-Self-Repairing addressed the feasibility limitation of GenCAD by introducing guided diffusion denoising and a self-repair pipeline for infeasible boundary representations (Tsuji et al., 29 May 2025). Against this background, GenCAD-3D is distinguished by multimodal latent space alignment for geometry-to-program translation and by SynthBal, which explicitly targets dataset imbalance in complex CAD programs (Yu et al., 17 Sep 2025).

2. System architecture and latent representation

The framework consists of three key technical components: a CAD Program Autoencoder, multimodal encoders with contrastive latent alignment, and a conditional latent diffusion model. Together, these define a geometry-to-latent-to-program pipeline in which the CAD latent space serves as the reference representation and 3D geometric inputs are mapped into that space before generation or retrieval (Yu et al., 17 Sep 2025).

The CAD Program Autoencoder learns a compact latent representation of CAD command sequences. Its input is a CAD program represented as a padded matrix of command and parameter vectors under the DeepCAD convention of $60 \times 17$ per program. The architecture is a causal transformer autoencoder, described as following prior GenCAD practice, and it maps programs into a continuous latent space $\mathbf{z}_{\mathcal{C}} \in \mathbb{R}^{d_m}$. Because the autoencoder is causal and sequence-based, it preserves sequential dependencies among commands through causal self-attention (Yu et al., 17 Sep 2025).

The modality-specific encoders process nonparametric geometry. For point clouds, GenCAD-3D uses DGCNN; for point clouds with normals, it uses DGCNN with additional normal features, yielding 6D input; for meshes, it uses FeaStNet with mean-pooling for global embedding. These encoders map geometry $\mathcal{M}$ to a latent embedding $\mathbf{z}_{\mathcal{M}} \in \mathbb{R}^{d_m}$ so that it can be compared directly with the CAD latent produced by the autoencoder (Yu et al., 17 Sep 2025).

The resulting architecture can be summarized as follows.

Component	Input	Output
CAD Program Autoencoder	CAD program as $60 \times 17$ padded matrix	$\mathbf{z}_{\mathcal{C}}$
3D Encoder	Point cloud, point cloud + normals, or mesh	$\mathbf{z}_{\mathcal{M}}$
Conditional Latent Diffusion + Decoder	$\mathbf{z}_{\mathcal{M}}$	CAD command sequence

This organization reflects a clear separation of roles. The CAD autoencoder establishes a latent space of valid, realistic CAD programs; the geometric encoders project nonparametric inputs into that space; and the diffusion prior models the conditional transformation from geometric latent to CAD latent before decoding (Yu et al., 17 Sep 2025).

3. Multimodal latent space alignment and conditional generation

The alignment mechanism is based on contrastive learning. The motivation is to ensure that embeddings of the same object across geometry and CAD modalities are close in latent space, while non-corresponding pairs are distant. For a batch of $B$ matched pairs $\{(\mathcal{C}_i,\mathcal{M}_i)\}$, GenCAD-3D uses cosine similarity

$\mathbf{z}_{\mathcal{C}} \in \mathbb{R}^{d_m}$0

and optimizes the loss

$\mathbf{z}_{\mathcal{C}} \in \mathbb{R}^{d_m}$1

where $\mathbf{z}_{\mathcal{C}} \in \mathbb{R}^{d_m}$2 is a learned temperature parameter. The CAD encoder is frozen, and the geometry encoders are trained to align their embeddings to the CAD latent space (Yu et al., 17 Sep 2025).

This contrastive formulation extends the cross-modal alignment idea already present in GenCAD, where CAD and image embeddings were jointly embedded in a unified latent space for retrieval and conditioning (Alam et al., 2024). The difference is that GenCAD-3D aligns CAD with 3D geometric modalities rather than rendered images, thereby targeting reverse engineering and geometry-conditioned reconstruction more directly (Yu et al., 17 Sep 2025).

Conditional generation is performed in latent space with a diffusion prior. The framework factorizes the conditional model as

$\mathbf{z}_{\mathcal{C}} \in \mathbb{R}^{d_m}$3

where $\mathbf{z}_{\mathcal{C}} \in \mathbb{R}^{d_m}$4 is modeled using conditional latent diffusion. The denoising objective is

$\mathbf{z}_{\mathcal{C}} \in \mathbb{R}^{d_m}$5

with $\mathbf{z}_{\mathcal{C}} \in \mathbb{R}^{d_m}$6 implemented as a time-conditional MLP ResNet receiving the noised latent $\mathbf{z}_{\mathcal{C}} \in \mathbb{R}^{d_m}$7, time step $\mathbf{z}_{\mathcal{C}} \in \mathbb{R}^{d_m}$8, and conditioning latent $\mathbf{z}_{\mathcal{C}} \in \mathbb{R}^{d_m}$9 (Yu et al., 17 Sep 2025).

The diffusion model therefore does not directly generate command tokens. Instead, it generates a CAD-program latent that is subsequently decoded into a full CAD command sequence. This suggests a two-level inductive bias: the autoencoder constrains the output to the learned program manifold, while the diffusion prior handles conditional multimodal mapping within that latent manifold (Yu et al., 17 Sep 2025).

4. SynthBal and synthetic dataset balancing

SynthBal is the synthetic data augmentation strategy introduced to balance and expand CAD datasets, with explicit focus on improving representation of complex CAD geometries. Its premise is that real-world CAD datasets such as DeepCAD are heavily skewed toward simple models with shorter command sequences, whereas complex, long-featured CAD programs are rare. As a result, standard aggregate metrics can be dominated by performance on simple shapes and can obscure failures on long sequences (Yu et al., 17 Sep 2025).

The SynthBal algorithm constructs a synthetically balanced dataset with equal representation across all CAD program sequence lengths. For each sequence length $\mathcal{M}$0 from the minimum to the maximum in the dataset, a target number of examples is set as

$\mathcal{M}$1

The dataset is then filled with as many real examples as possible, subject to the real/synthetic ratio parameter $\mathcal{M}$2, and the remainder is populated with synthetic augmentations until the target count for that sequence length is reached (Yu et al., 17 Sep 2025).

Two augmentation operations are specified. The first is noise, which perturbs continuous parameters such as coordinates in commands. The second is replace-sketch, which randomly grafts sketch portions from one CAD program into another of the same length. All generated CAD programs are validated and must compile successfully and be free of self-intersections. The ratio parameter $\mathcal{M}$3 controls the real/synthetic mix; the paper gives the example $\mathcal{M}$4, corresponding to 20% original and 80% synthetic data (Yu et al., 17 Sep 2025).

Two released balanced datasets are described:

Dataset	Size	Real / synthetic composition
SynthBal	$\mathcal{M}$5170,000 examples	16.7% real, 83.3% synthetic
SynthBal-1M	1 million examples	13.9% real, 86.1% synthetic

The reported role of SynthBal is not merely to enlarge the training set but to enforce sequence-length balancing. The paper states that balancing, rather than augmentation alone, is key. It also emphasizes the necessity of explicit validity checks, contrasting this with prior methods such as DeepCAD or ContrastCAD, which can learn invalid programs (Yu et al., 17 Sep 2025). A plausible implication is that GenCAD-3D treats complexity distribution as a first-class modeling variable rather than a secondary dataset property.

5. Empirical performance, evaluation protocol, and released resources

GenCAD-3D reports results for autoencoding, retrieval, and conditional generation. The evaluation metrics include command and parameter accuracy, Chamfer distance, IoU, invalid ratio, retrieval Top-$\mathcal{M}$6 accuracy, and sequence-length normalized averages designed to highlight performance on long and complex models (Yu et al., 17 Sep 2025).

On autoencoder learning, sequence-length normalized metrics show that SynthBal substantially outperforms prior methods, especially on complex CAD programs. Relative error reductions versus the next-best methods on complex cases are reported as 16% for command accuracy, 24% for parameter accuracy, 57% for Chamfer distance, and 58% for invalid ratio. The paper further states that invalid CAD generations dropped from approximately 3.4% to less than 1% with SynthBal (Yu et al., 17 Sep 2025).

A more detailed model figure is provided for an autoencoder variant denoted SynthBal-1MFT. Its normalized metrics are command accuracy 98.5%, parameter accuracy 94.7%, Chamfer distance 2.0 compared with 7.8 in DeepCAD, and invalid ratio 3.6% (Yu et al., 17 Sep 2025). Because the paper also emphasizes sequence-length normalized evaluation, these numbers are specifically intended to expose behavior on long programs rather than only on short-sequence majorities.

For retrieval, Top-1 accuracy in a library of 2048 CAD models increased by up to 25% for point clouds, 22% for point cloud plus normal inputs, and 7% for meshes. A specific mesh-encoder result is also reported: 81.6% Top-1 accuracy in batches of size 2048 for SynthBal-1MFT (Yu et al., 17 Sep 2025). The retrieval task is a direct consequence of the aligned latent space: geometry embeddings can query a CAD codebook, and matched programs can be recovered without sequence generation.

For conditional CAD generation from geometry, command accuracy improved by 14–16%, parameter accuracy by 13–14%, and geometric error measured by Chamfer and IoU decreased. The paper highlights especially strong improvements on high-complexity CAD sequences. In mesh-modality conditional reconstruction, normalized command accuracy reaches up to 60.5% (Yu et al., 17 Sep 2025).

The released resources include multimodal DeepCAD and SynthBal datasets for point cloud, mesh, and CAD command representations, as well as a set of 51 3D-printed and laser-scanned parts with corresponding ground-truth CAD sequences. These scan-CAD pairs are intended to provide realistic test cases that include scanning artifacts not captured in purely synthetic data (Yu et al., 17 Sep 2025).

6. Relation to adjacent lines of work and technical significance

GenCAD-3D belongs to a broader shift from geometry-only 3D generation toward editable and manufacturable representations. Earlier GenCAD modeled image-to-CAD synthesis with autoregressive sequence reconstruction, contrastive CAD-image pretraining, and a CAD diffusion prior (Alam et al., 2024). GenCAD-3D preserves the central motif of a CAD latent space aligned to another modality, but it targets point clouds and meshes and introduces dataset balancing as a core methodological component rather than as a preprocessing convenience (Yu et al., 17 Sep 2025).

Its treatment of validity is also noteworthy in comparison with neighboring work. GenCAD-Self-Repairing identified a major limitation in GenCAD: approximately 10% of generated designs were infeasible, and it improved feasibility from 0.931 to 0.970 on the GenCAD validation set through classifier-guided and regressor-guided diffusion together with a self-repair pipeline (Tsuji et al., 29 May 2025). GenCAD-3D does not present the same repair mechanism, but it does enforce validity checks during SynthBal augmentation and reports reductions in invalid ratio. This suggests a complementary distinction between repair-after-generation and balancing-plus-validation during training (Yu et al., 17 Sep 2025).

The relation to CAD-GPT and GeoCAD further clarifies scope. CAD-GPT reformulates CAD synthesis around multimodal LLMs and a discrete spatial tokenization mechanism for precise 3D spatial inference from a single image or textual description (Wang et al., 2024). GeoCAD focuses on local geometry-controllable editing, using complementary captioning and LLM-based infilling of masked parts under geometric instructions (Zhang et al., 12 Jun 2025). By contrast, GenCAD-3D is neither an MLLM-centric spatial reasoning system nor a local editing framework; its primary contribution is multimodal latent alignment from geometry to program space and sequence-length-balanced synthetic augmentation (Yu et al., 17 Sep 2025).

Within the larger CAD and 3D representation literature, the framework also participates in a recurring theme: retaining native or procedural structure improves downstream utility. STEP-based graph neural classification showed that operating directly on native CAD structure can outperform point-cloud methods and match or exceed multi-view image approaches (Mandelli et al., 2022). "Learning 3D Representations from Procedural 3D Programs" showed that procedural shape generation can provide scalable training data for 3D representation learning without semantic object realism (Chen et al., 2024). GenCAD-3D’s SynthBal is consistent with that broader tendency, but it applies it specifically to parametric CAD programs and complexity balancing rather than to generic self-supervised point-cloud pretraining (Yu et al., 17 Sep 2025).

From an applications standpoint, the paper identifies reverse engineering, automated engineering design, and human-in-the-loop support as primary use cases. Given a point cloud or mesh, including one derived from a physical scan, the framework aims to infer an exact or approximate CAD program ready for modification. It also supports cross-modal search and retrieval and can push generated CAD programs to commercial CAD software such as Onshape via provided APIs for further user edits and workflow integration (Yu et al., 17 Sep 2025). This suggests that GenCAD-3D is designed not only as a generative benchmark but also as an infrastructural component for CAD reconstruction pipelines in which editability, validity, and complexity coverage are all operationally important.