Seq-to-Seq CAD Learning

• Sequence-to-sequence CAD learning is a suite of AI-driven techniques that transform diverse inputs into parametric CAD commands while preserving design history.
• It employs transformer models, dual decoders, and latent space methods to accurately generate and edit command sequences with both discrete and continuous parameters.
• The approach enhances reverse engineering, text-to-CAD synthesis, and interactive design automation, addressing challenges like sequence reconstruction and multi-modal alignment.

Sequence-to-sequence CAD learning is a family of techniques that model the conversion between an input sequence (which may be natural language, sketches, vector drawings, images, point clouds, or other modalities) and the parametric command sequence needed to generate a computer-aided design (CAD) model. This problem is central to modern AI-driven design automation, reverse engineering, human–AI interaction in CAD software, and various workflows where the design process must be captured, reconstructed, or synthesized from sequential or multi-modal data. The rapid progress in neural sequence modeling and advances in transformer architectures have enabled increasingly precise, robust, and multi-modal sequence-to-sequence CAD solutions.

1. Problem Definition and Sequence Representation

Sequence-to-sequence CAD learning is formulated as the conditional sequence modeling problem:

$$P(Y|X) = \prod_{i=1}^N P(y_i | y_1,\dots, y_{i-1}, X),$$

where $X$ is the input sequence (such as a sequence of text tokens, image pixels, or vector primitives), and $Y$ is the output CAD command sequence $\{y_1,\dots,y_N\}$.

CAD sequences typically represent the modeling history as an ordered list of parametric commands, where each token $y_i$ encodes not only the operation type (e.g., line, arc, circle, extrusion, revolution, Boolean) but also continuous parameters (such as endpoints, radii, angles, and offsets). The representation varies: some frameworks use JSON-encoded command sequences (Govindarajan et al., 13 Jul 2025), others use domain-specific languages (DSLs) (Li et al., 9 Jan 2025), executable code (e.g. CadQuery Python code in (Rukhovich et al., 18 Dec 2024)), or compressed vector sequences as in transformer autoencoders (Jung et al., 2 Apr 2024, Yu et al., 17 Sep 2025).

CAD sequences differ fundamentally from geometry-only representations (B-rep, meshes, voxels) in that they preserve the full procedural history, enabling editability and semantic traceability—crucial capabilities for engineering workflows and reverse engineering.

2. Core Modeling Architectures

Sequence-to-sequence CAD learning has been implemented through several neural architectures, each reflecting key technical trade-offs:

• Autoregressive Transformers: The prevailing approach models CAD generation as a language modeling task using transformer encoder-decoder or decoder-only (causal) networks (Alam et al., 8 Sep 2024, Khan et al., 25 Sep 2024, Jung et al., 2 Apr 2024). These models predict the next command in sequence, capturing both local and long-range dependencies in the construction history.
• Hierarchical and Cyclic Architectures: Some frameworks, such as Cseq2seq (Zhang et al., 2016), introduce cyclic feedback mechanisms where the decoder state is recurrently fed back into the encoder, creating dynamic context updates. Hierarchical strategies, as in TransCAD (Dupont et al., 17 Jul 2024), decompose the generation process into structured stages (e.g., loop–extrusion hierarchies).
• Latent Space Models and Diffusion Priors: Modern systems align CAD command sequences with latent embeddings using contrastive learning, then sample these spaces via conditional diffusion models (Alam et al., 8 Sep 2024, Yu et al., 17 Sep 2025). This enables robust multi-modal generation, retrieval, and interpolation.
• Dual-Decoder or Decoupled Models: To enhance the precision of both discrete command type and continuous parameter prediction, architectures such as Drawing2CAD (Qin et al., 26 Aug 2025) employ dual-decoder transformers, separately generating command types and parameters conditioned on each other.
• Integration with LLMs: Recent work leverages code-oriented LLMs (such as Qwen2.5-Coder) for text-to-CAD sequence generation and interpretable code reconstruction (Govindarajan et al., 13 Jul 2025, Rukhovich et al., 18 Dec 2024), directly harnessing LLM capacity for structured programmatic output.

3. Data Construction, Augmentation, and Supervision

The success of sequence-to-sequence CAD models hinges on access to large, high-quality, and diverse paired datasets of input–output sequences:

• Synthetic Data Generation: Since real human-annotated CAD histories paired with other modalities are rare, most works procedurally generate construction sequences and synthesize paired images, point clouds, or text prompts (Rukhovich et al., 18 Dec 2024, Li et al., 9 Jan 2025, Khan et al., 25 Sep 2024). Synthetic balancing strategies (SynthBal (Yu et al., 17 Sep 2025)) are introduced to redress the long-tail distribution of sequence complexities, particularly extending representation to high-complexity CADs.
• Augmentation Techniques: To address the problem of multiple valid construction sequences for a given shape and to enhance model robustness on imbalanced datasets, specialized augmentation is employed. For sequence permutation robustness, contrastive dropout and Random Replace and Extrude (RRE) methods have been applied (Jung et al., 2 Apr 2024), yielding latent representations insensitive to non-shape-preserving sequence rearrangements or local command replacements.
• Multi-level and Multi-modal Annotation: Recent frameworks employ large language and vision-LLMs for automated annotation, generating multi-level textual prompts (from abstract to expert) for each CAD model (Khan et al., 25 Sep 2024, Govindarajan et al., 13 Jul 2025).
• Intermediate Supervision and Decomposition: Introducing intermediate sub-task outputs as part of the training target provably enables learnability on otherwise intractable composite CAD workflows (Wies et al., 2022), ensuring gradient signal at all reasoning levels.

4. Robustness, Evaluation, and Metrics

Evaluating sequence-to-sequence CAD learning requires metrics beyond geometry-only fidelity:

• Command/Parameter Accuracy: Mean accuracy of reconstructed command types and parameters (Alam et al., 8 Sep 2024, Dupont et al., 17 Jul 2024).
• Geometric Fidelity: Mean Chamfer Distance (median/mean CD), Coverage (COV), Minimum Matching Distance (MMD), and Intersection-over-Union (IoU) between output and target shapes (Rukhovich et al., 18 Dec 2024, Yu et al., 17 Sep 2025).
• Topological and Mesh Quality: Sphericity Discrepancy (SD), Discrete Mean Curvature Difference (DMCD), Euler Characteristic Match (EECM), and watertightness are introduced to quantify mesh and topological quality (Govindarajan et al., 13 Jul 2025).
• Sequence and Program Metrics: Invalidity Ratio (fraction of generated sequences that fail to render valid CAD), command sequence edit distance, and mean Average Precision of CAD Sequence (APCS) (Dupont et al., 17 Jul 2024) that jointly assess the fidelity of the full procedural history.
• Retrieval and Multi-modal Alignment: Contrastive retrieval accuracy (top-n) between latent representations (e.g., matching images to CAD programs) demonstrates the quality of joint embedding spaces (Alam et al., 8 Sep 2024, Yu et al., 17 Sep 2025).

5. Advanced Training Protocols and Feedback Mechanisms

Several works advance the training and supervision of sequence-to-sequence CAD systems:

• Parameter Sharing: Weight tying between encoder and decoder, as in Cseq2seq (Zhang et al., 2016), reduces model redundancy and provides regularization (up to a 31% parameter reduction without performance loss).
• Self-Regulated and Cost-Aware Learning: In interactive or human-in-the-loop scenarios, self-regulation strategies select among feedback types (full corrections, weak feedback, self-supervision, or none) to optimize improvement cost-effectively, employing strategies akin to $\epsilon$-greedy bandits (Kreutzer et al., 2019).
• Distribution Matching and Local Augmentation: Instead of maximum likelihood, distribution matching frameworks align entire local output distributions (not just single-point targets) via KL-divergence minimization, using augmenters to simulate the diversity of plausible outputs, thereby alleviating data sparsity (Chen et al., 2018).
• Reinforcement Learning CAD Gym: For tasks involving action selection over the CAD command space, reinforcement learning gyms (like RLCAD) employ reward functions combining geometric, topological, and surface metrics to guide the agent’s command selection, supporting advanced operators such as revolution and boolean combinations (Yin et al., 24 Mar 2025).

6. Applications, Challenges, and Outlook

Sequence-to-sequence CAD learning empowers a range of applications:

• Reverse Engineering: Models like TransCAD (Dupont et al., 17 Jul 2024), GenCAD-3D (Yu et al., 17 Sep 2025), and CAD-Recode (Rukhovich et al., 18 Dec 2024) convert observed data (images, point clouds, or B-Rep geometries) into editable, parametric CAD histories suitable for further design iteration.
• AI-driven Design Generation: Natural language–driven frameworks such as Text2CAD (Khan et al., 25 Sep 2024), CADmium (Govindarajan et al., 13 Jul 2025), and Drawing2CAD (Qin et al., 26 Aug 2025) enable end-users to specify models via descriptive text, freehand vector drawings, or sketches, automating the synthesis of full CAD programs.
• Interactive and Human-in-the-Loop Design: Systems like Sketch2CAD (Li et al., 2020) leverage auto-completion by inferring sequential design intent from sketch and model context, supporting natural and fluid iterative modeling.
• Dataset and Complexity Scaling: With public releases of large, balanced datasets (DeepCAD, SynthBal, CAD-VGDrawing), future research has increasingly turned to high-complexity designs, multi-modal cross-alignment, and conditional CAD generation with diffusion priors.

Several ongoing challenges remain: reliably reconstructing long and complex command histories, aligning latent spaces across disparate modalities, minimizing invalid or non-editable outputs, and extending capabilities to broader CAD operator vocabularies and multi-part assemblies. A plausible implication is that further progress will require continued integration of multimodal contrastive learning, program synthesis approaches, and refined evaluation metrics that holistically assess both geometric and procedural quality.

7. Representative Methods and Experimental Findings

A summary table of selected approaches and their distinguishing properties is provided below.

Paper (arXiv ID)	Input/Output	Key Features
(Zhang et al., 2016)	Seq→Seq	Cyclic feedback; parameter sharing; >2 BLEU gains
(Jung et al., 2 Apr 2024)	Seq→Seq	Contrastive latent, permutation-invariant; RRE
(Rukhovich et al., 18 Dec 2024)	Point cloud→Python code	LLM decoding; synthetic dataset; code interpretable
(Khan et al., 25 Sep 2024)	Text→Seq	Multi-level annotation; transformer decoder
(Govindarajan et al., 13 Jul 2025)	Text→JSON CAD	Fine-tuned code LLM; sphericity/topology metrics
(Yin et al., 24 Mar 2025)	B-Rep→Seq (RL)	RL gym, revolution ops, multi-metric rewards
(Yu et al., 17 Sep 2025)	Mesh/PC↔Seq	Latent alignment; diffusion priors; SynthBal data
(Qin et al., 26 Aug 2025)	Drawing→Seq	Dual-decoder transformer; vector primitive input

While most recent models employ variants of transformers and diffusion models for sequence modeling and generation, each addresses distinct subproblems in input modality, sequence encoding, data balancing, or evaluation. Experimental results consistently show that approaches using latent alignment, sophisticated augmentation, and hierarchical or dual-stream decoding yield state-of-the-art performance across command/parameter accuracy, geometric fidelity, and robustness, with especially notable gains in reconstructing complex and editable CAD objects.

---

Sequence-to-sequence CAD learning now constitutes a foundational paradigm in both academic and applied design automation, with active research encompassing new input modalities, learning algorithms, evaluation strategies, and human–AI interaction protocols. The integration of program synthesis, multi-modal contrastive pre-training, and large-scale curated datasets is driving both precision and breadth in automated CAD workflows, with further advances likely as models scale and the range of design scenarios expands.