SynthBal: Synthetic Data Balancing for CAD Programs

• SynthBal is a synthetic dataset generation scheme that balances CAD program datasets across sequence lengths by combining controlled augmentations with stringent validity checks.
• It addresses long-tail imbalances in DeepCAD by reducing the overrepresentation of short programs and improving reconstruction and retrieval performance on complex sequences.
• Empirical results reveal that SynthBal significantly lowers invalid output rates and improves metrics such as Chamfer distance and accuracy, outperforming baseline methods.

Searching arXiv for the specified topic and source paper to ground the article in current literature. SynthBal is a synthetic data augmentation and balancing strategy introduced within the GenCAD-3D framework for CAD program datasets with long-tail complexity distributions. In its precise formulation, it is “a synthetic dataset generation scheme that produces a balanced CAD program dataset across sequence lengths (complexity levels) by combining controlled program-level augmentations with strict validity checking” (Yu et al., 17 Sep 2025). It is designed for CAD programs represented as sketch-and-extrude command sequences, where complexity is defined operationally as sequence length $\ell$, and it addresses a core failure mode of existing CAD generative models: severe overrepresentation of short programs and scarcity of long programs, which degrades reconstruction, retrieval, and geometry-conditioned generation on high-complexity geometries.

1. Definition and motivation

Within GenCAD-3D, SynthBal begins from a “real” CAD program dataset $\mathcal{R}$, derived from valid DeepCAD programs, and constructs a synthetic dataset $\mathcal{S}$ such that the number of samples per sequence length $\ell$ is approximately equal, most samples are synthetic variants of real CAD programs, and every synthetic program is validated by compiling it and checking geometric validity (Yu et al., 17 Sep 2025). The strategy is therefore both a balancing mechanism and a validity-constrained synthetic expansion mechanism.

The need for such balancing is tied to the extreme imbalance in DeepCAD. Sequence lengths range from 3 to 59; programs with length $\le 6$ make up 42% of the dataset; programs with length exactly 6 make up 26%; and long programs such as those with $\ell > 30$ occur only at fractions of a percent each (Yu et al., 17 Sep 2025). Because complexity is defined as sequence length, these statistics imply a long-tail distribution in geometric and procedural complexity. The paper states four consequences of this imbalance: models overfit to simple programs, evaluation metrics hide failures, invalid CAD generations become frequent on complex sequences, and generalization to industrial-like complexity is poor.

A common misconception is to treat SynthBal as a latent-space generator or a diffusion-based synthesizer of CAD. In GenCAD-3D it is neither. It operates purely in CAD program space, without direct use of geometry or latent inputs, and its purpose is to restructure the training distribution seen by the CAD autoencoder. This suggests that its primary novelty is not a new generative architecture, but a complexity-aware data curriculum with validity guarantees.

2. Position in the GenCAD-3D pipeline

GenCAD-3D comprises three main stages: a CAD autoencoder that learns a latent representation $\mathbf{z}_{\mathcal{C}}$ for CAD programs, multimodal encoders with contrastive alignment between geometry latents $\mathbf{z}_{\mathcal{M}}$ and CAD latents $\mathbf{z}_{\mathcal{C}}$, and a conditional latent diffusion prior that maps $\mathbf{z}_{\mathcal{M}}$ to $\mathcal{R}$0 before decoding to a CAD program (Yu et al., 17 Sep 2025). SynthBal operates primarily at Stage 1, but its effects propagate to the contrastive and diffusion stages through the improved CAD latent space.

The integration sequence is explicit. First, invalid programs are filtered from DeepCAD to form $\mathcal{R}$1. Second, SynthBal constructs a balanced synthetic CAD dataset $\mathcal{R}$2 using sequence-length-aware augmentation and validity checking. Two variants are reported: SynthBal with $\mathcal{R}$3, comparable to DeepCAD, and SynthBal-1M with $\mathcal{R}$4 (Yu et al., 17 Sep 2025). Third, a CAD transformer autoencoder is trained on SynthBal or SynthBal-1M and then fine-tuned on a reduction-balanced subset of real DeepCAD, producing SynthBalFT and SynthBal-1MFT autoencoders.

The later stages do not directly use SynthBal-generated geometry. In contrastive training, the SynthBal-trained CAD encoder is frozen and used as the CAD side of contrastive learning, while geometry encoders are trained on the original geometric dataset derived from original DeepCAD. In conditional diffusion training, priors are also trained on the original dataset. At inference time, SynthBal is not directly invoked; its impact appears indirectly through better reconstruction across sequence lengths, improved latent alignment, and fewer invalid CAD outputs (Yu et al., 17 Sep 2025). This design is important: the synthetic balancing is upstream of multimodal alignment and diffusion, but not a replacement for real-data training in those downstream modules.

3. Dataset construction and augmentation mechanics

SynthBal’s input is a real CAD program dataset $\mathcal{R}$5 whose commands are Line, Circle, Arc, Extrude, plus End-of-Sequence. Each program is encoded as a fixed $\mathcal{R}$6 matrix, with the first column representing command type and the next 16 representing discretized parameters; the sequence length $\mathcal{R}$7 is the number of commands (Yu et al., 17 Sep 2025). Let $\mathcal{R}$8 denote the set of possible lengths, such as $\mathcal{R}$9, and let $\mathcal{S}$0 be the number of programs in dataset $\mathcal{S}$1 with length $\mathcal{S}$2.

The balancing objective is uniform over sequence lengths. For a synthetic dataset $\mathcal{S}$3 of size $\mathcal{S}$4, the target per length is

$\mathcal{S}$5

A hyperparameter $\mathcal{S}$6 bounds the maximum ratio of real data per sequence length in $\mathcal{S}$7. For each $\mathcal{S}$8, the number of real samples used is

$\mathcal{S}$9

The remaining slots are filled with synthetic samples obtained by augmenting real programs of the same length (Yu et al., 17 Sep 2025). This construction prevents any sequence length from being dominated by copied real data and forces synthetic coverage especially where real data are scarce.

Two augmentation families are used. The first is noise augmentation, which perturbs continuous parameters of a fraction $\ell$0 of sketch and extrude commands while avoiding discrete or near-discrete fields such as orientations typically at $\ell$1 or $\ell$2. Perturbations are sampled as

$\ell$3

and clipped to $\ell$4 to match DeepCAD quantization. Two noise regimes are defined: large noise with $\ell$5, $\ell$6, and small noise with $\ell$7, $\ell$8 (Yu et al., 17 Sep 2025). The second is Replace-Sketch augmentation: for two programs of the same sequence length, some sketch operations in one program are replaced with corresponding commands from the other, while keeping overall sequence length unchanged.

These augmentations are mixed probabilistically. For each CAD model to augment, there is a 40% probability of applying large noise only, and a 60% probability of applying small noise followed by Replace-Sketch. Only intra-split pairing is allowed, preserving train, validation, and test partitioning and avoiding leakage (Yu et al., 17 Sep 2025).

Validity checking is the critical design choice. Every augmented CAD program is validated offline rather than on-the-fly. OpenCascade is used to interpret the $\ell$9 encoding as CAD commands and compile it to a B-Rep; compilation failure renders the program invalid. The resulting B-Rep is then checked for self-intersections, and intersecting outputs are discarded (Yu et al., 17 Sep 2025). This validity enforcement distinguishes SynthBal from earlier augmentation schemes and is central to the claim that the synthetic data lie on, or close to, the manifold of valid CAD programs.

4. Training protocol and evaluation methodology

SynthBal is a data generation procedure, not a new loss function. Its training role is defined through a two-stage schedule adapted from SYNAuG-style long-tail training: first train on balanced synthetic data, then fine-tune on a balanced real subset (Yu et al., 17 Sep 2025). For the SynthBal autoencoder, Phase 1 uses either SynthBal or SynthBal-1M. For the 170k SynthBal dataset, the reported hyperparameters are 1000 epochs, learning rate $\le 6$0, 2000 warmup steps, and batch size 512. The architecture is a causal transformer decoder that enforces sequential dependency among commands.

Phase 2 fine-tunes on a reduction-balanced subset of DeepCAD containing 4,503 CAD models balanced across sequence lengths by undersampling. The reported hyperparameters are 100 epochs, learning rate $\le 6$1, 200 warmup steps, and batch size 512; all autoencoder parameters remain trainable (Yu et al., 17 Sep 2025). The naming convention is explicit: the SynthBal autoencoder is trained only on SynthBal, SynthBalFT is SynthBal plus fine-tuning, SynthBal-1MFT is SynthBal-1M plus fine-tuning, and the Original autoencoder is trained only on the original unbalanced DeepCAD data.

The two dataset variants differ in scale and real-to-synthetic ratio. For SynthBal, $\le 6$2 and $\le 6$3; 16.7% of programs are copied real and 83.3% are synthetic but valid. For SynthBal-1M, $\le 6$4 and $\le 6$5; 13.9% are copied real and 86.1% are synthetic (Yu et al., 17 Sep 2025). This indicates that the larger dataset contains a higher absolute amount of real data but a lower proportion of real data.

To evaluate long-tail behavior, GenCAD-3D introduces sequence-length-normalized metrics. Given a metric $\le 6$6 on dataset $\le 6$7, with $\le 6$8 the subset of programs of length $\le 6$9, the normalized version is

$\ell > 30$0

The standard average $\ell > 30$1 is dominated by overrepresented short lengths, whereas $\ell > 30$2 assigns equal weight to each sequence length and therefore better exposes performance on complex programs (Yu et al., 17 Sep 2025). This metric design is integral to SynthBal’s evaluation, because many of its gains are most visible only after complexity normalization.

5. Empirical behavior and comparison with baselines

The strongest reported effects of SynthBal are on autoencoder reconstruction quality for high-complexity programs, invalid CAD generation rate, and downstream retrieval and conditional generation (Yu et al., 17 Sep 2025). On sequence-length-normalized autoencoder metrics, the DeepCAD autoencoder reports command accuracy $\ell > 30$3, parameter accuracy $\ell > 30$4, Chamfer distance $\ell > 30$5, IoU$\ell > 30$6, and invalid ratio $\ell > 30$7. The SynthBal-1MFT autoencoder reaches command accuracy$\ell > 30$8, parameter accuracy$\ell > 30$9, Chamfer distance$\mathbf{z}_{\mathcal{C}}$0, IoU$\mathbf{z}_{\mathcal{C}}$1, and invalid ratio$\mathbf{z}_{\mathcal{C}}$2 (Yu et al., 17 Sep 2025).

Relative to ContrastCAD+RRE, described as the best prior baseline in the provided material, SynthBal-1MFT reduces $\mathbf{z}_{\mathcal{C}}$3 from $\mathbf{z}_{\mathcal{C}}$4 to $\mathbf{z}_{\mathcal{C}}$5, which is reported as approximately 58% relative error reduction; reduces Chamfer$\mathbf{z}_{\mathcal{C}}$6 from 4.69 to 2.00, approximately 57% relative reduction; raises parameter accuracy$\mathbf{z}_{\mathcal{C}}$7 from 93.02% to 94.72%, corresponding to approximately 24% error reduction; and raises command accuracy$\mathbf{z}_{\mathcal{C}}$8 from 98.24% to 98.52%, corresponding to approximately 16% error reduction (Yu et al., 17 Sep 2025). Unnormalized metrics also improve: invalid ratio drops from 3.32% in the Original autoencoder to 0.807% in SynthBal-1MFT, while Chamfer improves from 0.772 to 0.734 and parameter accuracy from 97.78% to 98.61%.

The sequence-length-wise behavior is central. DeepCAD and ContrastCAD exhibit degrading command and parameter accuracy and rising invalid ratio for $\mathbf{z}_{\mathcal{C}}$9, whereas SynthBalFT and SynthBal-1MFT maintain higher accuracy and lower invalid ratio across long sequence lengths. At $\mathbf{z}_{\mathcal{M}}$0, DeepCAD command accuracy is reported at approximately 91.13% and invalid ratio at approximately 25.56%, while the SynthBal variants remain substantially better (Yu et al., 17 Sep 2025). This is the clearest empirical confirmation of the claim that average metrics can conceal long-tail failure modes.

Downstream retrieval also benefits. For mesh retrieval with library size $\mathbf{z}_{\mathcal{M}}$1, Top-1 accuracy improves from 80.2% for the Original autoencoder to 81.5% for SynthBalFT and 81.6% for SynthBal-1MFT. For point cloud retrieval at 2048, the improvement is larger: 72.3% to 75.6% and then to 79.4%, corresponding to up to approximately 25% relative error reduction in Top-1 retrieval for point cloud (Yu et al., 17 Sep 2025). Conditional generation and reconstruction likewise improve. In the mesh modality, sequence-length-normalized command accuracy rises from 52.23% with the Original autoencoder to 55.98% with SynthBalFT and 60.10% with SynthBal-1MFT, or 60.48% when SynthBal is also used in contrastive training. Invalid ratios for diffusion models remain high, at $\mathbf{z}_{\mathcal{M}}$2, but reconstruction metrics still improve by command accuracy +9–16%, parameter accuracy +6–8%, IoU +4–5%, and CD -19% relative to the Original autoencoder (Yu et al., 17 Sep 2025).

The baseline comparison clarifies what SynthBal changes. DeepCAD uses random replacement augmentations applied on-the-fly during training, does not enforce balanced sequence lengths, and does not validate synthetic programs. ContrastCAD introduces RRE augmentation—Random Replace-and-Extrude, including replace-extrude, re-extrude, and arc-augment—but also applies on-the-fly augmentation without validity checking (Yu et al., 17 Sep 2025). SynthBal differs by explicit balancing over sequence length, offline augmentation with compile and self-intersection validation, and a two-stage synthetic-then-real training schedule.

6. Limitations, misconceptions, and broader significance

Several limitations are either stated or directly implied in the source material. First, domain coverage is restricted by DeepCAD’s command vocabulary: sketch-and-extrude programs do not cover many industrial CAD operations such as fillets, lofts, revolves, or reference-based features (Yu et al., 17 Sep 2025). SynthBal’s augmentations are therefore limited to that vocabulary, and the synthetic data may not span the full procedural complexity of real engineering design.

Second, sequence length $\mathbf{z}_{\mathcal{M}}$3 is a coarse complexity metric. The paper notes that some long programs may be structurally simple and vice versa, implying that topology, feature interactions, or constraint structure could be more expressive balancing axes (Yu et al., 17 Sep 2025). Third, attempts to apply SynthBal-like ideas directly to geometry representations did not transfer well: training contrastive models on synthetic geometry datasets degraded retrieval because the synthetic geometric distribution diverged too much from real scans and meshes. This is an important caveat against a naive generalization of the method from CAD programs to geometry modalities.

Fourth, SynthBal reduces but does not eliminate invalidity downstream. Conditional diffusion priors still generate a significant fraction of invalid CAD programs, reported at approximately 15–25%, which suggests that the latent manifold still includes invalid regions even when the autoencoder is trained on validity-checked synthetic data (Yu et al., 17 Sep 2025). A plausible implication is that validity-aware dataset construction improves the learned representation but does not by itself impose hard constraints on the diffusion prior.

A further misconception is to conflate SynthBal with a general-purpose notion of “synth balancing” from audio latent-space control. The provided material includes “Universal audio synthesizer control with normalizing flows” (Esling et al., 2019), which addresses organized latent audio spaces, invertible mappings between latent representations and synthesizer parameters, macro-control learning, and preset exploration. That work is technically distinct from SynthBal as defined in GenCAD-3D. The overlap is conceptual rather than terminological: both emphasize structured latent organization and controllable generation, but only GenCAD-3D defines SynthBal as a sequence-length-balanced, validity-checked synthetic dataset construction scheme for CAD programs (Yu et al., 17 Sep 2025).

The broader significance of SynthBal lies in its demonstration that long-tail imbalance in structured generative datasets can be addressed effectively before multimodal alignment and generative prior training. In the language of the paper, balancing and expanding the dataset at the representation-learning stage yields autoencoders that better reconstruct complex programs and drastically reduce invalid outputs, and those benefits propagate to retrieval and conditional generation (Yu et al., 17 Sep 2025). This suggests that, for CAD program generation, the statistical structure of the training corpus is itself a major systems bottleneck, not merely an implementation detail.