CAD-Coder: An Open-Source Vision-Language Model for Computer-Aided Design Code Generation (2505.14646v1)

Abstract: Efficient creation of accurate and editable 3D CAD models is critical in engineering design, significantly impacting cost and time-to-market in product innovation. Current manual workflows remain highly time-consuming and demand extensive user expertise. While recent developments in AI-driven CAD generation show promise, existing models are limited by incomplete representations of CAD operations, inability to generalize to real-world images, and low output accuracy. This paper introduces CAD-Coder, an open-source Vision-LLM (VLM) explicitly fine-tuned to generate editable CAD code (CadQuery Python) directly from visual input. Leveraging a novel dataset that we created--GenCAD-Code, consisting of over 163k CAD-model image and code pairs--CAD-Coder outperforms state-of-the-art VLM baselines such as GPT-4.5 and Qwen2.5-VL-72B, achieving a 100% valid syntax rate and the highest accuracy in 3D solid similarity. Notably, our VLM demonstrates some signs of generalizability, successfully generating CAD code from real-world images and executing CAD operations unseen during fine-tuning. The performance and adaptability of CAD-Coder highlights the potential of VLMs fine-tuned on code to streamline CAD workflows for engineers and designers. CAD-Coder is publicly available at: https://github.com/anniedoris/CAD-Coder.

• The paper introduces CAD-Coder, an open-source vision-language model trained on the GenCAD-Code dataset to generate editable CAD code directly from images.
• CAD-Coder achieves 100% syntax validity on a tested subset and outperforms state-of-the-art models like GPT-4.5 and Qwen2.5-VL-72B in generating syntactically correct CAD code.
• This model significantly reduces manual effort and time required for CAD modeling by converting visual information into operable scripts, making design more accessible.

CAD-Coder: Vision-LLM for Automated CAD Code Generation

The paper "CAD-Coder: An Open-Source Vision-LLM for Computer-Aided Design Code Generation" presents an academic contribution towards automating the creation of CAD models via machine learning methodologies. Exploring the landscape of computational design automation, the authors introduce CAD-Coder, a vision-LLM specifically trained to generate computer-aided design code from visual inputs. This endeavor addresses challenges such as the high level of expertise and significant time investment traditionally required for manual CAD modeling.

In engineering design, model precision and editability are paramount. Conventional workflows demand manual sketching, constraints definition, and body extrusions—skills honed through extensive practice. While AI-driven CAD generation offers potential efficiencies, it has been encumbered by poor real-world applicability and incomplete operational definitions. CAD-Coder seeks to overcome these barriers by deploying a Vision-LLM (VLM) paradigm fine-tuned for CAD tasks, relying on data from GenCAD-Code, a substantial dataset comprising over 163,000 pairs of CAD model images and corresponding codes.

The CAD-Coder model notably surpasses state-of-the-art baselines like GPT-4.5 and Qwen2.5-VL-72B in delivering syntactically valid outputs consistently. It achieves a 100% syntax validity rate within the tested subset, significantly reducing error-prone outputs—a known issue with existing models. Furthermore, CAD-Coder demonstrates high performance on 3D solid geometrical similarity metrics, suggesting robustness in generating accurate and editable CAD code.

At its core, CAD-Coder employs a two-stage training regimen to integrate an image encoder with a LLM, utilizing pre-trained architectures such as CLIP-ViT-L-336px for vision tasks and Vicuna-13B-v1.5 for language tasks. This approach ensures effective feature alignment and domain-specific fine-tuning, achieving optimal deployment in generating CadQuery code from image cues.

The paper also highlights CAD-Coder's potential generalizability, briefly demonstrated through its application to images of real-world objects—a scenario not explicitly covered by its training dataset. However, this facet remains a point for future expansion as further refinement is necessary to handle diverse object perspectives and lighting conditions effectively.

While CAD-Coder shows promise with unseen CAD operations, challenges remain, particularly around the model's ability to leverage operations not explicitly seen during fine-tuning. Early tests reveal that careful optimization of learning rates and training strategies can help retain the pre-trained model's breadth of knowledge, thereby broadening functionality.

Among practical implications, CAD-Coder's brilliance lies in its ability to convert visual information directly into operable CAD scripts, vastly reducing manual inputs in the CAD creation process. For practitioners, this advancement may translate to decreases in production time and errors, plus an enhancement of experimentation capabilities and design accessibility for less experienced users.

In conclusion, CAD-Coder is positioned as a forward-looking solution for integrating AI into CAD workflows. It brings together sophisticated VLM-based machine learning to address historic inefficiencies in computational design. The model's ability to operate across variable domain inputs opens notable pathways for future research, especially in enhancing real-world image translation accuracy and expanding model versatility across diverse design paradigms. The open-source release facilitates ongoing community engagement, promising collaborative enhancements in the domain of AI-driven CAD automation.