FastCAD: Real-Time CAD Retrieval & Alignment
- The paper introduces a single-stage approach that jointly predicts CAD retrieval codes and alignment parameters, achieving a 50x speedup over traditional multi-stage methods.
- FastCAD leverages a learned CAD embedding space refined by contrastive loss, segmentation, and Chamfer distance prediction to improve geometric matching.
- Empirical results demonstrate 61.7% alignment accuracy on RGB-D scans and robust online reconstruction at 10 FPS, highlighting its practical impact for AR and robotics.
Searching arXiv for the primary FastCAD paper and closely related CAD retrieval work to ground the article in current literature. {"query":"(Langer et al., 2024) FastCAD Real-Time CAD Retrieval and Alignment from Scans and Videos"} to=arxiv_search _植物百科通_code {"query":"(Langer et al., 2024) FastCAD Real-Time CAD Retrieval and Alignment from Scans and Videos","max_results":5} FastCAD is a real-time method for CAD model retrieval and alignment from 3D scene observations, introduced for the problem of converting noisy RGB-D scans or video-based reconstructions into clean, CAD-model-based scene representations suitable for augmented reality and robotics. Its central claim is that retrieval and alignment need not be implemented as separate, object-wise stages: instead, a single-stage network can jointly predict alignment parameters and shape embeddings for all detected objects in a scene, yielding a reported inference speedup over prior scan-based methods while matching or exceeding their alignment accuracy on Scan2CAD, and enabling online CAD-based reconstruction from videos at $10$ FPS (Langer et al., 2024).
1. Problem setting and scope
FastCAD addresses CAD model retrieval and alignment in cluttered 3D scenes. The input is a noisy scene representation derived either from an RGB-D scan or from an online video reconstruction; the output is a scene in which objects are represented by aligned CAD models rather than raw partial geometry. In the formulation reported for FastCAD, earlier pipelines were typically decomposed into four expensive stages: object detection or segmentation, separate per-object shape encoding, CAD retrieval, and a second-stage pose optimization. This decomposition was accurate but too slow for online use, especially because each object was processed individually and alignment often depended on iterative optimization (Langer et al., 2024).
The significance of this problem follows from the downstream representation. A CAD assembly is more structured than a raw point cloud or mesh, and the paper explicitly motivates this in terms of clean scene representations for AR and robotics. In that sense, FastCAD belongs to the line of work that uses CAD models as semantic and geometric surrogates for scene structure, but it does so in a regime centered on real-time inference rather than offline registration (Langer et al., 2024).
2. Single-stage retrieval and alignment formulation
The defining architectural move in FastCAD is to treat CAD retrieval and alignment as a single-stage 3D detection problem. A detector operating on the full scene point cloud predicts, for each object, four outputs: object category, an oriented 3D bounding box, a front-facing side, and a CAD shape embedding denoted by . The detector also predicts class probabilities , bounding-box parameters , and front-facing side . At inference time, the predicted embedding retrieves the nearest CAD model in the learned embedding space, while the predicted box and front-facing side determine placement and orientation (Langer et al., 2024).
This formulation eliminates the conventional second optimization stage. Rather than detecting a box, cropping the object, encoding it again, and then optimizing alignment afterward, FastCAD predicts the retrieval code directly from the scene detection head. The paper identifies this as the source of both the runtime gain and the reduction of the distribution-shift problem that affects two-step retrieval pipelines. The result is a scene-level forward pass that jointly handles all detected objects rather than a sequence of object-wise retrieval-and-fit procedures (Langer et al., 2024).
A related implication is that alignment is parameterized in a deliberately economical way. Orientation is not left entirely to the embedding; instead, FastCAD predicts the front-facing side separately. This decomposition reduces the burden placed on the shape code and simplifies the final CAD placement step.
3. Learned CAD embedding space and distillation
FastCAD depends on a learned CAD embedding space shared by noisy partial object scans and clean CAD models. The embedding encoder is trained with a triplet-style contrastive loss in which the anchor is a noisy object point cloud from a real scan, the positive is the CAD model annotated for that object, and the negative is a different CAD model from the same category. The objective encourages the anchor embedding to be closer to the correct CAD than to an incorrect CAD by at least a margin (Langer et al., 2024).
Two auxiliary tasks are added to improve embedding quality. The first is foreground/background segmentation, in which the encoder predicts which points belong to the object rather than the background. The second is prediction of positive-versus-negative CAD similarity via a shallow MLP trained to estimate the Chamfer distance between the positive and negative CADs from their embeddings. The paper’s motivation is that these auxiliary objectives make the embedding reflect not only category membership but also fine-grained geometric similarity (Langer et al., 2024).
After the embedding encoder is trained, embeddings for all CAD models are precomputed. FastCAD is then trained to predict the embedding of the ground-truth CAD directly from the scene, which the paper describes as an embedding distillation step. This step is crucial to the method’s single-stage character: the retrieval code used at test time is no longer obtained by running a separate object encoder, but by regressing the target CAD embedding inside the detector itself (Langer et al., 2024).
4. Training objective, symmetry handling, and inference procedure
The detector is optimized with a composite loss comprising focal loss for class prediction, DIoU loss for bounding-box regression, cross-entropy loss for front-facing side prediction, and MSE loss for embedding regression. Only matched detections contribute the box, front-face, and embedding terms. This training design is the mechanism by which FastCAD learns to emit retrieval-ready scene detections rather than generic detection outputs requiring later refinement (Langer et al., 2024).
Front-facing side prediction is also the method’s main strategy for orientation handling under object symmetry. The paper incorporates the symmetry annotations provided by Scan2CAD, distinguishing non-symmetric, 2-fold, 4-fold, and fully rotationally symmetric objects, and softens the front-face target distribution accordingly. The stated rationale is that explicit front-face prediction is cheaper and more robust than attempting to store multiple discrete CAD orientations in the embedding itself (Langer et al., 2024).
Inference follows a short fixed sequence. FastCAD detects objects in the point cloud, predicts class, box, front side, and embedding for each object, retrieves the nearest CAD model from the learned embedding space for the predicted category, and places the CAD using the predicted box and front-side orientation. When combined with DG-Recon in the video setting, the paper reports a $100$ ms total per-frame update, split into $50$ ms for DG-Recon and $10$0 ms for FastCAD, enabling online CAD-based reconstruction at $10$1 FPS (Langer et al., 2024).
5. Empirical results and ablation findings
FastCAD is evaluated in three regimes: standard scan-based evaluation, zero-shot evaluation, and video-based evaluation. On Scan2CAD alignment from RGB-D scans, FastCAD reports 61.7% instance alignment accuracy, compared with 61.2% for SceneCAD, while reducing runtime from 2600 ms to 50 ms. In the video setting, FastCAD reports 48.2% alignment accuracy, compared with 43.0% for RayTran, 38.6% for Vid2CAD, and 29.2% for ODAM. The paper further states that video alignment accuracy improves from 43.0% to 48.2% and reconstruction accuracy from 22.9% to 29.6% when FastCAD is used in the video pipeline (Langer et al., 2024).
| Setting | Key quantitative result | Runtime |
|---|---|---|
| RGB-D scans | 61.7% instance alignment accuracy; 41.7% reconstruction accuracy; 83.1% shape accuracy | 50 ms |
| Video pipeline | 48.2% alignment accuracy; 24.7% reconstruction accuracy; 79.8% shape accuracy | 100 ms total per frame |
| Video with same retrieval setup as Vid2CAD | 29.6% reconstruction accuracy; 87.7% shape accuracy | 100 ms total per frame |
The paper also introduces Scan2CAD reconstruction accuracy and Scan2CAD shape accuracy to evaluate the quality of the retrieved CAD model itself, not only its pose. For scans, the reported values are 41.7% reconstruction accuracy and 83.1% shape accuracy. For videos, the reported values are 24.7% reconstruction accuracy and 79.8% shape accuracy; under the same retrieval setup as Vid2CAD, these improve to 29.6% and 87.7%, respectively (Langer et al., 2024).
The ablations isolate the method’s critical components. A two-step pipeline that first predicts a bounding box and then runs a separate retrieval encoder performs substantially worse than embedding distillation: reported reconstruction accuracy is 15.6% for predicted-bbox retrieval, 30.6% for nearest ground-truth-bbox retrieval, and 41.7% for direct embedding distillation. Additional ablations show that contrastive learning, segmentation, and Chamfer prediction improve the embedding; a Perceiver encoder improves reconstruction and shape accuracy relative to PointNet++; and separate front-face prediction with symmetry information improves alignment relative to encoding multiple orientations inside the embedding (Langer et al., 2024).
6. Position in the CAD literature and conceptual boundaries
FastCAD belongs to the retrieval-and-alignment branch of CAD-based scene understanding rather than to text-to-CAD generation, parametric primitive analysis, or executable CAD-program synthesis. This distinction is often blurred in broader CAD-automation surveys. Systems such as OpenECAD, CAD-Coder, and FutureCAD generate editable CAD commands or scripts from images or language, whereas FastCAD retrieves and aligns existing CAD models from scene geometry (Yuan et al., 2024, He et al., 13 May 2025, Li et al., 12 Mar 2026).
Within retrieval-based scene reconstruction, FastCAD can be situated between earlier CAD-prior reconstruction pipelines and search-based retrieval accelerators. Earlier work on CAD priors for instance reconstruction used a roughly matching prior CAD model as a proxy for detection, segmentation, pose-graph construction, and subsequent refinement in dynamic clutter, but was framed around accurate instance reconstruction from multiple point clouds rather than real-time scene-wide retrieval (Birdal et al., 2017). HOC-Search, by contrast, accelerated CAD retrieval from RGB-D scans through a search tree and a Monte Carlo Tree Search–inspired procedure under a render-and-compare objective, but still assumed an approximate 3D bounding box and searched over candidate CADs and poses rather than predicting retrieval codes and alignment parameters directly in one pass (Ainetter et al., 2023).
A common misconception is therefore to treat FastCAD as a general-purpose CAD generation framework. It is not. Its contribution is narrower and more specific: real-time retrieval and placement of CAD exemplars for objects already present in a scanned or reconstructed scene. A plausible implication is that its principal value lies where online geometric scene understanding is more important than generative editability—especially AR, robotics, and CAD-model-based mapping. Another source of confusion is nomenclature: the name “FAST-CAD” has also been used by an unrelated fairness-aware framework for non-contact stroke diagnosis, but that usage is outside the 3D scene digitization literature (Sha et al., 12 Nov 2025).
