PI3DETR: End-to-End 3D Curve Detection

• PI3DETR is a transformer-based framework that detects 3D parametric curves from point clouds using a unified, end-to-end architecture.
• It leverages PointNet++ sampling and multi-layer transformers with geometry-aware matching to optimize detection of Bézier, line, circle, and arc primitives.
• The framework achieves state-of-the-art results (e.g., CD = 0.0024, mAP = 0.8090 on the ABC dataset) and generalizes robustly to real sensor data.

PI3DETR is an end-to-end, transformer-based framework for direct detection of 3D parametric curve instances from raw point clouds. Building upon the set prediction paradigm introduced by 3DETR, PI3DETR eliminates the dependency on intermediate representations and multi-stage refinement common in previous approaches. The model is capable of simultaneously detecting cubic Bézier curves, line segments, circles, and arcs in a unified architecture. PI3DETR introduces a geometry-aware matching mechanism and specialized loss functions that are type-specific, facilitating robust detection under variable noise conditions and diverse sampling densities. The framework achieves state-of-the-art results on the ABC dataset and generalizes effectively to real sensor data, supporting downstream requirements in 3D edge and curve estimation (Oberweger et al., 3 Sep 2025).

1. Architecture Overview

The PI3DETR architecture processes unordered 3D point clouds $P \subset \mathbb{R}^3$ with $N$ points lacking normals or color features. Farthest-point sampling, via a PointNet++ SA module, reduces the input to 2048 seed points, each aggregating local features from 64 neighbors within a fixed radius. Features are projected into a 768-dimensional space before entering a stack of transformer blocks: 3 encoder layers and 9 decoder layers, each equipped with 8 attention heads.

Non-parametric queries, chosen as seed locations by farthest-point sampling and encoded with sine positional embeddings, generate query embeddings for the transformer. Decoder output consists of $K$ embeddings ($K = 128$ during training, up to 512 at inference), each of which feeds into fully connected prediction heads. These heads yield:

• 5-way class probabilities for {no-object, Bézier, line, circle, arc}
• Primitive-specific parameterizations: Bézier control points ($\mathbb{R}^{4\times 3}$), line midpoint/unit direction/length, circle center/unit normal/radius, arc endpoints ($\mathbb{R}^{3\times 3}$)

All outputs are normalized to $[-1,1]^3$ coordinate range. During optimization, a geometry-aware matcher assigns each ground-truth curve uniquely to a query and class, ensuring only relevant heads contribute gradients (Oberweger et al., 3 Sep 2025).

2. Geometry-Aware Matching Strategy

PI3DETR employs a bipartite assignment using the Hungarian algorithm to match predictions and ground-truth curves. For $M$ annotated curves, each matching cost is computed as:

$$\mathcal{C}_\text{match}(j, i) = -\hat{p}_j(c_i) + \mathcal{L}_\text{param}(j, i)$$

where $-\hat{p}_j(c_i)$ encodes negative log-probability of the predicted class, and $N$0 is a type-specific parameter loss. The parameter loss is defined either by a sequence loss for Béziers/arcs, or a hybrid loss for lines/circles, handling order and sign ambiguities. Each query predicts parameters for all classes, but gradients are masked so only the true type’s head is optimized for each matched curve (Oberweger et al., 3 Sep 2025).

3. Specialized Loss Functions

PI3DETR integrates three principal loss components:

• Classification loss ($N$1): class-weighted cross-entropy over five classes.
• Parameter loss ($N$2): sequence or hybrid; sequence is order-invariant L1 for Béziers/arcs, hybrid for lines/circles addresses directional sign ambiguity and scale.
• Chamfer distance loss ($N$3): evaluates sampled points along predicted/ground-truth curves.

For Béziers/arcs, the sequence loss is

$N$4

For lines/circles, the hybrid loss is

$N$5

where $N$6 denotes (midpoint/center, normal/direction, radius/length). The Chamfer distance between two sets $N$7 (each with 64 samples) is also directly employed:

$N$8

This design maintains invariance to direction and sign when necessary and aligns optimization directly with faithful geometric matching (Oberweger et al., 3 Sep 2025).

4. Parametric Curve Representations

Each primitive follows a compact parameterization adapted from the ABC dataset conventions and is normalized into the network’s output space:

• Cubic Bézier: Four 3D control points $N$9
• Line segment: Midpoint, unit direction, positive length
• Circle: Center, unit normal, positive radius
• Arc: Three ordered 3D points (start, mid, end)

At inference, each query outputs a full set of curve parameters for each type with an associated class confidence; geometry-aware matching ensures the appropriate parameterization is used for each ground-truth curve. A plausible implication is that this encoding strategy supports efficient extension to additional parametric curve types (Oberweger et al., 3 Sep 2025).

5. Optional Post-Processing Strategies

PI3DETR remains end-to-end, but two lightweight post-processing methods can further reduce maximum (Hausdorff) error:

• Snap-Fit: Projects samples from each predicted curve to their nearest neighbors in the original point cloud and re-fits the primitive via least-squares for lines/circles, reducing spatial offsets.
• IoU Filter: Samples points from all predictions, computes pairwise set IoU at a fixed tolerance (0.01); overlapping curves of the same class above 0.6 IoU are suppressed based on class score.

These steps require only milliseconds per model, with measurable improvements in maximum error and minimal impact on inference throughput (Oberweger et al., 3 Sep 2025).

6. Implementation Details and Evaluation Protocol

PI3DETR is implemented in PyTorch with a custom transformer stack modeled after DETR. The SAModule uses PointNet++ with hyperparameters (radius: 0.2, 64 neighbors, 256 feature dims), and prediction heads are small FFNs (hidden dim 768 for points, 3 layers for scalars). Training proceeds for 1700 epochs with AdamW, scheduled learning rates, gradient clipping, and strong data augmentations (point dropout, 3D random rotation).

Evaluation is performed on the ABC dataset (10240 meshes, each with ≤100 curves, sampled to 32768 points) and on novel real sensor scans subsampled to 32768 points. Metrics include Chamfer (CD) and Hausdorff (HD) distances (in $K$0), as well as mAP (true positive: matched class and CD < 0.005) (Oberweger et al., 3 Sep 2025).

7. Quantitative Results and Comparative Analysis

PI3DETR consistently outperforms the NerVE (PWL/CAD post-fit) baselines on both clean and degraded test sets. For standard density $K$1, the framework delivers CD = 0.0024, HD = 0.0635, and mAP = 0.8090. NerVE (PWL) achieves CD = 0.0046, HD = 0.1534, and NerVE (CAD post-fit) achieves CD = 0.0401, HD = 0.2478, excluding failed predictions. As density and noise degrade, PI3DETR’s errors increase marginally, in contrast to the rapid performance degeneration of NerVE baselines (e.g., at $K$2: PI3DETR CD=0.0050, HD=0.0857; NerVE CAD CD=0.4562, HD=0.8665 with 347 fails).

Ablations show diminishing returns for increasing queries $K$3 beyond 256, and lightweight post-processing (Snap-Fit, IoU Filter) provides slight but consistent improvements, primarily in worst-case error (HD).

Model	CD (@32768)	HD (@32768)	mAP	Notes
PI3DETR	0.0024	0.0635	0.8090	Snap-Fit postproc
NerVE (PWL)	0.0046	0.1534	—
NerVE (CAD fit)	0.0401	0.2478	—	77 fails (failures excluded)

On both synthetic and real data, PI3DETR detects valid, native parametric curves in every case, establishing a new standard for 3D edge and CAD primitive detection in noisy point clouds (Oberweger et al., 3 Sep 2025).