PI3DETR: Unified 3D Curve Detection

• PI3DETR is a transformer-based approach that unifies the detection of diverse 3D curve primitives (cubic Bézier, line, circle, arc) directly from point clouds.
• It employs a geometry-aware matching strategy with type-specific loss functions to optimize heterogeneous parameterizations in a single forward pass.
• The method demonstrates state-of-the-art performance and robustness on benchmarks, with potential applications in CAD, robotics, and advanced 3D sensing.

Searching arXiv for the PI3DETR paper and closely related work. PI3DETR denotes “Parametric Instance Detection of 3D Point Cloud Edges with a Geometry-Aware 3DETR,” an end-to-end framework for directly predicting 3D parametric curve instances from raw point clouds (Oberweger et al., 3 Sep 2025). It extends 3DETR, the transformer-based 3D detector for point clouds (Misra et al., 2021), to unify the detection of cubic Bézier curves, line segments, circles, and arcs in a single forward pass (Oberweger et al., 3 Sep 2025). The method is formulated to avoid intermediate representations and multi-stage processing, while introducing a geometry-aware matching strategy and specialized loss functions for differently parameterized curve types (Oberweger et al., 3 Sep 2025). Within the broader literature, the name is distinct from Pi3DET, a separate benchmark and cross-platform 3D object-detection framework for LiDAR-based robotic platforms (Liang et al., 23 Jul 2025).

1. Definition and scope

PI3DETR is a 3D point-cloud method for parametric instance detection of curve primitives rather than 3D bounding boxes or semantic segments (Oberweger et al., 3 Sep 2025). Its target output is a set of curve instances, each assigned a class and a type-specific parameterization, predicted directly from unstructured input geometry (Oberweger et al., 3 Sep 2025). The supported primitive families are cubic Bézier curves, line segments, circles, and arcs, and these are handled jointly by one model rather than by separate pipelines (Oberweger et al., 3 Sep 2025).

The central problem addressed by PI3DETR is that prior approaches to 3D edge and curve extraction often depend on intermediate encodings, such as voxel grids, range images, or point-wise segmentation, followed by fitting or post hoc refinement stages (Oberweger et al., 3 Sep 2025). PI3DETR replaces that decomposition with a direct set-prediction formulation derived from the DETR paradigm and adapted to point clouds through 3DETR (Misra et al., 2021). This places the method in the line of end-to-end transformer detectors, but specialized for geometric primitives with heterogeneous parameter spaces (Oberweger et al., 3 Sep 2025).

A plausible implication is that PI3DETR occupies a hybrid position between object detection and geometric reconstruction. It uses detection-style bipartite matching and query-based prediction, yet its outputs are explicit parametric structures suitable for downstream CAD, robotics, and 3D sensing pipelines (Oberweger et al., 3 Sep 2025).

2. Architectural design

PI3DETR is built on 3DETR, which adapts the DETR framework to point clouds (Misra et al., 2021). The input is a 3D point cloud containing only XYZ coordinates (Oberweger et al., 3 Sep 2025). Downsampling is performed by a Set Aggregation Module with Farthest Point Sampling, and the sampled points are processed by a PointNet++ neighborhood embedder to extract local geometric features (Oberweger et al., 3 Sep 2025). These features are then passed through a transformer encoder-decoder stack (Oberweger et al., 3 Sep 2025).

Unlike image-based DETR variants with fixed learned queries, PI3DETR uses actual point locations sampled via Farthest Point Sampling as queries, together with sine positional embeddings for encoder outputs and input queries (Oberweger et al., 3 Sep 2025). The decoder produces per-query embeddings $o_j^\mathrm{f}$, each of which acts as a proposal for a potential curve instance (Oberweger et al., 3 Sep 2025).

For each query, PI3DETR predicts both a class distribution and multiple type-specific parameter vectors (Oberweger et al., 3 Sep 2025). The class head outputs probabilities over $$\{\text{no-object}, \text{Bézier}, \text{line}, \text{circle}, \text{arc}\}$$ (Oberweger et al., 3 Sep 2025). Separate parameter heads produce:

• Cubic Bézier: $4$ control points in $\mathbb{R}^{4\times 3}$ (Oberweger et al., 3 Sep 2025).
• Line segment: midpoint in $\mathbb{R}^{3}$, direction vector in $\mathbb{R}^{3}$ with unit length, and a length scalar (Oberweger et al., 3 Sep 2025).
• Circle: center in $\mathbb{R}^{3}$, normal in $\mathbb{R}^{3}$ with unit length, and a radius scalar (Oberweger et al., 3 Sep 2025).
• Arc: start, midpoint, and end in $\mathbb{R}^{3\times 3}$ (Oberweger et al., 3 Sep 2025).

A defining design choice is that all heads predict every type for every query, while the training and matching machinery ensures that each query is optimized only for the matched ground-truth type (Oberweger et al., 3 Sep 2025). This is the mechanism that makes unified multi-primitive detection possible without separate specialized detectors.

3. Geometry-aware matching and supervision

The main technical novelty of PI3DETR is its geometry-aware matching strategy for assigning predicted queries to ground-truth curve instances (Oberweger et al., 3 Sep 2025). For each ground-truth curve $i$ with class $$\{\text{no-object}, \text{Bézier}, \text{line}, \text{circle}, \text{arc}\}$$0, and for each prediction $$\{\text{no-object}, \text{Bézier}, \text{line}, \text{circle}, \text{arc}\}$$1, the matching cost is defined as

$$\{\text{no-object}, \text{Bézier}, \text{line}, \text{circle}, \text{arc}\}$$2

where $$\{\text{no-object}, \text{Bézier}, \text{line}, \text{circle}, \text{arc}\}$$3 is the negative class score for the correct primitive type and $$\{\text{no-object}, \text{Bézier}, \text{line}, \text{circle}, \text{arc}\}$$4 is a type-dependent parameter loss (Oberweger et al., 3 Sep 2025). The global assignment is then obtained by the Hungarian algorithm, following the DETR family’s bipartite matching formulation (Oberweger et al., 3 Sep 2025, Misra et al., 2021).

The parameter loss depends on the primitive class:

$$\{\text{no-object}, \text{Bézier}, \text{line}, \text{circle}, \text{arc}\}$$5

For Bézier curves and arcs, PI3DETR uses an order-invariant sequence loss:

$$\{\text{no-object}, \text{Bézier}, \text{line}, \text{circle}, \text{arc}\}$$6

which accounts for reversed orientation of ordered control-point sequences (Oberweger et al., 3 Sep 2025). For lines and circles, it uses a hybrid loss that is sign-invariant with respect to direction or normal vectors and adds an $$\{\text{no-object}, \text{Bézier}, \text{line}, \text{circle}, \text{arc}\}$$7 penalty on the scalar parameter:

$$\{\text{no-object}, \text{Bézier}, \text{line}, \text{circle}, \text{arc}\}$$8

This formulation encodes the geometric symmetries specific to each primitive family rather than forcing all curves into a single undifferentiated regression space (Oberweger et al., 3 Sep 2025). A plausible implication is that the matching process is not merely a training convenience but a representation-level mechanism: it defines the equivalence classes under which predicted geometric objects are considered correct.

4. Loss functions and optimization objective

The total supervised loss for a matched prediction-ground-truth pair is

$$\{\text{no-object}, \text{Bézier}, \text{line}, \text{circle}, \text{arc}\}$$9

where $4$0 is cross-entropy over class labels, $4$1 is the geometry-aware parameter loss, and $4$2 is a curve-aware Chamfer Distance between uniformly sampled points on the predicted and ground-truth curves (Oberweger et al., 3 Sep 2025).

The Chamfer term is defined as

$4$3

and is computed using $4$4 uniformly sampled points along both predicted and ground-truth curves (Oberweger et al., 3 Sep 2025). Hausdorff Distance is additionally used for evaluation as a bound on worst-case pointwise mismatch (Oberweger et al., 3 Sep 2025).

To address class imbalance, PI3DETR uses class weights computed from class counts $4$5:

$4$6

The training setup also includes auxiliary supervision at all decoder layers and label smoothing for classes (Oberweger et al., 3 Sep 2025). These details situate PI3DETR within contemporary transformer optimization practice while preserving the centrality of geometry-specific supervision.

5. Supported primitive families and inference behavior

A key property of PI3DETR is that it unifies multiple parametric object families with incompatible parameterizations under one decoder and one matching framework (Oberweger et al., 3 Sep 2025). This differs from pipelines that either fit primitives after segmentation or train separate models for each geometric type.

The four primitive classes differ substantially in their parameter spaces. Cubic Bézier curves require four control points, arcs are represented by start, midpoint, and end points, line segments use midpoint-direction-length, and circles use center-normal-radius (Oberweger et al., 3 Sep 2025). PI3DETR does not collapse these to a common surrogate representation. Instead, it retains native parameterizations and resolves the heterogeneity at the level of matching and loss design (Oberweger et al., 3 Sep 2025).

During training, each query predicts all primitive types, but only the type corresponding to the matched ground-truth instance contributes supervision for that query (Oberweger et al., 3 Sep 2025). During inference, the class head selects the likely type and the corresponding parameter head provides the curve parameters (Oberweger et al., 3 Sep 2025). This preserves end-to-end inference while avoiding the need for explicit branching logic at deployment time.

A common misconception is that “unified prediction” necessarily means a single homogeneous regression head. PI3DETR does not adopt that strategy. Its unification is architectural and objective-level: one model, one query set, one matching procedure, and multiple type-specific heads trained jointly (Oberweger et al., 3 Sep 2025). This distinction is important for interpreting both its robustness claims and its extensibility.

6. Post-processing, robustness, and empirical behavior

PI3DETR includes two optional post-processing procedures: Snap & Fit and an IoU Filter (Oberweger et al., 3 Sep 2025). Snap & Fit samples points along each predicted curve, snaps them to nearest neighbors in the raw point cloud, and refits the curve using a class-specific fitting method (Oberweger et al., 3 Sep 2025). The IoU Filter removes overlapping or duplicate predictions by sampling points along predicted curves and computing a set-wise Intersection over Union based on proximity; for same-type curves with overlap above a threshold such as IoU $4$7, the lower-confidence prediction is removed (Oberweger et al., 3 Sep 2025).

These steps are explicitly optional. The reported method achieves high performance without them, although they can marginally improve metrics, especially Hausdorff Distance (Oberweger et al., 3 Sep 2025). This is significant because it preserves the claim of end-to-end direct prediction rather than concealing essential fitting stages behind a nominal detector.

The paper reports robustness under varying point density and additive Gaussian noise (Oberweger et al., 3 Sep 2025). PI3DETR remains valid at densities from $4$8 down to $4$9 points, while NerVE is described as producing more failures as density decreases (Oberweger et al., 3 Sep 2025). Under Gaussian noise with standard deviation up to $\mathbb{R}^{4\times 3}$0, where $\mathbb{R}^{4\times 3}$1 is point cloud size, PI3DETR maintains lower Hausdorff Distance and often lower Chamfer Distance than NerVE, together with a drastic reduction in invalid outputs (Oberweger et al., 3 Sep 2025). The stated explanation is that direct global end-to-end regression reduces dependence on voxel occupancy or intermediate segmentation, improving resilience to non-uniform and sparse point clouds (Oberweger et al., 3 Sep 2025).

The reported real-data behavior is also framed in terms of generalization: PI3DETR generalizes to real sensor data from robotic structured-light scans without retraining or domain adaptation (Oberweger et al., 3 Sep 2025). This suggests that the primitive-level objective may induce invariances that are not tightly coupled to a single synthetic data distribution, though that interpretation goes beyond the explicit claims.

7. Benchmark performance, implementation, and relation to adjacent work

On the ABC dataset test set with $\mathbb{R}^{4\times 3}$2 points, PI3DETR reports the following results relative to NerVE (Oberweger et al., 3 Sep 2025):

Metric	NerVE (CAD)	NerVE (PWL)	PI3DETR
Chamfer $\mathbb{R}^{4\times 3}$3	0.0401 (77 fail)	0.0046	0.0024
Hausdorff $\mathbb{R}^{4\times 3}$4	0.2478 (77 fail)	0.1534	0.0635
mAP $\mathbb{R}^{4\times 3}$5	--	--	0.8090

These numbers are presented as a new state of the art on the ABC dataset and as evidence that the method produces structurally interpretable curve outputs rather than failure-prone reconstructions (Oberweger et al., 3 Sep 2025).

The implementation details reported for PI3DETR are specific. The Set Aggregation Module downsamples to $\mathbb{R}^{4\times 3}$6 points and gathers $\mathbb{R}^{4\times 3}$7 neighbors in a ball of radius $\mathbb{R}^{4\times 3}$8 (Oberweger et al., 3 Sep 2025). The transformer configuration uses $\mathbb{R}^{4\times 3}$9 encoders, $\mathbb{R}^{3}$0 decoders, $\mathbb{R}^{3}$1 heads, and an FFN hidden dimension of $\mathbb{R}^{3}$2, with LayerNorm and ReLU throughout (Oberweger et al., 3 Sep 2025). The optimizer is AdamW, with learning-rate warm-up, a $\mathbb{R}^{3}$3-epoch training schedule, gradient clipping, and data augmentation including point dropout and random rotations (Oberweger et al., 3 Sep 2025). With $\mathbb{R}^{3}$4 queries and $\mathbb{R}^{3}$5 input points, the end-to-end pipeline with Snap & Fit runs at approximately $\mathbb{R}^{3}$6 seconds per sample on an RTX 4090 (Oberweger et al., 3 Sep 2025).

PI3DETR’s immediate antecedent is 3DETR, from which it inherits the query-based transformer formulation for point clouds (Misra et al., 2021). A later system, PIRATR, is explicitly described as extending PI3DETR to multi-class 6-DoF pose and parametric attribute estimation for robotic objects in sparse, occlusion-affected LiDAR data (Schwingshackl et al., 5 Feb 2026). PIRATR retains the modular class-specific head design and geometry-aware matching logic while shifting the prediction target from parametric curve primitives to object poses and task-relevant parameters such as gripper opening (Schwingshackl et al., 5 Feb 2026). This suggests that PI3DETR’s deeper contribution is not only its benchmark performance on curve detection, but also a general design pattern for set-prediction over heterogeneous geometric parameterizations.