Direct Raw Point Registration

• Direct Raw Point Registration is a technique that estimates rigid-body transforms from unstructured point cloud data without relying on precomputed correspondences.
• Modern methods employ neural architectures, global optimizations, and unsupervised losses to directly minimize geometric misalignments under noise, partial overlap, and outlier conditions.
• These approaches find applications in SLAM, robotics manipulation, and 3D mapping, significantly improving registration speed and accuracy in complex real-world environments.

Direct Raw Point Registration is the problem of estimating a rigid-body transformation (rotation and translation) that aligns two or more point clouds directly from the raw unstructured point coordinates, without relying on precomputed point-to-point correspondences or explicit intermediate feature matching. Modern approaches solve this task via correspondence-less neural architectures, global optimization over transformation parameters, nonparametric geometric losses, or probabilistic models defined on the full raw input sets. This paradigm eliminates or fundamentally reimagines the traditional correspondence-estimation step, resulting in improved robustness to partial overlap, noise, outliers, and nonuniform density.

1. Problem Formulation and Losses

In direct raw point registration, given two sets $$T = \{ t_i = (x_i, f_i) \}_{i=1}^n \subset \mathbb{R}^3 \times \mathbb{R}^c$$ (template) and $$S = \{ s_j = (y_j, g_j) \}_{j=1}^m \subset \mathbb{R}^3 \times \mathbb{R}^c$$ (source), the objective is to estimate a rigid transformation $\hat{T} \in SE(3)$, parameterized by rotation $\hat{R} \in SO(3)$ and translation $\hat{t} \in \mathbb{R}^3$, such that the transformed template best aligns with the source.

Loss formulations in direct approaches include:

• Supervised Dual-Quaternion Loss: For methods such as DeepCLR, transformations are encoded via dual quaternions $\sigma = p + \epsilon q$, with a loss:

$$L = \beta \|\mathbf{p}^* - \mathbf{p}/\|\mathbf{p}\|\|^2_2 + \|\mathbf{q}^* - \mathbf{q}\|^2_2$$

where $\mathbf{p}^*$, $\mathbf{q}^*$ are ground-truth rotation and translation terms, and $\beta$ controls their relative weighting (Horn et al., 2020).

• Chamfer Distance-Based Loss: In unsupervised settings, e.g., Deep-3DAligner, the transformed source is compared to the target via the symmetric Chamfer distance:

$$S = \{ s_j = (y_j, g_j) \}_{j=1}^m \subset \mathbb{R}^3 \times \mathbb{R}^c$$0

(Wang et al., 2020).

• Monte-Carlo Geometric Losses: The random line-intersection metric measures difference in 1D intersection patterns between transformed source and target under random lines $$S = \{ s_j = (y_j, g_j) \}_{j=1}^m \subset \mathbb{R}^3 \times \mathbb{R}^c$$1, producing an expectation-based loss over such lines (Deng et al., 2021).
• Global Inlier Counting: Semi-exhaustive search methods maximize the number of inlier point pairs under a threshold, treating correspondence discovery and registration as a single optimization over the transformation group (Cheng et al., 31 Jan 2025, Li et al., 2023).

2. Algorithmic and Architectural Paradigms

Direct raw point registration admits a spectrum of design regimes:

(a) Deep Correspondence-Less Architectures

DeepCLR exemplifies the use of fully neural, end-to-end frameworks that:

• Downsample and encode input clouds using shared PointNet++ set abstraction blocks.
• Fuse source and template via flow embedding: for each template point, a local max-pooling over source neighbors integrates positional and feature differences, yielding per-point "flow" features.
• Apply global pooling and an MLP regressor to estimate the rigid transform as a dual quaternion, without ever forming explicit point-to-point matches.
• Training and inference are performant (<80 ms per pair), suitable for real-time odometry (Horn et al., 2020).

(b) Semi-Exhaustive and Global Optimization

DSES proposes a direct global optimization over $$S = \{ s_j = (y_j, g_j) \}_{j=1}^m \subset \mathbb{R}^3 \times \mathbb{R}^c$$2:

• Discretize rotations (over Euler angles) and, for each rotation, find the inlier-maximizing translation efficiently via histogram-mode over all pairwise offsets.
• Use top candidate solutions to refine under a true continuous distance metric, with all stages massively parallelized on GPU.
• Achieves provably global optima under chosen norms/gap thresholds and outperforms neural and RANSAC baselines in robustness and recall (Cheng et al., 31 Jan 2025).

Residual Projection + Interval Stabbing further reduces the 6D search to three decoupled 2D (rotation-row + translation-component) subproblems using axis projection and 1D interval stabbing, leading to efficient, deterministic registration even with high outlier rates (Li et al., 2023).

(c) Unsupervised Direct Optimization

Unsupervised frameworks optimize for alignment using only raw inputs and differentiable feature extraction:

• Deep-3DAligner replaces intermediate encoders with a free latent vector (Spatial Correlation Representation) trained jointly per pair, decoded to a transform via MLP, and supervised via Chamfer loss (Wang et al., 2020).
• Differentiable rendering strategies (e.g., UnsupervisedR&R) align 3D point clouds by projecting to RGB-D images, imposing photometric, depth, and geometric consistency losses, and backpropagate through Procrustes/Kabsch (Banani et al., 2021).

(d) Geometric and Physical Analogs

• Minimum Potential Energy (MPE) models the point clouds as interacting under an artificial gravity-inspired potential; the global optimum is found by root-finding on net force/torque, followed by local ICP (Wu et al., 2020).
• Random Line Intersection Loss forgoes pairwise correspondences entirely, measuring alignment by the agreement of 1D intersection patterns with random lines in 3D; used both for direct optimization and as a differentiable loss function in learning-based frameworks (Deng et al., 2021).

(e) Hierarchical, Transformer, and Tree Representations

• RPSRNet employs a Barnes-Hut 2^D-tree to hierarchically partition space, extract multi-scale features via convolutional operations, and fuse cross-cloud information via transformer blocks, culminating in a differentiable SVD pose estimator. An iterative coarse-to-fine pose refinement further boosts accuracy (Ali et al., 2021).

(f) Patch-based Direct Registration

• Approaches such as DPR utilize rotation-invariant and -variant autoencoders for local patches, with the relative latent difference input to a pose regressor for each putative patch correspondence, thereby generating a hypothesis directly from local geometric context (Deng et al., 2019).

3. Training, Inference, and Evaluation Practices

Direct raw point registration methods encompass both supervised and unsupervised protocols, with dataset choices and augmentation strategies spanning:

• Supervised: Learn explicit regression from ground-truth SE(3) transformations, often with dual-quaternion parametrization and rotation-heavy loss weighting (Horn et al., 2020).
• Unsupervised: Directly minimize geometric or photometric consistency terms without pose labels (Wang et al., 2020, Banani et al., 2021, Deng et al., 2021).
• Data: Standard benchmarks include KITTI odometry (LiDAR), ModelNet40 CAD (objects with known transforms), 3DMatch, ScanNet, and large-scale multi-view datasets aggregated from multiple sources (Horn et al., 2020, Wang et al., 2020, Pan et al., 1 Dec 2025).
• Evaluation: Reporting focuses on rotation error (chordal or angle), translation error (Euclidean), recall/success rates under thresholds, and runtime.

For example, DeepCLR on KITTI achieves rotation RMSE as low as 0.0283°, translation RMSE 0.0264 m, at inference times of 55–81 ms. DSES achieves mean MAE(R) = 0.36°, MAE(t) = 0.004 m on ModelNet40, with recall >98%, and 100% success on real robot pose correction tasks (Horn et al., 2020, Cheng et al., 31 Jan 2025).

4. Comparative Strengths and Limitations

Approach	Principal Strengths	Limiting Factors
DeepCLR (Horn et al., 2020)	State-of-the-art runtime and accuracy; no correspondences required	Generalizes best when sufficient overlap and feature richness present
DSES (Cheng et al., 31 Jan 2025)	Globally optimal, very high recall, outlier robustness	Cubic scaling in rotation discretization; less ideal for low overlap; best with coarse init for local mode
Random Line Loss (Deng et al., 2021)	Robust to noise, outliers, and low overlap; unsupervised-friendly	Less effective under severe non-overlap or strong symmetries; sensitive to line count and sampling
RPSRNet (Ali et al., 2021)	Real-time scalability to 250K points, robust to density heterogeneity	Current BH-octree built on CPU; extremely partial scans require further modules
MPE (Wu et al., 2020)	Global convex-like alignment; nonparametric, noise tolerant	O(NM) scaling; initial down-sampling required for very large clouds
Deep-3DAligner (Wang et al., 2020)	No explicit encoder; high generalization to unseen categories	Optimization overhead on test pairs due to per-pair latent updates

Across direct methods, key tradeoffs exist between global optimality, runtime, scalability, degree of supervision, and robustness to low overlap and distribution shift.

5. Practical Applications

Direct raw point registration methods have demonstrated effectiveness across:

• LiDAR Odometry and SLAM: Real-time rigid alignment for frame-to-frame pose estimation, supporting on-line odometry and multi-session/global map merging (notably KITTI, Waymo, Oxford Spires, KAIST) (Horn et al., 2020, Pan et al., 1 Dec 2025).
• Robotics Manipulation: Object model-to-scan alignment for robot pose estimation, grasping, and error correction, with strong performance under partial observability and sensor noise (Cheng et al., 31 Jan 2025).
• Aerial and Mobile Mapping: Integration into rigorous network optimization frameworks, e.g., Dynamic Networks for airborne LiDAR strip adjustment, enabling joint trajectory adjustment, boresight calibration, and sub-decimeter accuracy even with MEMS IMUs or GNSS outages (Brun et al., 2022).
• 3D Scene Understanding: Scene relocalization, bundle adjustment, map merging, and 3D reconstruction for indoor and outdoor settings at varying scales (Pan et al., 1 Dec 2025).
• Unsupervised Learning and Domain Adaptation: Fully unsupervised pipelines for RGB-D/scene registration, enabling training on vast unlabeled scans and generalization to new sensor modalities (Banani et al., 2021, Wang et al., 2020).

6. Ongoing Research and Future Directions

Key avenues for further development in direct raw point registration include:

• Low-Overlap and Partial-to-Partial Alignment: Addressing partial overlap remains challenging for many direct approaches; future work focuses on overlap-aware losses, occlusion modeling, and partial matching mechanisms (Cheng et al., 31 Jan 2025, Ali et al., 2021).
• Scalability Enhancements: GPU-accelerated data structures (e.g., Barnes-Hut, efficient keypoint sampling), dynamic batching for multi-view scenarios, and attention mechanisms offer prospects for further runtime reductions (Pan et al., 1 Dec 2025, Ali et al., 2021).
• Extension to Non-Rigid and Piecewise Rigid Motions: Direct paradigms are being generalized to handle scene flow and category-level deformation by adapting loss structures and neural architectures.
• Integration with Downstream Perception and Planning: Direct methods are increasingly used as SE(3) initialization modules for SLAM, multi-session merging, and collaborative mapping in dynamic multi-robot systems (Pan et al., 1 Dec 2025).
• Domain Generalization: Large-scale, mixed-dataset training has enabled direct models to robustly generalize across domains (CAD, terrestrial LAS, TLS, urban/outdoor) without retraining (Pan et al., 1 Dec 2025).

Ongoing research continues to push the boundaries of correspondence-free and direct point registration, challenging the necessity of handcrafted descriptors and improving robustness and efficiency across broader operating conditions.