LiDAR Bundle Adjustment Techniques

• LBA is the process of jointly optimizing sensor poses and scene geometry to produce globally consistent LiDAR maps.
• It leverages diverse formulations such as point-to-plane, polynomial surface smoothing, and semantic probabilistic models to overcome sparse data challenges.
• State-of-the-art methods ensure scalability and accuracy via analytic elimination, robust optimization back-ends, and integration with inertial and camera data.

Searching arXiv for recent LiDAR bundle adjustment papers to ground the article in published work. arXiv search: query="LiDAR bundle adjustment", max_results=10 LiDAR Bundle Adjustment (LBA) is the simultaneous determination of sensor poses and scene geometry for LiDAR mapping. In practical mapping pipelines, the central objective is to align multiple scans so that the reconstructed point cloud is globally consistent rather than merely pairwise consistent. Within the recent literature, LBA spans pose-only formulations in which geometric features are analytically eliminated, progressive spatial smoothing with polynomial surface constraints, photometric formulations on LiDAR range images, surfel- and semantic-GMM-based joint structure optimization, tightly coupled LiDAR–inertial and LiDAR–camera objectives, and scalable solvers designed for large-scale or distributed mapping (Liu et al., 2022, Liu et al., 2022, Li et al., 2024, Ćwian et al., 7 Jan 2025).

1. Problem definition and relation to pose-graph optimization

A standard distinction in the literature is between LiDAR bundle adjustment and pose-graph optimization. In the hierarchical formulation of large-scale mapping, LBA is defined as directly minimizing the geometric inconsistency of the reconstructed point cloud by optimizing all sensor poses jointly so as to reduce summed point-to-plane distances across the map, whereas pose-graph optimization uses pairwise relative-pose constraints and therefore does not directly optimize the mapping consistency (Liu et al., 2022). In one common form,

$$E_{\rm BA}(\{T_i\}) = \sum_{i=1}^N \sum_{p \in \mathcal{P}_i} \bigl(n_p^\top(R_i\,p + t_i - c_p)\bigr)^2,$$

with $T_i=(R_i,t_i)\in SE(3)$, feature points $\mathcal{P}_i$, and associated plane parameters $(n_p,c_p)$ (Liu et al., 2022).

A persistent difficulty is that LiDAR BA cannot generally rely on exact point matching in the way classical visual BA relies on repeated landmark observations. BALM states this explicitly: in lidar SLAM, BA had been hard to use because sparse feature points such as edge and plane points make the exact point matching impossible (Liu et al., 2020). This has driven the development of indirect residuals such as point-to-plane, point-to-line, point-to-polynomial-surface, photometric warp residuals, and probabilistic soft-assignment objectives rather than explicit point identity constraints.

Another defining property of LBA is that it is not tied to a single temporal scope. The literature includes local sliding-window refinement for odometry back-ends, global refinement for entire trajectories, hierarchical decompositions for large maps, and continuous-time formulations that model ego-motion within a single scan (Liu et al., 2020, Liu et al., 2022). This suggests that “bundle adjustment” in the LiDAR setting is better understood as a family of joint geometric estimators than as a single fixed algorithmic template.

2. Core objective functions and analytic elimination

The canonical geometric formulation uses planes and edges as latent scene features. BALM defines, for a set of transformed world-frame points $p_i = R_{s_i}p_{f_i}+t_{s_i}$, the plane feature cost

$$J_{\rm plane}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \bigl(n^\top(p_i-q)\bigr)^2,$$

and the edge feature cost

$$J_{\rm edge}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \|(I-nn^\top)(p_i-q)\|^2.$$

A central result is that, for fixed poses $T$, the optimal feature parameters are available in closed form from the covariance matrix $A$ of the transformed points: the minimized plane cost is $\lambda_3(A)$, and the minimized edge cost is $T_i=(R_i,t_i)\in SE(3)$0. The global problem therefore becomes pose-only optimization over the sum of these eigenvalue-based feature costs (Liu et al., 2020).

The paper on efficient and consistent bundle adjustment on lidar point clouds retains the same analytic elimination principle but reformulates the computation with “point clusters” and “point cluster coordinates.” A cluster is summarized by a $T_i=(R_i,t_i)\in SE(3)$1 symmetric matrix

$T_i=(R_i,t_i)\in SE(3)$2

which allows rigid transformation and merging to be implemented as matrix operations. The resulting solver derives closed-form Jacobians and Hessians, states the gauge-related null spaces, and propagates raw-point noise to pose covariance through the reduced Hessian (Liu et al., 2022).

A distinct second-order formulation appears in RSO-BA, which introduces the Mean Squared Group Metric (MSGM). Instead of processing every plane-supporting point individually during optimization, the method pre-accumulates a group matrix $T_i=(R_i,t_i)\in SE(3)$3 and expresses the normalized group cost as

$T_i=(R_i,t_i)\in SE(3)$4

Robust kernels are then applied to these group metrics, and an explicit second-order estimator is built from analytic gradients and Hessians (Ma et al., 2024).

Across these formulations, a recurrent structural choice is whether geometry should be optimized numerically or removed analytically. Pose-only eigenvalue formulations remove explicit feature variables, whereas group-metric or joint-structure formulations retain feature blocks. The literature does not present one choice as universally dominant; rather, each choice is tied to assumptions about scalability, observability, and map representation.

3. Scene representations and correspondence models

The most visible evolution in LBA concerns how the scene is represented and how cross-scan correspondence is induced. Early and still highly influential systems rely on plane and edge features, but later work broadens the representational vocabulary substantially.

Representation	Residual or association model	Representative papers
Plane and edge features	Point-to-plane, point-to-line, eigenvalue feature costs	(Liu et al., 2020, Liu et al., 2022, Liu et al., 2022)
Polynomial surfaces	Point-to-polynomial-surface residuals with progressive smoothing	(Li et al., 2024, Li et al., 2024)
Surfels	Point-to-plane residuals on surfel centers displaced along normals	(Ćwian et al., 7 Jan 2025)
Semantic GMM landmarks	Semantic Gaussian mixture likelihood with soft assignments	(Ji et al., 2024)
Photometric range-image model	Intensity, depth, and normal image residuals without explicit landmarks	(Giammarino et al., 2023)

The plane-dominant lineage remains important because it provides compact geometry and efficient residuals, but its limitations are stated directly in the literature. PSS-BA notes that current solutions, including LiDAR bundle adjustment, predominantly depend on the local plane assumption, which may be inadequate in complex environments lacking of planar geometries or substantial initial pose errors. Its response is a progressive spatial smoothing module that estimates robust normals, constructs local tangent frames, and fits second-order polynomial surfaces

$T_i=(R_i,t_i)\in SE(3)$5

with Gaussian weighting across scales before pose adjustment (Li et al., 2024).

The correspondence problem has also been attacked at the preprocessing level. The voxel-octree plane extraction method proposed specifically for LiDAR BA partitions space into root voxels, recursively splits them with an octree, tests planarity through a PCA-based quartering criterion, and merges nearby patches to avoid excessive numbers of small planes. On the HILTI dataset, the method reported the lowest mean ATE among the compared plane extraction methods, with mean ATE $T_i=(R_i,t_i)\in SE(3)$6 cm and total runtime $T_i=(R_i,t_i)\in SE(3)$7 s (Liu et al., 2023).

Two later directions relax the dependence on predefined geometric primitives. SGBA argues that existing works usually rely on predefined geometric features for landmark representation and that this restricts generalizability. It models the environment as a semantic Gaussian mixture model with landmarks $T_i=(R_i,t_i)\in SE(3)$8, semantic compatibility constraints, and posterior assignment probabilities

$T_i=(R_i,t_i)\in SE(3)$9

An adaptive semantic selection framework then evaluates the condition number of the linearized system and retains only informative semantic clusters (Ji et al., 2024).

MAD-BA takes the opposite route of explicit geometric structure refinement. It represents the map as surfels $\mathcal{P}_i$0, optimizes a scalar offset $\mathcal{P}_i$1 along each surfel normal, and weights each point-to-plane residual by a generalized LiDAR uncertainty estimate $\mathcal{P}_i$2:

$\mathcal{P}_i$3

The paper frames this as a response to current LiDAR systems that often prioritize pose optimization while omitting structure refinement or treating it separately (Ćwian et al., 7 Jan 2025).

A further departure is photometric LiDAR BA. The photometric LiDAR and RGB-D formulation treats LiDAR scans as intensity, depth, and normal images, defines warping residuals over an overlap graph, and states that it is free from data association and without making assumptions about the environment (Giammarino et al., 2023). A plausible implication is that the notion of “landmark” in LBA has become optional rather than essential.

4. Optimization back-ends, consistency, and scalability

Levenberg–Marquardt and Gauss–Newton remain the dominant nonlinear solvers, but modern LBA research devotes substantial attention to how the linear systems are structured, reduced, or approximated. BALM derives analytical derivatives up to second order and reports that a local BA on a sliding window of 20 scans can run in real time at $\mathcal{P}_i$4 Hz when incorporated into a LOAM back-end (Liu et al., 2020).

Scalability is the primary pressure point for global mapping. Hierarchical LiDAR BA addresses the cubic cost of full global optimization through bottom-up local BA in overlapping windows and a top-down pose graph update. With proper setups, the method reports that it could generate a globally consistent map with around $\mathcal{P}_i$5 of the sequence time (Liu et al., 2022). PSS-GOSO attacks the same issue differently: it introduces graph optimality-aware sparsification, stochastic clustering, and Schur-complement-based graph marginalization. On a cross-platform scene, the paper reports that the full graph had $\mathcal{P}_i$6 edges and $\mathcal{P}_i$7 s runtime with GCP error $\mathcal{P}_i$8 m, whereas the sparsified graph had $\mathcal{P}_i$9 edges and $(n_p,c_p)$0 s runtime with GCP error $(n_p,c_p)$1 m (Li et al., 2024).

BALM3.0 moves the computational bottleneck into a majorization–minimization surrogate. By constructing a point-to-plane surrogate that decouples scan poses, it reduces the optimization time complexity from cubic to linear, reports up to $(n_p,c_p)$2 times faster optimization speed, and reduces memory usage to $(n_p,c_p)$3. The distributed implementation is reported to optimize $(n_p,c_p)$4 poses with $(n_p,c_p)$5 GB point clouds on four consumer-level laptops (Li et al., 26 Feb 2025).

Consistency has emerged as a separate concern from raw speed. The consistency-improved LiDAR–inertial BA introduces a stereographic-projection representation for planes and edges, combines a MAP formulation with First-Estimate Jacobians, and explicitly targets nullspace preservation and covariance correctness. The reported NEES remained within $(n_p,c_p)$6 versus BALM’s $(n_p,c_p)$7–$(n_p,c_p)$8, and turning off FEJ caused NEES to blow up to $(n_p,c_p)$9–$p_i = R_{s_i}p_{f_i}+t_{s_i}$0 (Li et al., 6 Feb 2026). This suggests that second-order efficiency alone is not sufficient; the estimator’s linearization policy can materially affect whether the inferred uncertainty is meaningful.

5. Multi-sensor, continuous-time, and calibration extensions

LBA has also become a backbone for multi-sensor state estimation and calibration rather than only a LiDAR-only mapping back-end. BA-LINS formulates a frame-to-frame BA for LiDAR-inertial navigation in a sliding-window factor graph, combining IMU-preintegration residuals with LiDAR BA factors built from same-plane points tracked across keyframes. It also introduces adaptive covariance estimation for the BA measurements. On the private dataset, the reported improvements over FF-LINS were $p_i = R_{s_i}p_{f_i}+t_{s_i}$1 in absolute translation accuracy and $p_i = R_{s_i}p_{f_i}+t_{s_i}$2 in state-estimation efficiency, with estimator runtime decreasing from $p_i = R_{s_i}p_{f_i}+t_{s_i}$3 ms to $p_i = R_{s_i}p_{f_i}+t_{s_i}$4 ms (Tang et al., 2024).

In LiDAR–camera fusion, a joint optimization approach combines camera reprojection and stereo-depth terms with LiDAR point-to-plane cloud registration. The variables are camera poses, 3D landmarks, and the LiDAR-to-camera extrinsic, and the paper reports an averaged accuracy of $p_i = R_{s_i}p_{f_i}+t_{s_i}$5 mm and resolution of $p_i = R_{s_i}p_{f_i}+t_{s_i}$6 points per square cm against survey-scanner ground truth (Zhen et al., 2019). LVBA extends the coupling differently: it first optimizes LiDAR poses via a global LiDAR BA, then performs photometric visual BA with planar features from the LiDAR point cloud and a LiDAR-assisted global visibility algorithm for RGB point cloud mapping (Li et al., 2024).

Extrinsic calibration can itself be cast as an LBA problem. DLBAcalib proposes a dual-LBA framework for non-overlapping LiDARs in which a reference map is first built from a target LiDAR by sliding-window LiDAR bundle adjustment and extrinsics are then estimated through a robust joint LBA optimization with adaptive weighting. For non-overlapping configurations, the reported average translational error is $p_i = R_{s_i}p_{f_i}+t_{s_i}$7 mm and the rotational error is $p_i = R_{s_i}p_{f_i}+t_{s_i}$8, with initial error tolerance up to $p_i = R_{s_i}p_{f_i}+t_{s_i}$9 m and $$J_{\rm plane}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \bigl(n^\top(p_i-q)\bigr)^2,$$0 (Ye et al., 12 Jul 2025).

A separate extension concerns time parameterization. The continuous-time multi-scan alignment approach explicitly states that utilizing a continuous-time trajectory allows the ego-motion of the LiDAR scanner while recording a single scan to be considered directly in the least squares adjustment. Together with pruning the search space of correspondences and an out-of-core circular buffer, the method is reported to align thousands of point clouds efficiently, including multi-session alignment (Wiesmann et al., 2024). This marks a shift from scanwise rigid-body assumptions toward within-scan motion modeling in the BA objective itself.

6. Empirical behavior, limitations, and recurring misconceptions

Experimental results across the literature consistently show that LBA can reduce drift and improve map consistency, but they also clarify where specific formulations succeed or fail. In simulation, PSS-BA reports the best Absolute Pose Error in four scenes—MBS $$J_{\rm plane}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \bigl(n^\top(p_i-q)\bigr)^2,$$1 m, Hangar $$J_{\rm plane}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \bigl(n^\top(p_i-q)\bigr)^2,$$2 m, Crane $$J_{\rm plane}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \bigl(n^\top(p_i-q)\bigr)^2,$$3 m, and Street $$J_{\rm plane}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \bigl(n^\top(p_i-q)\bigr)^2,$$4 m—and in a real-world helmet-mounted dataset reduces the occupied-voxel count from $$J_{\rm plane}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \bigl(n^\top(p_i-q)\bigr)^2,$$5 initially to $$J_{\rm plane}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \bigl(n^\top(p_i-q)\bigr)^2,$$6, compared with $$J_{\rm plane}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \bigl(n^\top(p_i-q)\bigr)^2,$$7 for BALM (Li et al., 2024). BALM reports a closed-loop Livox Horizon drift reduction from $$J_{\rm plane}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \bigl(n^\top(p_i-q)\bigr)^2,$$8 m to $$J_{\rm plane}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \bigl(n^\top(p_i-q)\bigr)^2,$$9 m and, on a Velodyne VLP-16 path, from $$J_{\rm edge}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \|(I-nn^\top)(p_i-q)\|^2.$$0 cm for LOAM and $$J_{\rm edge}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \|(I-nn^\top)(p_i-q)\|^2.$$1 cm for LeGO-LOAM to $$J_{\rm edge}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \|(I-nn^\top)(p_i-q)\|^2.$$2 cm for BALM (Liu et al., 2020).

One common misconception is that LBA must rely on hard geometric associations to predefined planes and edges. SGBA explicitly replaces hard assignment with soft probabilistic feature association and shows that the probabilistic variant achieves average ATE $$J_{\rm edge}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \|(I-nn^\top)(p_i-q)\|^2.$$3 m versus $$J_{\rm edge}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \|(I-nn^\top)(p_i-q)\|^2.$$4 m for hard “nearest” assignment on MCD / OS-128 LiDAR. On KITTI, SGBA with ground-truth semantics reports average ATE $$J_{\rm edge}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \|(I-nn^\top)(p_i-q)\|^2.$$5 m versus $$J_{\rm edge}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \|(I-nn^\top)(p_i-q)\|^2.$$6 m for BALM (Ji et al., 2024). Another misconception is that LBA is necessarily pose-only. MAD-BA instead jointly optimizes poses and surfel structure and reports on KITTI an average ATE RMSE of $$J_{\rm edge}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \|(I-nn^\top)(p_i-q)\|^2.$$7 m, compared with $$J_{\rm edge}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \|(I-nn^\top)(p_i-q)\|^2.$$8 for the initial LOAM trajectory, $$J_{\rm edge}(T,n,q) = \frac{1}{N}\sum_{i=1}^N \|(I-nn^\top)(p_i-q)\|^2.$$9 for BALM2, and $T$0 for HBA; on Newer College “quad-easy,” it reports map Chamfer-L1 improving from $T$1 cm for the initial surfel map to $T$2 cm, with F-score increasing to approximately $T$3 (Ćwian et al., 7 Jan 2025).

At the same time, the literature is explicit about failure modes. The local plane assumption may be inadequate in complex environments with low planar structures or substantial initial pose errors (Li et al., 2024). SGBA reports that using all semantic classes can diverge or become too slow, which is why adaptive semantic selection is introduced (Ji et al., 2024). RSO-BA states that pure LiDAR BA can struggle in low-feature corridors and that degenerate plane orientations may occur, even though the method reports mean ATE improvements such as $T$4 cm versus $T$5 cm on Hilti and $T$6 cm versus $T$7–$T$8 cm on ETHZ (Ma et al., 2024).

The resulting research picture is therefore not that LBA has converged to a single mature formulation, but that several design axes remain active: whether to eliminate geometry analytically or optimize it directly, whether to use deterministic or probabilistic correspondence, whether to prioritize real-time local correction or large-scale global consistency, and whether uncertainty and observability should be treated as first-class optimization targets rather than by-products. The current body of work indicates that each axis leads to a materially different estimator, residual structure, and scalability profile.