Sliding-Window LiDAR Bundle Adjustment

• Sliding-window LiDAR BA is a local joint-estimation method that optimizes a limited set of recent scans, poses, and landmark statistics to mitigate front-end drift.
• It incorporates diverse formulations—from pose-only adjustments with analytical feature elimination to dense, semantic, and continuous-time methods—each offering unique trade-offs in accuracy and computational cost.
• Empirical studies demonstrate that carefully designed window management and marginalization strategies enable real-time performance and improved drift reduction over traditional LiDAR odometry.

Sliding-window LiDAR bundle adjustment denotes a class of local joint-estimation methods in which a bounded set of recent LiDAR scans, poses, and sometimes auxiliary states or landmarks is optimized together in order to reduce front-end drift while keeping computation bounded. In the literature, the term covers several distinct formulations: pose-only edge/plane adjustment, explicit pose-and-plane BA, landmark-map BA with short pose windows, semantic local-map BA, dense correspondence-free multi-scan adjustment, and tightly coupled LiDAR-inertial or LiDAR-visual-inertial local smoothers (Liu et al., 2020, Zhang et al., 2022, Ji et al., 2024, Skuddis et al., 2024, Tang et al., 17 Mar 2026).

1. Conceptual scope and representative formulations

A useful classification is by what remains active inside the window and by how geometric structure is represented. "BALM" formulates local LiDAR BA as a sliding-window optimization over scan poses, while analytically eliminating edge and plane features so that the final objective depends only on poses; in experiments its local BA uses the 20 most recent scans and is triggered every 5 scans (Liu et al., 2020). "LMBAO" instead keeps the pose window very short—fixed at 4 scans—but retains older geometric information through a landmark map and landmark-level statistics, so the map is not identical to the active pose window (Zhang et al., 2022). "SGBA" uses a sliding window of keyframes with window size $w=10$, but the local map is a semantic Gaussian mixture model rather than explicit planes or edges, and the optimization alternates between soft association, pose update, and GMM update (Ji et al., 2024). "DMSA" uses a time-based sliding window of duration $d_{window}$, a continuous trajectory, IMU preintegration, and a dense correspondence-free objective based on Gaussian voxel landmarks induced from the merged cloud (Skuddis et al., 2024). "PA-LVIO" is a fixed-size sliding-window factor graph in which the LiDAR part is pose-only: same-plane clusters across multiple frames define LiDAR factors, and frame-to-map LiDAR registration is injected as compact pose factors rather than as explicit map-point states (Tang et al., 17 Mar 2026). "RSO-BA" is an explicit sliding-window BA over keyframe poses and plane landmarks, with group-wise plane measurements and an analytical second-order estimator (Ma et al., 2024).

Formulation	Active variables	Distinctive property
BALM	Scan poses	Edge/plane features analytically eliminated
LMBAO	$x_k=\{T_k,v_k,\omega_k\}$ over 4 scans	Landmark map retained outside pose window
SGBA	Window poses and semantic GMM parameters	Soft semantic association via ECM
DMSA	Continuous-time control poses in a time window	Dense correspondence-free Gaussian-cell objective
RSO-BA	Keyframe poses and plane landmarks	MSGM with analytical second-order Hessian
PA-LVIO	Window poses, velocities, biases, calibration; no LiDAR landmarks	Pose-only LiDAR BA-style factors plus F2M pose factors

This diversity matters because "sliding-window LiDAR BA" does not imply a single canonical state-space model. In some systems, the window is a true fixed-lag optimization over poses and landmarks; in others, it is a local pose-only smoother whose geometric structure has been eliminated analytically or compressed into grouped statistics.

2. State design, residuals, and landmark models

The earliest representative pose-only formulation in the supplied literature is BALM. Its central move is to replace exact point correspondences with local geometric consistency: for a plane voxel the cost is the smallest eigenvalue $\lambda_3(\mathbf A)$, and for an edge voxel the cost is $\lambda_2(\mathbf A)+\lambda_3(\mathbf A)$, where $\mathbf A$ is the covariance of transformed feature points inside the voxel. Because the optimal plane normal or edge direction can be solved in closed form from the covariance eigendecomposition, the feature parameters are removed analytically and the window optimization depends only on scan poses (Liu et al., 2020).

A contrasting explicit formulation appears in RSO-BA. There, the active state is $\mathbf X=\{\boldsymbol{\chi},\boldsymbol{\Pi}\}$, with $\boldsymbol{\chi}=\{\mathbf T_i\}_{i=1}^{N_w}$ the sliding-window keyframe poses and $\boldsymbol{\Pi}$ the effective plane landmarks. Instead of accumulating point-to-plane residuals point by point, the method constructs an integrated group matrix $\mathbf Q_{i,j}=\mathbf P_{i,j}^T\mathbf P_{i,j}$ for the points of landmark $d_{window}$0 observed in keyframe $d_{window}$1, and a fixed group matrix $d_{window}$2 for historical map points. The proposed mean squared group metric is $d_{window}$3 and $d_{window}$4, and the robust objective applies a kernel $d_{window}$5 to these grouped metrics. The paper further derives analytical gradients and Hessians and accelerates the explicit pose-landmark system with the Schur complement (Ma et al., 2024).

SGBA changes the map representation more radically. Each landmark is a semantic Gaussian $d_{window}$6 with label $d_{window}$7, and the optimization variables are $d_{window}$8. The complete-data objective reduces to a weighted sum of Mahalanobis terms and covariance log-determinants, and soft responsibilities $d_{window}$9 replace hard scan-to-landmark assignments. In implementation, however, the method is not a monolithic joint solve; it uses Expectation Conditional Maximization to alternate between association, pose update, and Gaussian update inside the local window (Ji et al., 2024).

Dense formulations depart even further from classical landmark BA. DMSA merges all points in the active interval into a common reference frame, partitions them into coarse and fine voxel cells, models each occupied cell as a Gaussian landmark, and minimizes the average Mahalanobis scatter of points within each cell: $x_k=\{T_k,v_k,\omega_k\}$0 There are no direct correspondences between scans, and the Gaussian statistics are recomputed from the transformed cloud rather than kept as persistent landmark variables (Skuddis et al., 2024).

These formulations suggest that LiDAR BA has two recurring design axes. One axis is whether landmarks are explicit, implicit, or eliminated; the other is whether the residual is built from planes and edges, semantic distributions, or dense local statistics. The window machinery is then layered on top of that modeling choice rather than determining it.

3. Window management, marginalization, and numerical structure

Window management is a defining technical fault line. BALM uses a local BA over the 20 most recent scans; when the window is full, older scans are removed from the active state and their points are merged into fixed map statistics, so the retained map acts as an anchor for the current window (Liu et al., 2020). LMBAO makes this strategy explicit at the landmark level. When scan $x_k=\{T_k,v_k,\omega_k\}$1 leaves the $x_k=\{T_k,v_k,\omega_k\}$2-scan window, its contribution to a landmark covariance is accumulated into

$x_k=\{T_k,v_k,\omega_k\}$3

and the active covariance becomes

$x_k=\{T_k,v_k,\omega_k\}$4

This is a landmark-statistics marginalization rather than a Schur-complement prior over retained poses (Zhang et al., 2022).

Other systems use the term "sliding window" more loosely. SGBA is implemented as local BA on a sliding window of keyframes, but the paper does not provide explicit equations for landmark carry-over, Schur-complement marginalization, or prior-factor construction; it therefore behaves more like repeated local-batch optimization than a full fixed-lag smoother (Ji et al., 2024). AS-LIO is even more distinct: it uses a fixed-duration data window whose shift amount is controlled by the Spatial Overlap Degree, but the paper is explicit that "our sliding window does not maintain additional states," so it is a sliding measurement window rather than multi-state BA (Zhang et al., 2024).

When exact fixed-lag smoothing is required, the square-root marginalization literature becomes directly relevant. "Square Root Marginalization for Sliding-Window Bundle Adjustment" proposes a prior of the form

$x_k=\{T_k,v_k,\omega_k\}$5

eliminates nuisance landmarks by nullspace projection, and marginalizes old states by QR in Jacobian space rather than by Hessian-space Schur complement. The paper proves algebraic equivalence to Schur complement, including the Moore–Penrose case under rank deficiency, and reports that the square-root form is 36% faster than the baseline while avoiding numerical failures seen with conventional Hessian-based marginalization in single precision (Demmel et al., 2021).

PA-LVIO adds a more specialized variation. Its oldest keyframe is marginalized in the usual sliding-window manner, but the LiDAR frame-to-map pose factor attached to that oldest keyframe is explicitly discarded rather than marginalized into the prior. The stated motivation is consistency: the system keeps map-based absolute constraints transiently inside the active window, but prevents them from becoming inherited prior information (Tang et al., 17 Mar 2026). This suggests that not all factors in a sliding-window LiDAR backend should necessarily survive marginalization unchanged.

4. Correspondence generation, map structure, and continuous-time handling

Correspondence generation is often the practical bottleneck. BALM replaces nearest-neighbor feature matching with adaptive voxelization: points are inserted into a hash-indexed set of octrees, recursively subdivided until a voxel is sufficiently plane-like or edge-like, and each final voxel defines one BA cost term (Liu et al., 2020). "An Efficient Plane Extraction Approach for Bundle Adjustment on LiDAR Point clouds" refines this front-end specifically for BA by introducing a quarter-based PCA consistency test inside each voxel and a local plane-merging step. In the reported experiments, root voxel size is $x_k=\{T_k,v_k,\omega_k\}$6, minimum voxel size is $x_k=\{T_k,v_k,\omega_k\}$7, minimum points is 20, and the proposed extractor achieves the best mean BA accuracy and the best mean total time when plugged into BALM2 (Liu et al., 2023).

SGBA replaces deterministic feature association with semantics-gated soft assignment. A point $x_k=\{T_k,v_k,\omega_k\}$8 can only be assigned to Gaussian components with the same semantic label, and the posterior responsibility

$x_k=\{T_k,v_k,\omega_k\}$9

replaces hard nearest-neighbor assignment. The same paper also proposes adaptive semantic label selection based on the condition number

$\lambda_3(\mathbf A)$0

with $\lambda_3(\mathbf A)$1 and $\lambda_3(\mathbf A)$2, so the active residual set is chosen for both computational cost and conditioning (Ji et al., 2024).

DMSA removes explicit correspondences altogether. All points inside the window are merged under the current continuous trajectory, partitioned into two voxel resolutions, and modeled as Gaussian landmarks. This is paired with cubic Hermitian spline interpolation for position, spherical linear interpolation for orientation, and 1 ms trajectory sampling, so each LiDAR point is transformed according to its acquisition time rather than by a single rigid pose per scan (Skuddis et al., 2024). A plausible implication is that dense sliding-window LiDAR BA can trade explicit geometric correspondences for statistics of merged local structure, provided that the trajectory model is continuous enough to absorb intra-scan motion.

PSS-BA shows another direction when plane assumptions are weak. It constructs local tangent frames, fits second-order polynomial surfaces

$\lambda_3(\mathbf A)$3

and minimizes point-to-local-surface residuals while progressively shrinking the smoothing scale from an initial $\lambda_3(\mathbf A)$4 by a factor $\lambda_3(\mathbf A)$5. The paper is not a sliding-window method—it optimizes batches of 100 frames—but its factor design and coarse-to-fine continuation are directly relevant whenever local window BA must handle curved or weakly planar geometry (Li et al., 2024).

5. Integration with inertial, visual, and calibration tasks

Sliding-window LiDAR BA often appears as one component of a multisensor estimator rather than as a standalone mapper. PA-LVIO is a tightly coupled sliding-window factor graph whose state includes poses, velocities, biases, camera–IMU and LiDAR–IMU extrinsics, and temporal offsets. Its LiDAR frame-to-frame factor is a plane-thickness residual over same-plane point clusters from multiple keyframes,

$\lambda_3(\mathbf A)$6

while separate frame-to-map registration on the oldest and newest keyframes produces compact LiDAR pose factors. The system uses Ceres Solver, Levenberg–Marquardt, Huber robust cost, and explicit marginalization of old keyframes, except that frame-to-map LiDAR pose measurements are discarded rather than marginalized (Tang et al., 17 Mar 2026).

AS-LIO targets a different failure mode: aggressive field-of-view variation. It estimates the Spatial Overlap Degree between the current frame and the map, shortens the update stride when overlap drops, and retains historical observations inside a fixed-duration measurement window. Because it optimizes only the current ESKF state rather than multiple poses, it is better described as an adaptive overlap-aware odometry method than as LiDAR BA proper (Zhang et al., 2024). This distinction matters because many systems described as "sliding-window" do not jointly re-optimize a trajectory segment.

Cross-modal variants further expand the design space. "Photometric LiDAR and RGB-D Bundle Adjustment" is a global pose-only direct BA, not a sliding-window system, but it shows that LiDAR scans can be represented as spherical projective images with intensity, range, and normal channels, and optimized through robust multi-cue photometric residuals. The paper explicitly states that the residual factorization over overlapping frame pairs makes the method naturally reusable inside a local window (Giammarino et al., 2023). At the other end of the spectrum, "A Robust Stereo Camera Localization Method with Prior LiDAR Map Constrains" uses a local window optimization in which stereo poses and visual landmarks are optimized against coplanarity constraints from a fixed LiDAR map; the result is not LiDAR BA, but it demonstrates how a sliding-window optimizer can be anchored by LiDAR-derived plane factors even when LiDAR scans are not optimized online (Han et al., 2019).

Calibration tasks provide a further application. DLBAcalib first refines the trajectory of LiDAR A by sliding-window LiDAR BA, then constructs a target map

$\lambda_3(\mathbf A)$7

extracts planar voxels, and solves non-overlapping LiDAR extrinsics by weighted point-to-plane alignment

$\lambda_3(\mathbf A)$8

In this setting, sliding-window LiDAR BA is the map-quality backbone that makes non-overlapping calibration possible (Ye et al., 12 Jul 2025).

6. Empirical behavior, limitations, and open questions

The empirical literature supports sliding-window LiDAR BA as an effective drift-reduction mechanism, but it also shows that accuracy, robustness, and computational cost depend heavily on representation choice. BALM reports that when optimizing a sliding window of 20 scans, the local BA runs nearly real-time at 10Hz, and on an 817 m handheld Livox Horizon sequence its translation drift is 0.31 m $\lambda_3(\mathbf A)$9 versus 6.228 m $\lambda_2(\mathbf A)+\lambda_3(\mathbf A)$0 for LOAM (Liu et al., 2020). LMBAO reports that a 4-scan window, together with landmark-map maintenance and landmark-statistics marginalization, improves odometry APE RMSE on the "Library loop" from 25.873 for VLOM-odom to 13.098 for the full method while keeping odometry runtime per frame below the corresponding VLOM baseline on the reported datasets (Zhang et al., 2022).

Semantic and dense variants extend this behavior to more difficult scenes. On KITTI, SGBA with a sliding window of keyframes reports average ATE 3.11 m with 100% labels and 3.16 m with 60% labels via SPVNAS, compared with 3.34 m for KISS-ICP and 3.67 m for BALM; on MCD OS-128 it improves average ATE from 2.45 m for nearest-landmark association to 2.28 m for probabilistic association (Ji et al., 2024). DMSA-LIO, which uses a continuous-time windowed dense objective rather than feature BA, reports the lowest RMSE and lowest maximum error in 7 of 8 evaluated sequences, but its current implementation still needs between 1.5 and 4 times recording time, with the numerical Jacobian identified as the bottleneck (Skuddis et al., 2024).

Recent pose-only and application-driven systems show the same trade-off. PA-LVIO reports that replacing tightly coupled raw frame-to-map LiDAR factors inside the factor graph with compact LiDAR pose factors reduces average factor-graph optimization time by 20.6% while preserving comparable accuracy on the reported platform (Tang et al., 17 Mar 2026). DLBAcalib reports average translational error of 5 mm and rotational error of $\lambda_2(\mathbf A)+\lambda_3(\mathbf A)$1 for non-overlapping sensor configurations, attributing part of that result to the improved target map produced by the sliding-window LiDAR BA stage (Ye et al., 12 Jul 2025).

Several limitations recur across the literature. Plane- and edge-based methods depend on local planarity or linearity and require a sufficiently good initial pose estimate; BALM states both limitations explicitly (Liu et al., 2020). SGBA states that it is only a local BA method and cannot guarantee global consistency (Ji et al., 2024). PSS-BA limits optimization to 100 frames because of high memory usage (Li et al., 2024). Methods that use "sliding window" in the measurement sense rather than the state sense can be efficient, but they should not be conflated with fixed-lag BA (Zhang et al., 2024). This suggests that current research still divides between three design goals that are not yet fully unified: strong local refinement, principled information retention under marginalization, and robustness in weakly planar or aggressively dynamic environments.

A final caution concerns bibliographic scope rather than method design. The supplied record for "Efficient LiDAR Bundle Adjustment for Multi-Scan Alignment Utilizing Continuous-Time Trajectories" does not contain a LiDAR-estimation method at all; it is described in the record as an IPB formatting guide and therefore provides no technical content on sliding-window LiDAR BA (Wiesmann et al., 2024). That anomaly underscores a broader point: the phrase "LiDAR bundle adjustment" now covers a heterogeneous literature, and careful reading of the underlying state, residual, and marginalization design is necessary before drawing methodological comparisons.