Initialization-Free Bundle Adjustment

• Initialization-Free Bundle Adjustment is a family of methods that estimates camera poses and 3D structure without scene-specific initialization, supporting global SfM, SLAM, and large-scale reconstruction.
• Key approaches include parallax-manifold optimization, projective Object-Space Error with Variable Projection, power-series solvers, probabilistic Gaussians, and relative-rotation constraints that improve conditioning and reduce projective ambiguity.
• Practical success still depends on observation density, image connectivity, cheirality, generic camera priors, and metric upgrading, because a low surrogate objective does not guarantee an accurate Euclidean reconstruction.

Initialization-Free Bundle Adjustment (InitFree BA) denotes methods that recover camera poses and scene structure directly from image observations without scene-specific global pose and landmark initialization. Unlike conventional bundle adjustment (BA), which normally refines a geometrically meaningful reconstruction produced by feature matching, relative-pose estimation, triangulation, and incremental structure-from-motion (SfM), InitFree BA seeks to discover a valid reconstruction from random or generic initial configurations. The term does not have a single universally accepted operational meaning: some methods eliminate accurate pose and structure initialization but retain generic priors or auxiliary relative-motion estimates; others remove only map initialization, or merely improve robustness to poor initial states. The principal technical approaches include parallax-manifold BA with convexified initialization, projective Object-Space Error (OSE) and Variable Projection (VarPro), calibrated pOSE with relative rotations, probabilistic Gaussian formulations, and alternative local optimizers for poorly initialized BA.

1. Problem formulation and meaning of initialization freedom

For calibrated monocular SfM or SLAM, conventional BA jointly estimates camera poses and 3D structure by minimizing reprojection error. With camera rotations $R_i$, translations or camera centers, landmarks $X_j$, and image measurements $m_{ij}$, the classical objective has the form

$$\min_{\{R_i,t_i\},\{X_j\}} \sum_{(i,j)\in\Omega} \left\| \pi\!\left(K_i[R_i\mid t_i]X_j\right)-m_{ij} \right\|^2 .$$

The objective is intrinsically non-convex because of rotations, products between rotations and translations, perspective division, and simultaneous estimation of camera poses and depths. It also contains global gauge freedoms: translation, rotation, and, in monocular reconstruction, scale. Newton-type methods such as Gauss–Newton, Levenberg–Marquardt (LM), and DogLeg therefore require an estimate within a sufficiently large basin of attraction.

Poor initialization can produce inaccurate Jacobians, incorrect local minima, divergence, unstable updates, distorted maps, invalid depths, rejection of valid feature tracks, and failure during loop closure. In conventional SLAM, local mapping and incremental BA form a dependency chain: accurate poses are needed for accurate points, while accurate points are required to initialize later BA instances (Parra et al., 2019).

“Initialization-free” consequently has several meanings:

• Strict InitFree BA: camera poses and landmarks are initialized without scene-specific estimates, often from random projective camera configurations.
• Initialization-free global SfM: all poses are initialized simultaneously from the observation graph rather than through incremental temporal insertion.
• Map-initialization-free optimization: the optimizer does not require a preconstructed map, although it may require pose estimates.
• Initialization-robust BA: a local optimizer tolerates noisy or inaccurate initial estimates but still requires an initial state.

This distinction is central. DeepSFM explicitly assumes initial depth maps and camera poses, generally obtained from DeMoN, and is therefore initialization-robust rather than strictly initialization-free (Wei et al., 2019). The Optimal Control Algorithm (OCA) likewise requires an initial estimate $x_0$ containing camera parameters and 3D points (Xu et al., 2024). Event-based Photometric Bundle Adjustment (EPBA) avoids preinitializing a brightness map but still uses front-end rotation estimates, IMU dead reckoning, or previous event-based estimators as possible pose initializers (Guo et al., 2024).

By contrast, Power Variable Projection (PoVar) and related projective methods explicitly evaluate random camera initialization, while calibrated pOSE methods aim to obtain near-metric reconstructions from random starting solutions (Weber et al., 2024, Olsson et al., 30 Jun 2025).

2. Parallax-manifold bundle adjustment and convexified initialization

“Parallax Bundle Adjustment on Manifold with Convexified Initialization” introduces PMBA for monocular SfM/SLAM (Liu et al., 2018). Its principal objective is to address both the conditioning of feature variables and the initialization problem of conventional BA.

For each feature $j$, PMBA selects a main anchor $m_j$ and an associate anchor $a_j$ with the largest available parallax angle. Instead of XYZ coordinates or inverse depth, the feature is represented by

$$\mathbf F_j=(\cos\theta_j,\sin\theta_j,\mathbf n_j),$$

where $\mathbf n_j\in S^2$ is the unit observation direction in the main-anchor frame and $X_j$0 is the parallax angle between the anchor rays. The feature position is reconstructed from the anchor poses as

$X_j$1

The parallax angle directly describes triangulation geometry. XYZ coordinates become highly uncertain for nearly parallel viewing rays; inverse depth improves behavior for distant points but remains problematic near zero parallax; the parallax representation retains angular and bearing information even when Euclidean depth is poorly determined. Low-parallax features are therefore treated as valid measurements rather than discarded.

PMBA optimizes rotations on $X_j$2, poses on $X_j$3, and ray directions on $X_j$4. A ray perturbation uses a tangent basis $X_j$5 satisfying

$X_j$6

with retraction

$X_j$7

The complete feature update is

$X_j$8

This lift–optimize–retract procedure preserves unit-norm ray directions, avoids Euler-angle singularities, and gives ray uncertainty a rotational interpretation in the tangent plane.

PMBA also replaces the pixel reprojection residual with a normalized observation-ray residual. If $X_j$9 is the measured calibrated bearing and $m_{ij}$0 is the predicted local unit ray, then

$m_{ij}$1

For unit vectors,

$m_{ij}$2

where $m_{ij}$3 is the angle between the predicted and measured rays. The residual is bounded by $m_{ij}$4, globally continuous, and has bounded derivatives. Because it compares signed directions, it distinguishes a point in front of the camera from one behind it.

Convexified pose-graph initialization

The full PMBA objective remains nonlinear. Initialization therefore begins with pairwise geometry and rotation averaging:

1. relative rotations and translation directions are estimated from image pairs;
2. global orientations are recovered by chordal rotation averaging;
3. anchors are selected without explicit linear triangulation;
4. local parallax quantities are initialized from bearings and rotations;
5. an affine approximation of the observation ray is formed;
6. a reduced position-only pose-graph problem is solved.

The affine ray approximation is linear in camera positions. A linearly constrained quadratic program, QPLC, minimizes cross-product residuals between predicted ray vectors and measured ray lines while imposing a front-facing constraint,

$m_{ij}$5

The reduced model contains camera positions rather than all feature variables. In noise-free epipolar-geometry pairs, the associated position formulation is convex; in practice, QPLC is used as the computational convexification.

The resulting pipeline does not require incremental bootstrap, temporal insertion order, a motion prior, manually supplied poses, BAL-provided initialization, or third-party camera estimates. Images may arrive sequentially or out of order. The output of QPLC initializes full manifold PMBA, which remains a local nonlinear refinement.

Theoretical results concern both conditioning and initialization. The PMBA feature block of the normal matrix is block diagonal and satisfies

$m_{ij}$6

Theorem 1 states that the feature block is consistently non-singular for any state. The result removes the feature-induced singularity associated with long flat valleys in XYZ, inverse-depth, and original parallax formulations, but does not make the complete BA objective convex.

Theorem 2 gives a near-optimality bound for the convexified position problem. If $m_{ij}$7 is the global minimum of the reduced model and $m_{ij}$8 is the converged position, then

$m_{ij}$9

When orientations are close to optimal, this estimate is near-optimal for the original PMBA problem. In the noise-free case, the optimum is zero, implying exact recovery in the idealized setting. The guarantee concerns the convexified position model and does not establish global optimality for the complete nonlinear BA problem under arbitrary noise.

3. Projective OSE, Variable Projection, and scalable random initialization

PoVar addresses initialization-free large-scale BA by replacing the usual metric reprojection formulation with a projective object-space formulation and eliminating landmarks through Variable Projection (Weber et al., 2024).

A projective camera is represented by $$\min_{\{R_i,t_i\},\{X_j\}} \sum_{(i,j)\in\Omega} \left\| \pi\!\left(K_i[R_i\mid t_i]X_j\right)-m_{ij} \right\|^2 .$$0 and a landmark by homogeneous coordinates $$\min_{\{R_i,t_i\},\{X_j\}} \sum_{(i,j)\in\Omega} \left\| \pi\!\left(K_i[R_i\mid t_i]X_j\right)-m_{ij} \right\|^2 .$$1. The object-space formulation has the separable structure

$$\min_{\{R_i,t_i\},\{X_j\}} \sum_{(i,j)\in\Omega} \left\| \pi\!\left(K_i[R_i\mid t_i]X_j\right)-m_{ij} \right\|^2 .$$2

For fixed cameras, landmarks enter linearly. Variable Projection computes

$$\min_{\{R_i,t_i\},\{X_j\}} \sum_{(i,j)\in\Omega} \left\| \pi\!\left(K_i[R_i\mid t_i]X_j\right)-m_{ij} \right\|^2 .$$3

and substitutes this minimizer into the objective:

$$\min_{\{R_i,t_i\},\{X_j\}} \sum_{(i,j)\in\Omega} \left\| \pi\!\left(K_i[R_i\mid t_i]X_j\right)-m_{ij} \right\|^2 .$$4

Landmarks are therefore reoptimized for every camera configuration rather than held fixed during camera updates. This removes poorly conditioned nonlinear directions and gives the camera optimization access to a lower-valued implicitly minimized objective.

The projective pOSE formulation used in the experiments combines an object-space term with an affine-depth term controlled by $$\min_{\{R_i,t_i\},\{X_j\}} \sum_{(i,j)\in\Omega} \left\| \pi\!\left(K_i[R_i\mid t_i]X_j\right)-m_{ij} \right\|^2 .$$5. The experiments use $$\min_{\{R_i,t_i\},\{X_j\}} \sum_{(i,j)\in\Omega} \left\| \pi\!\left(K_i[R_i\mid t_i]X_j\right)-m_{ij} \right\|^2 .$$6. The projective formulation is initialization-tolerant but does not directly provide Euclidean cameras. Projective reconstruction has a global projective ambiguity, and a later metric upgrade is required.

Power Variable Projection

Variable Projection produces a reduced camera system that can become dense. PoVar avoids direct factorization by approximating the inverse with a finite power expansion,

$$\min_{\{R_i,t_i\},\{X_j\}} \sum_{(i,j)\in\Omega} \left\| \pi\!\left(K_i[R_i\mid t_i]X_j\right)-m_{ij} \right\|^2 .$$7

when $$\min_{\{R_i,t_i\},\{X_j\}} \sum_{(i,j)\in\Omega} \left\| \pi\!\left(K_i[R_i\mid t_i]X_j\right)-m_{ij} \right\|^2 .$$8. In the VarPro normal equations, the camera block is damped while the landmark block is undamped:

$$\min_{\{R_i,t_i\},\{X_j\}} \sum_{(i,j)\in\Omega} \left\| \pi\!\left(K_i[R_i\mid t_i]X_j\right)-m_{ij} \right\|^2 .$$9

Eliminating landmarks yields

$x_0$0

Defining

$x_0$1

the paper establishes that the eigenvalues of $x_0$2 lie in $x_0$3. The camera update is approximated by

$x_0$4

The implementation uses a maximum expansion order $x_0$5. PoVar preserves the sparse BA structure and avoids explicit factorization of the dense reduced camera system.

Riemannian projective refinement

Homogeneous camera and landmark vectors possess local scale freedom. RiPoBA removes this radial direction by representing normalized projective cameras on $x_0$6 and landmarks on $x_0$7, constructing tangent bases, projecting the Jacobians, applying the power expansion to the projected Schur complement, and normalizing after the update.

The complete projective pipeline is:

1. randomly initialize projective camera parameters;
2. compute initial landmarks through the VarPro landmark solve;
3. optimize the pOSE/object-space stage with PoVar;
4. normalize projective camera and landmark vectors;
5. refine them with Riemannian PoBA, or RiPoBA;
6. perform a projective-to-metric upgrade;
7. recover Euclidean camera and landmark parameters.

The BAL evaluation uses all 97 real-world problems, with 16 to 13,682 cameras and observations reaching several tens of millions. Cameras are randomly drawn from an isotropic Gaussian, and landmarks are computed by the landmark update. PoVar is reported to outperform direct-factorization and PCG VarPro baselines, particularly at the highest accuracy tolerance. The strongest tested combination is PoVar followed by RiPoBA. No global optimality guarantee or guarantee that every random initialization succeeds is given.

4. Calibrated initialization-free reconstruction

“Towards Initialization-free Calibrated Bundle Adjustment” extends pOSE by incorporating calibrated pairwise relative rotations directly into the optimization (Olsson et al., 30 Jun 2025). Ordinary pOSE is projectively invariant: for any invertible $x_0$8,

$x_0$9

preserves all camera-point products. Consequently, pOSE alone can produce a projective reconstruction that requires a subsequent metric upgrade.

The proposed method adds pairwise relative-rotation penalties. If $j$0 and $j$1 are the left blocks of camera matrices and $j$2 is a calibrated two-view relative rotation, the residual is

$j$3

The weighting matrix $j$4 approximates the effect of relative-rotation perturbations on the two-view reprojection objective after optimally eliminating translation and structure. The method does not impose explicit nonlinear constraints $j$5 or $j$6; the pairwise terms softly encourage rotation-like camera blocks and agreement with calibrated relative rotations.

The complete objective combines pOSE, affine regularization, and pairwise rotations. Variable Projection eliminates structure and translations, leaving a reduced nonlinear optimization in camera blocks. Unlike a projective-only formulation, the relative rotations restrict the projective ambiguity to a similarity transformation. The resulting reconstruction is therefore near metric rather than arbitrary projective.

The method’s random-start experiments use 100 initializations per dataset, with a maximum of 200 iterations. The proposed pOSE-plus-relative-rotation formulation achieved success rates of 58% on Ahlstromer, 88% on Park Gate, and 100% on De Guerre, UWO, and Togo. It required approximately one-third as many iterations as pOSE alone in the reported convergence experiments.

On the near-metric reconstruction study, averaged after linear metric upgrade, pOSE-plus-relative-rotation achieved rotation AUC@20 of 53.87, translation AUC@20 of 57.08, and landmark RMSE of 16.63. The corresponding values for pOSE were 34.25, 34.10, and 24.19. The central tradeoff is that the additional rotation constraints can increase the pOSE objective while substantially improving metric consistency.

5. Probabilistic, learned, and modality-specific formulations

Several approaches broaden initialization robustness without solving the same strict problem as projective InitFree BA.

Probabilistic Gaussian BA

ProBA replaces deterministic points with uncertain 3D Gaussian landmarks (Chui et al., 27 May 2025). A landmark generated by back-projecting image observation $j$7 at depth $j$8 is represented as

$j$9

After projection, its image covariance is approximated by

$m_j$0

The uncertainty-aware reprojection loss is a Gaussian negative log-likelihood,

$m_j$1

ProBA also encourages overlap between Gaussian estimates of the same landmark using the Bhattacharyya coefficient. Its total objective is

$m_j$2

The experiments use identity camera poses, random depths sampled from $m_j$3, no accurate focal-length initialization, AdamW, and 10,000 iterations. The principal claim is initialization robustness with unknown or uncertain focal length and random depth, not complete independence from image correspondences or scene structure. On the reported datasets, approximately 93% of experiments converge with mAA below $m_j$4, and ProBA-1 generally improves over ProBA-0.

DeepSFM

DeepSFM alternates learned depth and pose refinement using depth-based and pose-based cost volumes (Wei et al., 2019). Its update cycle is

$m_j$5

followed by

$m_j$6

The method uses 64 inverse-depth planes and 1,000 locally sampled pose hypotheses in the reported implementation. It is trained and tested with initial depth and pose, primarily from DeMoN. Its P-CV samples locally around the current pose, so it does not demonstrate recovery from arbitrary random or identity initialization. DeepSFM is best characterized as a learned, geometry-aware, initialization-robust BA refinement framework.

Event-based Photometric Bundle Adjustment

EPBA jointly optimizes a semi-dense panoramic brightness map and a rotation trajectory for a purely rotating event camera (Guo et al., 2024). Its event-native residual is

$m_j$7

The method uses individual event timestamps, pixel coordinates, polarities, and the event contrast threshold directly, without converting events into image-like representations. Rotations are interpolated continuously between control rotations, and the sparse Jacobian connects each event to nearby control poses and two map pixels.

EPBA can optimize the brightness map without a precomputed panorama. It is therefore map-initialization-free, but its pose trajectory is initialized using front-end rotation estimates, IMU dead reckoning, or previous event-based estimators. The pure-rotation assumption makes the map depth-independent; translation is not modeled, although small translations are tolerated in practice. Experiments report photometric-error reductions of up to approximately 90%.

OCA for cryo-electron tomography

OCA replaces LM with an optimal-control-derived update for cryo-ET BA (Xu et al., 2024). It introduces a control weight $m_j$8 and recursively accumulates gradient and Hessian information. A bisection procedure adapts $m_j$9 according to objective improvement.

OCA is an alternative local optimizer, not an initialization-free reconstruction method. It requires an initial state $a_j$0 containing camera parameters and 3D marker positions. On the VEEV dataset, OCA reached the stopping criterion in 28 iterations compared with 236 for LM, with identical reported final residuals. The demonstrated benefit is reduced iteration count and oscillation under poor initialization, not global convergence or elimination of initialization requirements.

6. Evaluation, limitations, and the optimization–reconstruction gap

Initialization-free BA must be evaluated end to end. A low value of a surrogate objective does not necessarily imply a valid Euclidean reconstruction.

“Initialization-Free Bundle Adjustment Revisited: A Controlled Experimental Study” evaluates OSE formulations using a Blender- and EEVEE-based generator with exact cameras, landmarks, visibility control, occlusions, arbitrary trajectories, and COLMAP-compatible export (Weber et al., 18 Aug 2026). The study compares pOSE, rOSE, RpOSE, expOSE, and pOSE with relative rotations under normal, uniform, and unit-circle camera initialization.

Its central finding is an optimization–reconstruction gap: projective solutions with similarly low OSE values can produce substantially different Euclidean reconstructions after metric upgrade. The projective-to-metric upgrade is therefore not a post-processing detail but a central component of InitFree BA.

After linear metric upgrade, pOSE-plus-relative-rotation achieved the strongest average performance:

• rotation AUC@20: 53.87;
• translation AUC@20: 57.08;
• landmark RMSE: 16.63.

The unit-circle initialization prior achieved rotation AUC@20 of 42.60, translation AUC@20 of 45.47, and landmark RMSE of 19.27, compared with 29.60, 32.21, and 24.63 for normal initialization. This suggests that current methods remain dependent on generic initialization priors even when they are free from scene-specific initialization.

Landmark observation density is also critical. Reducing average visibility to approximately three observations per landmark causes major failures, while increasing the minimum visibility to six improves rotation, translation, and landmark accuracy. Weakly observed landmarks perturb camera estimates and destabilize metric upgrade. Cauchy robustification has limited average effect but can rescue catastrophic failures; in one reported pOSE case, rotation AUC@20 improved from 0.4 to 97.8 and landmark RMSE from 40.66 to 0.10.

The principal limitations across current approaches are:

• Projective ambiguity: OSE methods require metric upgrade unless calibrated relative rotations are integrated directly.
• No universal global-optimality guarantee: VarPro and power expansions provide favorable optimization behavior, but not success from every random initialization.
• Dependence on generic priors: camera layouts such as unit-circle initialization can materially affect reconstruction.
• Observation density and connectivity: sparse or weakly connected observation graphs remain difficult.
• Cheirality and degeneracy: behind-camera configurations, coplanarity, narrow baselines, collinear motion, and insufficient bearing diversity can remain problematic.
• Correspondence quality: incorrect tracks or dense-match failures can defeat probabilistic and surrogate objectives.
• Gauge freedoms: global similarity ambiguity remains in calibrated reconstruction; projective ambiguity remains before metric upgrade.
• Computational cost: reduced or global formulations may require substantial memory, repeated linear solves, power expansions, or many learned-optimization iterations.
• Model restrictions: EPBA assumes pure rotation; ProBA uses a restricted intrinsic model; DeepSFM operates on pairwise or averaged multi-view costs; OCA remains a local optimizer.
• Metric-upgrade instability: low OSE or pOSE cost does not certify stable self-calibration.

The resulting research perspective is that InitFree BA is not a single algorithm but a family of strategies for enlarging the basin of meaningful reconstruction. PMBA uses parallax coordinates, manifold retractions, bounded ray residuals, and convexified pose-graph initialization. PoVar combines projective OSE, VarPro, power-series inverse expansions, and Riemannian refinement. Calibrated pOSE incorporates relative rotations to reduce projective ambiguity to similarity. ProBA modifies the objective through uncertainty propagation and Gaussian overlap. DeepSFM and EPBA address learned refinement and map-initialization-free event-based estimation, respectively, while OCA changes the local optimization dynamics.

The most stringent interpretation of initialization-free bundle adjustment is therefore:

$a_j$1

A method qualifies strongly only when it addresses the entire chain, including metric validity, cheirality, gauge handling, observation connectivity, and reconstruction quality after upgrade. The current evidence indicates that minimizing a surrogate BA objective is necessary but insufficient: successful InitFree BA must produce a projective solution that is also well-conditioned for metric reconstruction.