Multi-Camera Pose-Only Constraint

• Multi-Camera Pose-Only Constraint is a formulation where camera poses are optimized while scene structure is fixed or analytically eliminated.
• It underpins methods for generalized-camera relative pose estimation and multi-view optimization, ensuring cross-view consistency through tailored constraints.
• The approach achieves notable efficiency improvements, reducing memory usage and speeding up optimization in various benchmarks like ETH3D and KITTI.

Multi-camera pose-only constraint denotes, in the strictest geometric sense, a formulation in which the unknowns are camera or rig poses while scene structure is fixed, analytically eliminated, or represented only implicitly through image rays. In generalized-camera relative pose, this yields constraints directly on $R$ and $\mathbf{t}$, sometimes after eliminating translation or point depths; in broader multi-view systems, the same phrase is often used more loosely for cross-view consistency terms that still couple cameras to body, object, depth, or other latent variables. Recent work therefore distinguishes genuine pose-only formulations from joint camera–body or camera–structure objectives (Liang et al., 26 Apr 2026, Li et al., 2024).

1. Scope of the concept

In the rigid multi-camera literature, a pose-only constraint usually means that scene points are not optimized as free variables. They may be eliminated algebraically, replaced by hidden depths tied to a minimal sample, or treated as fixed map points. This is the sense used by generalized-camera relative pose solvers and by recent multi-camera pose adjustment methods that explicitly “eliminate 3D points from the parameter space” (Liang et al., 26 Apr 2026).

A broader usage appears in self-supervised multi-view learning. In "Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry" (Bouazizi et al., 2021), the closest pose-only component is a cross-view 3D consistency loss over per-view predicted skeletons, but it remains a training-time supervision term rather than an inference-time multi-camera optimizer. In "Self-learning Canonical Space for Multi-view 3D Human Pose Estimation" (Li et al., 2024), the relevant multi-view constraint is explicitly not pure pose-only: the canonical parameter space contains per-view camera pose $\{\pi^i\}_{i=1}^N=\{R^i,t^i\}_{i=1}^N$, global orientations $\{\theta_g^i\}_{i=1}^N$, and shared body parameters $\{\theta_b,\beta\}$. In "Unconstrained Multi-view Human Pose Estimation with Algebraic Priors" (Qin et al., 27 Apr 2026), the nearest equivalent is a calibration-free projective consistency constraint over predicted cameras and observed joints after algebraically eliminating the latent 3D point and projective depths, rather than a constraint over cameras alone.

This suggests a spectrum. At one end are strict generalized-camera formulations in which only motion variables remain. At the other are hybrid objectives in which camera pose is explicit but inseparable from auxiliary state such as SMPL variables, object similarity transforms, dense depth, or scene flow. The phrase “pose-only” is therefore precise only when the elimination or fixing of structure is explicit.

2. Generalized-camera rigid geometry

The canonical rigid formulation models a calibrated multi-camera rig as a generalized camera with known component-camera extrinsics $\{Q_i,\mathbf{s}_i\}$ and unknown rig motion $R\in SO(3)$, $\mathbf{t}\in\mathbb{R}^3$. For a correspondence observed by camera $i$ in the first rig pose and camera $i'$ in the second, the induced relative motion between the participating component cameras is

$\mathbf{t}$0

This yields the per-correspondence effective essential matrix

$\mathbf{t}$1

which makes the constraint bilinear in $\mathbf{t}$2 and $\mathbf{t}$3 (Zhao et al., 2021).

For point correspondences, the generalized epipolar condition is

$\mathbf{t}$4

For affine correspondences, the same per-correspondence essential matrix enters

$\mathbf{t}$5

A single affine correspondence therefore provides three independent constraints, whereas a point correspondence provides one. This is why the generic generalized-rig problem is minimal with $\mathbf{t}$6 PCs or $\mathbf{t}$7 ACs (Zhao et al., 2021, Guan et al., 2023).

Motion priors further change minimality. "Minimal Cases for Computing the Generalized Relative Pose using Affine Correspondences" (Guan et al., 2020) shows that under planar motion a single AC is minimal, while a second AC is introduced to handle the degenerate case; with known vertical direction, two ACs are minimal. In these formulations, the constraint is still pose-only because the unknowns remain motion parameters, not scene structure.

3. Elimination of structure and rotation-only formulations

A central line of work removes latent variables analytically. In "Equivalent Constraints for Two-View Geometry: Pose Solution/Pure Rotation Identification and 3D Reconstruction" (Cai et al., 2018), the calibrated two-view imaging equation

$\mathbf{t}$8

is shown to be equivalent to a Pair of Pose-Only constraints: the same-side constraint

$\mathbf{t}$9

and the intersection constraint, expressed through the angular relations among $\{\pi^i\}_{i=1}^N=\{R^i,t^i\}_{i=1}^N$0, $\{\pi^i\}_{i=1}^N=\{R^i,t^i\}_{i=1}^N$1, and $\{\pi^i\}_{i=1}^N=\{R^i,t^i\}_{i=1}^N$2. The stated significance is that the correct pose solution can be identified without triangulating 3D points, and the intersection inequality provides a criterion for pure rotation identification.

For generalized multi-camera systems, translation can be treated as a hidden variable. "On Relative Pose Recovery for Multi-Camera Systems" (Zhao et al., 2021) rewrites the AC constraints as

$\{\pi^i\}_{i=1}^N=\{R^i,t^i\}_{i=1}^N$3

then enforces rank deficiency through determinants of $\{\pi^i\}_{i=1}^N=\{R^i,t^i\}_{i=1}^N$4 minors and additional $\{\pi^i\}_{i=1}^N=\{R^i,t^i\}_{i=1}^N$5 block-rank constraints. After factoring out the common denominator from the Cayley parameterization, the resulting rotation-only system consists of $\{\pi^i\}_{i=1}^N=\{R^i,t^i\}_{i=1}^N$6, namely $\{\pi^i\}_{i=1}^N=\{R^i,t^i\}_{i=1}^N$7 equations of degree $\{\pi^i\}_{i=1}^N=\{R^i,t^i\}_{i=1}^N$8, and $\{\pi^i\}_{i=1}^N=\{R^i,t^i\}_{i=1}^N$9, namely $\{\theta_g^i\}_{i=1}^N$0 equations of degree $\{\theta_g^i\}_{i=1}^N$1. In the known-rotation-angle case, the added scalar constraint

$\{\theta_g^i\}_{i=1}^N$2

yields solvers with $\{\theta_g^i\}_{i=1}^N$3 solutions.

"A Pose-only Geometric Constraint for Multi-Camera Pose Adjustment" (Liang et al., 26 Apr 2026) extends elimination from minimal relative pose to multi-view optimization. Each observation is represented by a generalized-camera ray $\{\theta_g^i\}_{i=1}^N$4, and a world point $\{\theta_g^i\}_{i=1}^N$5 is implicitly represented by two base observations $\{\theta_g^i\}_{i=1}^N$6. The left-base depth is recovered as

$\{\theta_g^i\}_{i=1}^N$7

and every other observation satisfies

$\{\theta_g^i\}_{i=1}^N$8

The residual is a normalized spherical error,

$\{\theta_g^i\}_{i=1}^N$9

and only rig poses are optimized. On a synthetic problem with $\{\theta_b,\beta\}$0 poses and $\{\theta_b,\beta\}$1 points, the paper reports 19× lower memory usage and 2.5× faster optimization than MultiCol BA; on ETH3D, MCPA reaches up to 5.98× speedup, and on KITTI up to 3.82× (Liang et al., 26 Apr 2026).

4. Fixed-structure and graph-based pose-only optimization

Another strict interpretation keeps structure fixed rather than eliminated. "2D-3D Pose Tracking with Multi-View Constraints" (Yu et al., 2023) is a two-view, pose-only optimization over a fixed LiDAR map. The back-end estimates only two camera poses,

$\{\theta_b,\beta\}$2

through

$\{\theta_b,\beta\}$3

with

$\{\theta_b,\beta\}$4

The optimized variables are only the two poses; the 3D LiDAR points are fixed, and no bundle adjustment over landmarks is performed.

A graph-based variant appears in "Dense Dynamic Scene Reconstruction and Camera Pose Estimation from Multi-View Videos" (Sun et al., 12 Mar 2026). Although the full system jointly estimates poses and depth, the first stage constructs a spatiotemporal connection graph

$\{\theta_b,\beta\}$5

with temporal edges inside each camera, spatial edges between different cameras at the same timestamp, and spatio-temporal edges between current frames and historical frames from other cameras. The relative transform is

$\{\theta_b,\beta\}$6

and the basic weighted reprojection objective is

$\{\theta_b,\beta\}$7

Only the regularization terms are explicitly pose-only,

$\{\theta_b,\beta\}$8

but the paper is highly relevant because it operationalizes multi-camera pose constraints on a graph over frame nodes in $\{\theta_b,\beta\}$9.

"MultiCam: On-the-fly Multi-Camera Pose Estimation Using Spatiotemporal Overlaps of Known Objects" (Li et al., 24 Mar 2026) replaces image correspondences by known-object poses. The fundamental camera-camera relation is

$\{Q_i,\mathbf{s}_i\}$0

and world-frame alignment uses

$\{Q_i,\mathbf{s}_i\}$1

Because the scene graph persists over time, cameras without simultaneous FoV overlap can still be connected through temporal chains of shared object observations. This is pose-only with respect to camera calibration in the narrow sense that image-level structure is not optimized, but object poses remain explicit intermediate state.

5. Multi-view human pose estimation and the limits of “pose-only”

Human-pose systems use the phrase in several non-equivalent ways. The clearest pose-only component appears in "Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry" (Bouazizi et al., 2021). With synchronized calibrated cameras, a monocular regressor $\{Q_i,\mathbf{s}_i\}$2 predicts a 3D skeleton from each 2D pose, and the cross-view consistency loss

$\{Q_i,\mathbf{s}_i\}$3

enforces that predictions from different views represent the same 3D pose up to camera-frame transformation. The full training objective

$\{Q_i,\mathbf{s}_i\}$4

uses $\{Q_i,\mathbf{s}_i\}$5. The method uses all views during training, but inference is single-view. In ablation, adding $\{Q_i,\mathbf{s}_i\}$6 improves the reported value from 65.3 to 63.9.

Uncalibrated pipelines move closer to calibration recovery from body observations. "Multi-View Person Matching and 3D Pose Estimation with Arbitrary Uncalibrated Camera Networks" (Xu et al., 2023) first matches people across views by constrained clustering, then estimates essential matrices from matched 2D joints, triangulates 3D joints, and finally runs bundle adjustment. Its cross-view association is not pose-only, because it uses re-ID appearance, bounding boxes, and tracking, but the geometry stage becomes a pose-only camera-recovery problem once joint correspondences are fixed. "Multi-Person 3D Pose Estimation from Multi-View Uncalibrated Depth Cameras" (Li et al., 2024) adds a depth-guided pairwise camera-pose objective that combines essential-matrix estimation with rotation agreement to a 3D rigid transform derived from depth-projected keypoints. The ablation reported for camera pose estimation improves from 395 mm, 1.73° without depth guidance to 194 mm, 1.27° with depth guidance.

The algebraic-prior line treats camera calibration itself as a learned variable. "Unconstrained Multi-view Human Pose Estimation with Algebraic Priors" (Qin et al., 27 Apr 2026) predicts camera parameters and 3D pose, then enforces vanishing of generators of the multiview ideal through

$\{Q_i,\mathbf{s}_i\}$7

where the two-view term includes

$\{Q_i,\mathbf{s}_i\}$8

This is not pose-only over cameras alone, but it is pose-only in the sense that the 3D point and projective scales have been eliminated from the polynomial relations. The reported GC ablation improves from 40.2 / 38.5 for No-geo to 22.5 / 12.8 for Full GC.

By contrast, "Self-learning Canonical Space for Multi-view 3D Human Pose Estimation" (Li et al., 2024) explicitly states a joint camera–body optimization. Its practical multi-view fitting uses

$\{Q_i,\mathbf{s}_i\}$9

with shared $R\in SO(3)$0 and per-view $R\in SO(3)$1. This is a multi-camera consistency equation, but not a pose-only one, because the camera variables are inseparable from SMPL pose and shape.

6. Extensions, ambiguities, and recurrent misconceptions

Several recent directions preserve the language of pose-only while changing the underlying geometry. "Relative Pose for Nonrigid Multi-Perspective Cameras: The Static Case" (Li et al., 2024) extends the generalized epipolar constraint to a static non-rigid rig whose camera extrinsics depend on gravity through a cantilever model. The classical line-incidence form

$R\in SO(3)$2

is retained, but the Plücker lines themselves become gravity-dependent: $R\in SO(3)$3 The problem grows from $R\in SO(3)$4 DoF to $R\in SO(3)$5 DoF because gravity direction becomes a latent observable variable.

A different ambiguity concerns terminology. "Structure-Aware NeRF without Posed Camera via Epipolar Constraint" (Chen et al., 2022) repeatedly uses “epipolar constraint,” but its actual geometric term is a multi-view 3D coincidence loss over matched SIFT features: $R\in SO(3)$6 This is camera–structure consistency, not classical algebraic epipolar geometry.

The phrase is looser still in multimodal reasoning. "Predicting Camera Pose from Perspective Descriptions for Spatial Reasoning" (Zhang et al., 5 Feb 2026) makes camera pose the explicit geometric anchor for cross-view fusion by encoding source poses as dense Plücker ray maps and regressing a target camera-to-world matrix

$R\in SO(3)$7

with supervision

$R\in SO(3)$8

The paper explicitly does not formulate bundle adjustment, epipolar residual minimization, or reprojection consistency optimization. Pose is an architectural conditioning variable rather than an analytic constraint.

A comparable borderline case is "MV-ROPE: Multi-view Constraints for Robust Category-level Object Pose and Size Estimation" (Yang et al., 2023). Its final object-level graph uses composition residuals over camera and object nodes, but object states live in $R\in SO(3)$9, and the measurements are produced by NOCS-to-depth registration. This is therefore pose-graph-like rather than strictly pose-only.

Accordingly, the modern literature supports a precise distinction. A multi-camera pose-only constraint, in the narrow geometric sense, is a relation over camera or rig poses after fixing or eliminating structure. Generalized-camera epipolar equations, hidden-variable elimination, two-pose tracking over a fixed map, and implicit-point multi-camera pose adjustment fit this definition directly. Joint camera–body, camera–object, camera–depth, or pose-conditioned reasoning systems remain closely related, but they are better described as hybrid multi-view consistency frameworks than as pure pose-only geometry.