Canonical Camera-Space Transformation

• Canonical Camera-Space Transformation (CSTM) is a rigorously defined framework that provides explicit, canonical mappings between camera image spaces and world coordinates using projective and affine methods.
• CSTM underpins diverse applications in local affine correspondence, camera matrix factorization, and frustum mapping, enhancing tasks such as patch matching, view synthesis, and color calibration.
• By delivering closed-form transformations, CSTM bridges classical geometry with modern learning-based approaches, improving multi-view fusion, pose estimation, and cross-device adaptation.

The Canonical Camera-Space Transformation (CSTM) is a rigorously defined class of transformations in multiview geometry, projective camera modeling, and related computational imaging tasks. A CSTM provides a mathematically explicit, canonical mapping between camera image spaces (or between camera and world space), mediating the transfer of geometric, photometric, or semantic information across different viewpoints, modalities, or imaging configurations. CSTM variants appear in local affine correspondence, full projective camera factorization, generalized frustum-to-NDC mappings, neural canonicalization frameworks, and color constancy pipelines. The canonicalization principle underpins not only classical geometry but also modern learning-based representations, yielding unique advantages in camera–scene disentanglement, multi-view fusion, and cross-device compatibility.

1. Local Affine CSTM Between Calibrated Cameras

In calibrated multi-view geometry, the CSTM specifies the precise affine transformation $(A, t)$ mapping any infinitesimal image patch in one camera view to its canonicalized counterpart in another, given their relative pose and the local surface patch geometry. For two calibrated cameras with intrinsic matrices $K_1, K_2$, relative pose $(R, t)$, and a 3D surface patch centered at $X$ with unit normal $n$ and signed distance $d$ from the first camera center, the normalized image point $p_1 = [u_1; v_1; 1]$ in view 1 is mapped to view 2 by the projective homography

$H = R - \frac{t n^\top}{d},$

$p_2 \sim H p_1.$

The canonical affine transformation linearizes the mapping at $p_1$, yielding the local Jacobian $K_1, K_2$0 and offset $K_1, K_2$1:

$K_1, K_2$2

where the scaling $K_1, K_2$3, and $K_1, K_2$4 depend on $K_1, K_2$5 and the mapped center $K_1, K_2$6 (Hajder, 6 Feb 2026). These expressions are fully closed-form and directly computable from extrinsics, intrinsics, and the observed surface normal.

Key geometric assumptions for this CSTM include local planarity and perfect calibration. Estimating the required surface normal $K_1, K_2$7 can be achieved via dense stereo, RGB-D, or learning-based regression. The result is a canonical, differentiable map that aligns local coordinates up to first order, with critical applications in patch matching, view synthesis, and geometric warping (Hajder, 6 Feb 2026). Detailed derivations and efficient pseudocode are provided by Hajder.

2. Canonicalization by Camera Matrix Factorization

A complementary, global CSTM framework decomposes any $K_1, K_2$8 full-rank camera matrix $K_1, K_2$9 into a canonical chain of a $(R, t)$0 homogeneous projection $(R, t)$1 (central or parallel) and a $(R, t)$2 Euclidean reparameterizer $(R, t)$3:

$(R, t)$4

with

$(R, t)$5

where $(R, t)$6 is the camera center (nullspace of $(R, t)$7) and $(R, t)$8 the image plane (last row of $(R, t)$9 submatrix of $X$0) (Lu et al., 2014).

This LC-factorization yields a canonical form: $X$1 enforces a geometric projection (e.g., from $X$2 onto $X$3), while $X$4 subsumes translation, rotation, anisotropic scaling, shear, and principal point shifts needed to generate image coordinates. Both central and parallel projections are unified. The decomposition admits direct recovery of principal rays, image-plane geometry, and facilitates multi-view tasks such as midpoint triangulation in Euclidean or projective settings. The canonicality arises from the intrinsic determination of $X$5 and $X$6 by $X$7, affording a unique, well-parameterized camera-to-world-map (Lu et al., 2014).

3. CSTM for Frustum Parameterization, Projection, and Warping

In computer graphics, the CSTM paradigm governs the projective mapping from 3D camera space into a canonical cube (usually NDC) for both symmetric and general affine frusta. For standard frusta, a known $X$8 projection matrix $X$9 provides the mapping:

$n$0

with $n$1 (Glushkov et al., 2021).

For arbitrary affine frusta, CSTM generalizes $n$2 to handle any six nondegenerate plane equations. The explicit construction of $n$3 as a function of its plane parameters retains the property of mapping the entire visible frustum to NDC (Glushkov et al., 2021). Special operations—crop, reflection, and lens distortion—are reduced to structured updates of $n$4 and $n$5, supporting advanced rasterization and limited-resource pipelines.

The CSTM thus supports analytical back-projection, vanishing point, and frustum corner recovery via explicit $n$6 formulas, enabling robust geometric computations in rendering, simulation, and 3D vision.

4. CSTM in Learning-Based Canonicalization and Multi-View Fusion

Recent models in 3D vision and pose estimation leverage canonical parameter spaces to enforce geometric and semantic consistency across multiple views or modalities. For example, CMANet uses CSTM to map all per-view joint and shape parameters into a shared canonical body frame, with each camera view having its own extrinsics and per-view global orientation (Li et al., 2024). Letting $n$7 denote the $n$8th camera’s extrinsics and intrinsics, the transform

$n$9

maps canonical joint locations into the $d$0th view.

This canonical space allows fusion of intra- and inter-view information with disentangled optimization of view-independent ($d$1) and view-dependent ($d$2) parameters. Canonicalization yields direct multi-view geometric coupling via projected SMPL fits, mitigates per-view drift, and achieves improved 3D consistency and accuracy without external supervision (Li et al., 2024). Similar CSTM usage appears in category-level pose estimation via implicit MLPs (Liu et al., 2023), where learned transforms $d$3 map camera-space features $d$4 directly to canonical world-space features, eliminating the need for explicit deformation fields.

5. CSTM in Color Calibration and Cross-Camera Adaptation

CSTM appears in computational color constancy pipelines for synthesizing canonical representations of illumination across heterogeneous cameras. In CCMNet, the "canonical camera-space transformation" comprises calibrated $d$5 color correction matrices at anchor illuminants $d$6:

• Interpolated CSTM for arbitrary temperature $d$7:

$d$8

for $d$9 determined by $p_1 = [u_1; v_1; 1]$0 and the anchor temperatures (Kim et al., 10 Apr 2025).

• Canonicalization of spectra along the Planckian locus enables consistent mapping of XYZ colors into camera raw-RGB (or vice versa) across devices.
• The interpolated CSTM matrices provide the ground for sampling, guidance histogram creation, and device embedding via the camera-fingerprint encoder, ensuring cross-camera generalization.

This CSTM formalism is central in pipelines that require adapting color statistics and transforms on-the-fly, with virtually generated cameras supported by linear mixing of anchor CCMs for robust augmentation (Kim et al., 10 Apr 2025).

6. Practical Implementation Considerations

Canonical CSTM computation requires:

• Precise camera calibration (intrinsics and extrinsics) and, when working locally, accurate surface normal estimation (e.g., via disparity, RGB-D, or CNN-based methods) (Hajder, 6 Feb 2026).
• For global camera mapping, full-rank, projective camera matrices and their explicit factorization (Lu et al., 2014).
• In learning-based settings, architectures that enforce or learn explicit mappings from camera to canonical (world) space, co-trained with auxiliary losses for feature alignment, reconstruction, and pose supervision (Liu et al., 2023, Li et al., 2024).
• For color pipelines, extraction and interpolation of CCMs from firmware or metadata, and computation of histograms or embeddings for device fingerprinting (Kim et al., 10 Apr 2025).

Implementation details are domain specific and depend on whether CSTM is used for direct geometric transfer, learning-based regularization, or photometric normalization.

7. Applications and Theoretical Significance

The CSTM abstraction unifies geometric, photometric, and semantic mapping operations between cameras, world space, and canonicalized coordinates. Key applications include:

• Patch-level affine correspondence and robust matching in stereo, multi-view, or SLAM scenarios (Hajder, 6 Feb 2026).
• Camera calibration, reparameterization, and multi-view triangulation via LC factorization (Lu et al., 2014).
• Generalized projection and rasterization in graphics pipelines (supporting nonstandard frusta and advanced effects) (Glushkov et al., 2021).
• Learning-based canonicalization for 3D pose estimation, shape prediction, category-level pose, and cross-view consistency (Li et al., 2024, Liu et al., 2023).
• Device-invariant color adaptation and data augmentation in computational photography (Kim et al., 10 Apr 2025).

The canonical nature of CSTM ensures intrinsic, repeatable, and interpretable mappings that generalize across tasks and domains. The explicit algebraic forms and closed-form pipelines facilitate theoretical analysis, reproducibility, and principled design in both analytic and data-driven pipelines.