Rational Camera Models

• Rational camera models are a mathematical framework that maps 3D world points to 2D image coordinates using ratios of polynomials, generalizing the pinhole model.
• They capture complex imaging geometries, enabling practical applications such as satellite geolocation, image rectification, and multi-view reconstruction in various non-central systems.
• Parameter estimation involves regularized least squares over extensive 3D–2D correspondences, ensuring numerical stability through normalization and iterative refinement techniques.

A rational camera model is a general mathematical framework for expressing the mapping from 3D world points to 2D image coordinates using rational functions—ratios of polynomials—in the scene coordinates. This class of models subsumes the perspective (pinhole) camera as a special case and encompasses a wide variety of nonlinear and non-central imaging systems, such as push-broom and satellite cameras, rolling-shutter cameras, and two-slit or multi-slit systems. Rational camera models provide the basis for practical geolocation, image rectification, and multi-view geometry in modern computer vision, especially in remote sensing, computational photography, and multi-view stereo pipelines where standard perspective assumptions fail.

1. Mathematical Foundation and Model Classes

Rational cameras model the imaging process by expressing each image coordinate as a rational function of the world coordinates. In homogeneous notation, a rational camera is typically described by

$$x = \frac{P_1(X,Y,Z)}{Q_1(X,Y,Z)}, \qquad y = \frac{P_2(X,Y,Z)}{Q_2(X,Y,Z)},$$

where $P_i$ and $Q_i$ are polynomials (commonly cubic, i.e., total degree $\leq 3$) in the 3D coordinates. The best-known practical instantiation is the rational polynomial camera (RPC) or rational functional model (RFM) used for satellite and aerial sensors, formalized in both forward and inverse projection forms, and encoded as a parameter file with 78 coefficients for the polynomials plus normalization parameters (Akiki et al., 2021, Danyang et al., 2023).

The general rational-camera theory developed in (Trager et al., 2016) formalizes these models as compositions of (i) an essential map from $P^3$ to the Grassmannian of lines $\mathrm{Gr}(1,3)$ (describing the family of rays mapping scene points to the camera), (ii) intersection with a retinal plane, and (iii) projection to image coordinates. The algebraic degree and congruence properties of the underlying family determine the complexity of forward/inverse projection and multi-view constraints.

Beyond RPCs, advanced rational camera models encompass:

• Order-one rolling-shutter cameras ("RS$_1$") in which a moving projection center produces a single-valued rational mapping (with explicit algebraic inversion) between 3D space and image points (Hahn et al., 2024).
• Multi-slit, two-slit, and other multi-center models, which use rational maps determined by the intersection of the scene point rays with a retina plane and can be described compactly by pairs of linear maps or polynomials (Trager et al., 2016).

2. Parameterization, Estimation, and Normalization

Parameter fitting for rational cameras is typically formulated as a regularized least squares problem over a (possibly large) set of 3D–2D correspondences. For cubic RPCs, one encodes the 20 monomial coefficients for each numerator and denominator, fixes denominator constants to avoid ambiguity, and solves for 78 unknowns using a design matrix and normalization of both world and image coordinates to $[-1, 1]$ ranges (Akiki et al., 2021).

Regularization is essential to prevent ill-posedness (especially with limited or poorly-distributed correspondences) and is governed by Tikhonov weights, sometimes selected via L-curve heuristics. For highest precision (sub-pixel residuals), 1,000–25,000 correspondences (e.g., via a uniform grid over the 3D scene box) are typically sampled. Weighted least squares may be used to account for the denominator’s effect, updating weights iteratively based on current parameter estimates. Iterative refinement can also include bias-correction steps such as ICCV (Iteration by Correcting Characteristic Value) to address systematic errors.

Normalization offsets and scales for each variable are included both for numerical stability and to ensure model applicability across various scene extents and resolutions. Denormalization restores coefficients for use in real-world coordinate systems (Akiki et al., 2021).

3. Physical Interpretation and Model Limitations

A core distinction of rational camera models is their decoupling from explicit physical imaging geometry. Unlike perspective or pinhole models, which are parameterized by camera center, focal length, principal point, and (optionally) lens distortion, general rational cameras (notably RPCs) have no focal length or intrinsic/extrinsic decomposition. The mapping is purely empirical and sensor-specific, reflecting the actual scanning and projection of linear-array or synthetic-aperture sensors. While this leads to sub-pixel accuracy and universality across vendor/sensor types, it makes direct integration with standard bundle adjustment and epipolar geometry pipelines cumbersome (Danyang et al., 2023).

For multi-view and 3D reconstruction tasks, this incompatibility necessitates either custom warping (e.g., for deep stereo or MVS) or the derivation of an "equivalent" pinhole camera approximation ("REPM"—Refined Equivalent Pinhole Model) that fits a perspective model as closely as possible to the rational mapping over a given scene section. Such equivalence is only locally accurate (under weak perspective) and incurs systematic reprojection errors that grow with image size and elevation range, demanding further correction via polynomial refinement warps (Danyang et al., 2023, Gao et al., 2021).

4. Applications: Remote Sensing, Rolling Shutter, and Multi-Slit Geometry

Rational camera models are foundational in several modern imaging systems:

• Satellite and aerial imaging: High-resolution push-broom and optical satellite sensors universally adopt cubic RPCs both for vendor geolocation and open-source processing (Ikonos, WorldView, ZY3, Gaofen-7, Sentinel). These models enable sensor-agnostic rectification, map projection, and accurate DSM/DEM generation without explicit knowledge of onboard geometry (Akiki et al., 2021, Danyang et al., 2023, Gao et al., 2021).
• SAR geolocation: Rational polynomial models are fitted to SAR data using synthetic correspondences sampled via rigorous range-Doppler models, allowing efficient SAR-to-ground projection with millimeter accuracy (Akiki et al., 2021).
• Rolling-shutter camera pose: Order-one RS models (RS$_1$) generalize the pinhole model to account for image distortions from row-by-row readout and camera motion, defining a rational map with explicit inversion and multi-view constraints with minimal solvers (Hahn et al., 2024).
• Multi-slit/two-slit cameras: Abstract models for imaging systems whose rays pass through two or more fixed lines (slits) define rational maps with 14-dimensional parameter spaces (14 for projective, 11 for affine, 8 intrinsic for Euclidean calibration). Their epipolar tensors, primitive forms, and self-calibration algorithms have been formalized, laying a rigorous foundation for structure-from-motion and metric upgrades in such systems (Trager et al., 2016).

5. Integration in Learning-Based Pipelines and Practical Warping

Recent work has established the use of rational camera models within deep learning and differentiable vision pipelines. For instance, in satellite MVS, direct pixel-domain warping between views involves recasting the 20 monomial coefficients for each projection polynomial as fully symmetric tensors (4×4×4), so warping reduces to batched tensor contractions within standard network frameworks (Gao et al., 2021). This enables full differentiability with respect to world coordinates and hypothesized height planes, unlocking city-scale 3D reconstruction and change detection pipelines for any satellite platform without explicit homography or intrinsic parameter computation.

The SatMVS system, for example, replaces pinhole homography warping with a full RPC warper, utilizes UNet-style feature extractors, 3D cost volumes, ConvGRU/3D-conv regularizers, and performs supervised height regression, yielding superior accuracy compared to pinhole approximations. This reflects the broader principle that rational camera models, by encoding all non-linear physical, geometric, and sensor-specific distortions within their coefficients, enable true sensor-agnostic computer vision at scale (Gao et al., 2021).

6. Model Conversion, Error Analysis, and Residual Correction

To interface rational camera models with standard computer vision algorithms, a common strategy involves reformulating the rational map as an "equivalent" pinhole camera over a restricted scene ("weak perspective") and applying a polynomial correction for the residual bias:

• Fit a 3×4 projection matrix $P$ using SVD to align 3D scene points with their image projections.
• Decompose $P_i$0 into intrinsics ($P_i$1), rotation ($P_i$2), and translation ($P_i$3). This provides a perspective surrogate for the original rational mapping.
• Quantify the local reprojection bias using the error formula $P_i$4, reflecting growth with image size and terrain relief (Danyang et al., 2023).
• Further absorb systematic errors via a low-order bivariate polynomial transformation, whose coefficients are fitted to align the pinhole model's projections with the original rational camera outputs in a least-squares sense.

Empirical results across multiple open datasets (WHU-TLC, DFC2019, ISPRS-ZY3, GF7) show substantial improvements in 3D reconstruction RMSE and completeness—especially for very large satellite images when employing the polynomial correction, confirming the criticality of residual modeling for high-precision applications (Danyang et al., 2023).

7. Theoretical Significance, Model Hierarchies, and Future Directions

The rational camera framework provides a generalization of the central, pinhole model that is closed under projective transformations and suitable for both physical and abstract imaging systems. It yields a hierarchy:

• Pinhole (central) cameras: single center, linear projection, 5 intrinsic parameters.
• Two-slit/multi-slit models: multi-center, rational maps of projective degree, with explicit calibration and self-calibration procedures (Trager et al., 2016).
• Order-one rolling-shutter models: rational, single-valued, with full parameter classification and practical pose solvers (Hahn et al., 2024).
• General cubic rational polynomial cameras (RPCs): universal surrogate for practical satellite, SAR, and non-central sensors (Akiki et al., 2021, Danyang et al., 2023).

As sensor modalities and imaging geometries diversify (e.g., with multi-line, rolling, event-based, or plenoptic cameras), rational models offer a unified analytical foundation bridging empirical calibration, algebraic vision, and learning-based processing. A plausible implication is that future advances will further integrate higher-degree, multi-parameter rational model fitting, robust tensor-based warping, and analytic-aided self-calibration, especially for non-central and non-conventional imaging systems.