---
title: 'PGCM: Cross-View Geometric Consistency'
url: https://www.emergentmind.com/topics/perspective-geometry-consistency-metric-pgcm
type: topic
---

# PGCM: Cross-View Geometric Consistency

Perspective-Geometry Consistency Metric (PGCM) can be understood as a cross-view geometric consistency criterion that evaluates whether predictions from overlapping viewpoints describe the same scene after projection into a common camera frame. In the formulation most directly associated with the term, the consistency quantity is a **cross-view reprojection error in inverse-depth**, evaluated in a common frame and masked by visibility. Although the 2019 mesh-correction paper did not name this quantity a “Perspective-Geometry Consistency Metric,” it provided nearly all of the ingredients for one through its geometric consistency loss [1909.03471]. Later work preserved the same basic concern—agreement under viewpoint change—while instantiating it through forward-backward reprojection, RGB-depth verification on perspective graphs, ego-centric cross-view consistency, pose-divergence diagnostics for video, failure-aware multiview verification, and RPC-constrained satellite evaluation [2310.19583][2510.10993][2606.16960][2603.19048][2605.18754][2606.17564].

## 1. Conceptual basis

The core premise is static-scene coherence across overlapping views. The foundational statement is explicit:

> “Intuitively, since the reconstructions we wish to correct are static, the predictions made from overlapping views should be geometrically consistent. In other words, surfaces that appear in a certain location according to a prediction should appear in the same location in all predictions where they are in view.” [1909.03471]

In this view, perspective inconsistency is not merely a local depth error. It is a cross-view failure: a surface position inferred from one viewpoint does not agree with what a nearby viewpoint predicts for the same 3D surface once both are expressed in the same camera frame. The 2019 formulation ties this failure mode directly to **overly smooth corrections**, stating that “smooth predictions result in geometrical inconsistencies” and addressing it with “a loss function which penalises re-projection differences that are not due to occlusions” [1909.03471].

This suggests a precise definition of PGCM: a metric should measure the discrepancy between two viewpoint-conditioned geometric predictions only where those predictions are jointly visible and therefore capable of agreement. In the original formulation, the relevant signal is not RGB photometric error and not generic depth error, but **agreement of corrected inverse-depth across nearby views after perspective reprojection into a common camera frame** [1909.03471].

## 2. Canonical inverse-depth formulation

The original construction operates on inverse-depth corrections. The ground-truth correction is

\[
\Delta^{*}(p) = d^{*}(p) - d(p),
\]

where \(p\) is a pixel index, \(d^{*}\) is the inverse-depth image from the high-quality reconstruction, and \(d\) is the inverse-depth image from the low-quality reconstruction [1909.03471]. The network predicts \(\Delta\), and the corrected inverse-depth is formed by adding the predicted correction to the low-quality inverse-depth.

The key consistency term is the **geometric consistency loss**:

\[
L_{gc} = \sum_{n \in N}\sum_{p \in U_n} \left|d_{n,t} - \hat d_{n,t}\right|.
\]

Here \(t\) is the target view, \(n \in N\) is a nearby view, \(d_t\) and \(d_n\) are corrected inverse-depth images, \(d_{n,t}\) is the target-view prediction expressed in the frame of view \(n\), \(\hat d_{n,t}\) is the nearby-view inverse-depth warped to correspond to target-view geometry and also expressed in view \(n\)’s frame, and \(U_n\) is the set of pixels unoccluded in nearby view \(n\) [1909.03471].

In the 2019 paper this quantity is an **auxiliary training loss**, weighted in the full objective by \(\lambda_{gc}=0.1\), rather than a standalone benchmark metric [1909.03471]. A paper-faithful metric interpretation was stated explicitly in the supplied synthesis: a metric version would simply remove dependence on training and report the normalized masked average reprojection inconsistency,

\[
\mathrm{PGCM}(t;N) = \frac{1}{\sum_{n\in N}|U_n|} \sum_{n\in N}\sum_{p\in U_n} \left|d_{n,t}(p)-\hat d_{n,t}(p)\right|.
\]

Lower values indicate better perspective-geometry consistency [1909.03471].

The significance of this formulation is that it separates **single-view accuracy** from **multi-view agreement**. A corrected depth map may appear locally plausible, yet still be inconsistent under perspective reprojection. PGCM, in this sense, measures whether the geometry itself survives viewpoint transfer.

## 3. Projection model, warping, and visibility

The consistency mechanism is built on a perspective camera model and inverse-depth parameterization. The 2019 paper motivates inverse-depth for three reasons: it emphasizes surfaces closer to the camera, background or non-surface pixels can be assigned zero corresponding to infinite depth, and resampling inverse-depth is simpler than resampling depth [1909.03471]. The last point is directly relevant to PGCM because the consistency term depends on dense warping.

For target view \(t\) and nearby view \(n\), let \(\Delta_t\) and \(\Delta_n\) be CNN predictions, and define corrected inverse-depth images
\[
d_t = d_t^{lq} + \Delta_t, \qquad d_n = d_n^{lq} + \Delta_n.
\]
With target pixel coordinates \(\mathbf{p}_t\), intrinsic matrix \(K\), and \(SE(3)\) transform \(T_{n,t}\), the paper defines

\[
\hat d_{n,t}(\mathbf{p}_t) = d_n(\mathbf{p}_n),
\]

\[
d_{n,t}(\mathbf{p}_t) = \frac{\mathbf{x}_n^{(4)}}{\mathbf{x}_n^{(3)} + \epsilon},
\]

\[
\mathbf{p}_n = \frac{1}{\mathbf{x}_n^{(3)} + \epsilon}
\left(\mathbf{x}_n^{(1)} \;\; \mathbf{x}_n^{(2)}\right)^T,
\]

\[
\mathbf{x}_n = F_h \mathbf{x}_t,\qquad
\mathbf{x}_t = \left(\mathbf{p}_t \;\; 1 \;\; d_t(\mathbf{p}_t)\right)^T,
\]

\[
F_h = K_h T_{n,t} K_h^{-1}, \qquad
K_h = \left( \begin{array}{cc} K & \mathbf{0} \\ \mathbf{0}^T & 1 \end{array} \right).
\]

The rationale is explicit: “Since we are warping inverse-depth images, where the pixel values depend on the viewpoint, we need to compute the absolute difference in the same camera frame” [1909.03471]. PGCM therefore compares \(d_{n,t}\) and \(\hat d_{n,t}\), not quantities that remain expressed in different viewpoint-dependent parameterizations.

Visibility is equally central. The 2019 method states that “views cannot be consistent in the presence of occlusions” and therefore computes occlusion masks from the reference high-quality reconstruction [1909.03471]. The mask construction is mesh-based. Each mesh triangle is assigned an index by hashing its world-frame coordinates, triangle-index images are rendered, and for each pair of views the target-view triangle ID is reprojected into the nearby view. Instead of interpolating triangle IDs, the four nearest samples are returned separately; if the target ID matches at least one of those four samples, the pixel is considered unoccluded, otherwise occluded [1909.03471].

Two clarifications are methodologically important. First, the paper does **not** use a depth threshold, z-buffer difference threshold, or learned occlusion confidence; the occlusion test is discrete triangle identity matching [1909.03471]. Second, the visibility set \(U_n\) has “no relation to the set of valid pixels (\(V\)) from the previous losses,” because the consistency term is computed between predictions and can therefore supervise regions with no valid label [1909.03471].

Dense warping uses linear interpolation of inverse-depth under the “mild assumption that surfaces between pixels are planar,” which the paper describes as another advantage of the inverse-depth formulation [1909.03471]. This gives PGCM a differentiable resampling interpretation even though the original paper presents it as a loss rather than a benchmark score.

## 4. Relationship to edge preservation and standard evaluation

The 2019 system does not rely on geometric consistency alone. It couples \(L_{gc}\) to a data term, a gradient term, and regularization:

\[
L = \lambda_{data} \sum_s L_{data}^{s}
+ \lambda_{grad} L_{grad}
+ \lambda_{gc} L_{gc}
+ \lambda_{reg} L_{reg},
\]
with \(\lambda_{data}=1\), \(\lambda_{grad}=0.1\), \(\lambda_{gc}=0.1\), and \(\lambda_{reg}=10^{-6}\) [1909.03471].

The **data loss** uses the berHu norm, and the **gradient loss** preserves local structure and sharp transitions. The system also introduces edge-aware weighting:
\[
d(p) = \ln(1 + \mathrm{EDT}(p)),
\]
\[
w(p) = (w_{max} - w_{min}) \left(1 - \frac{d(p)}{\max_{p} d(p)}\right) + w_{min},
\]
with \(w_{min}=0.1\) and \(w_{max}=5\) [1909.03471]. Pixels near edges receive larger weights.

The relationship between these terms and PGCM is explicit but indirect. The paper treats edge preservation and geometric consistency as distinct mechanisms, yet states that excessive smoothing causes geometrical inconsistencies [1909.03471]. Thus edge-aware weighting does not appear inside \(L_{gc}\), but it improves the corrected inverse-depth fields supplied to the consistency term. A plausible implication is that a PGCM computed on oversmoothed predictions will often increase because occluding boundaries and sharp geometric transitions shift differently across viewpoints.

The paper also reports standard inverse-depth evaluation measures that are **not** PGCM measures. These include thresholded accuracy
\[
\delta = \mathbb{E}_{p \in V} \left[ \mathbb{I}\left(\textrm{max}\left( \frac{d^{*}(p)}{\hat d(p)}, \frac{\hat d(p)}{d^{*}(p)}\right) < thr\right) \right],
\]
with \(thr \in \{1.05, 1.15, 1.25, 1.25^2, 1.25^3\}\), as well as
\[
\textrm{iMAE} = \frac{1}{n}\sum_{p \in V}{\left|d^{*}(p) - \hat d(p)\right|},
\qquad
\textrm{iRMSE} = \sqrt{\frac{1}{n}\sum_{p \in V}{(d^{*}(p) - \hat d(p))^2}}
\]
[1909.03471]. These are per-view prediction quality metrics rather than cross-view consistency metrics.

This distinction motivates a common correction in interpretation: PGCM is not equivalent to gross-error correction, iMAE, iRMSE, PSNR, or any other single-view fidelity statistic. In the original study, the model “reduces gross errors by 45.3\%–77.5\%, up to five times more than previous work,” and adding geometric consistency improves performance “at all error scales,” but those gains remain indirect evidence for PGCM rather than the metric itself [1909.03471].

## 5. Generalizations across later research

Later work preserved the central PGCM intuition—agreement under viewpoint change—but instantiated it differently depending on geometry model, overlap regime, and task.

In multi-view stereo, GC-MVSNet defines consistency by **forward-backward reprojection agreement** between a predicted reference-view depth map and multiple source-view ground-truth depth maps. Its explicit per-source quantities are the **Pixel Displacement Error**
\[
PDE = \|P_0 - P''_0\|_2
\]
and the **Relative Depth Difference**
\[
RDD = \frac{1}{D_0}\|D''_{P''_0} - D_0\|_1,
\]
followed by thresholded inconsistency masks and multi-view aggregation through a normalized inconsistency frequency [2310.19583]. This formulation makes PGCM more explicitly cycle-consistent and multi-scale than the 2019 inverse-depth loss.

In 3D Gaussian inpainting, PAInpainter replaces direct reprojection residuals with a **dual-feature verification mechanism**. Neighbors are selected from a **perspective graph** built from LoFTR match confidence, anchor content is projected into adjacent views using depth and camera transforms, and candidate inpaintings are scored by
\[
S = \eta S_{rgb} + (1-\eta) S_{depth},
\qquad \eta = 0.7,
\]
with acceptance threshold \(T_s=0.9\) in iterative refinement [2510.10993]. This does not define a standalone PGCM by name, but it operationalizes local perspective-geometry consistency as graph-conditioned RGB-depth agreement.

For low-overlap surround-view driving, SurroundNEXO shifts the problem from dense cross-view correspondence to **ego-centric geometric comparability**. It introduces **Ego-Ray Positional Encoding**, in which image patches are mapped to ego-frame viewing directions,
\[
\mathbf{r}_{v,i}^{e} = R_{c_v\rightarrow e} \frac{K_v^{-1}\tilde{\mathbf{p}}_{v,i}} {\left\|K_v^{-1}\tilde{\mathbf{p}}_{v,i}\right\|_2},
\]
and evaluates cross-view depth consistency by calibrated reprojection only inside valid overlap [2606.16960]. This suggests that in weak-overlap regimes a PGCM should not rely on dense correspondence density alone.

Video-oriented variants make the same notion temporal. SGC measures 3D spatial geometric consistency by separating static from dynamic regions, estimating local camera poses for depth-clustered static subregions, and quantifying divergence through local and global rotational and translational variances plus warped depth consistency,
\[
E_{depth} = \frac{1}{|\mathcal{V}|} \sum_{\mathbf{u} \in \mathcal{V}} \left| D_{i-1 \to i}(\mathbf{u}) - D_i(\mathbf{u}) \right|
\]
[2603.19048]. PDI-Bench instead defines an object-centric Perspective Distortion Index,
\[
\text{PDI} = w_1 \cdot \operatorname{RMSE}(\epsilon_{scale}) + w_2 \cdot \operatorname{RMSE}(\epsilon_{traj}) + w_3 \cdot \epsilon_{rigidity},
\]
with default weights \((0.4,0.4,0.2)\), thereby turning perspective scale law, 3D trajectory smoothness, and rigidity preservation into an explicit video PGCM [2605.15185].

Ground-truth-free multiview verification introduces a different emphasis: **failure-aware consistency**. The COLMAP-based family in “Can These Views Be One Scene?” defines per-pixel bounded quality
\[
q_v(\mathbf{u})=
1-\mathrm{clip}\!\left(
\frac{\left|\mathcal{D}^{(v)}_{p}(\mathbf{u})-\mathcal{D}^{(v)}_{g}(\mathbf{u})\right|}
{\tau\,\max(\mathcal{D}^{(v)}_{g}(\mathbf{u}),10^{-6})},0,1\right),
\qquad \tau=0.2,
\]
and scene-level scores such as geometric–photometric consistency and coverage-weighted GPC [2605.18754]. Here the crucial principle is that inability to verify geometry should lower the score rather than vanish as missing data.

Satellite multi-view evaluation under RPC geometry extends PGCM beyond pinhole cameras. The proposed protocol defines **RPC projected 3D feature consistency** through
\[
s_{i,j} = \left\langle \tilde{\mathbf{f}}_r(\mathbf{x}_i), \tilde{\mathbf{f}}_j(\mathbf{x}_{i}^{(j)}) \right\rangle,
\qquad
s_i = \frac{1}{|\mathcal{V}_i|}\sum_{j \in \mathcal{V}_i} s_{i,j},
\]
and a **geometry-constrained dense matching proxy** using EPE and PCK@10 inside an epipolar band of \(\pm 4\) pixels around the curved, height-dependent RPC correspondence manifold [2606.17564]. Its central finding is that semantic agreement and geometric localization decouple: high similarity at the correct 3D point does not guarantee sharp, usable matchability.

Adjacent work broadens the conceptual perimeter of PGCM rather than its exact formula. A diffusion-model study introduces a vanishing-point-based perspective loss over edge profiles, effectively a single-image perspective-structure consistency measure [2312.00944]. MMPerspective, by contrast, operationalizes perspective consistency in multimodal reasoning as correctness preserved across perspective-preserving transformations such as cropping, masking, flipping, and rotation [2505.20426]. These are not cross-view reprojection metrics, but they indicate that PGCM-style reasoning can be instantiated as either geometric verification or invariance under geometry-preserving perturbation.

## 6. Limitations, misconceptions, and methodological boundaries

A first misconception is historical. In the 2019 mesh-correction work, the consistency term is clearly a **training loss**, not a standalone evaluation metric [1909.03471]. Treating the paper as having introduced a named benchmark metric would be inaccurate. A PGCM emerges by reinterpretation: the same masked reprojection discrepancy can be normalized and reported post hoc.

A second misconception is that any geometric-looking scalar is automatically a perspective-geometry metric. Standard per-view measures such as iMAE, iRMSE, thresholded accuracy, PSNR, SSIM, LPIPS, or FID do not, by themselves, quantify cross-view perspective agreement [1909.03471][2510.10993]. They may correlate with consistency, but they do not define it.

A third misconception concerns visibility. The 2019 formulation does not use thresholded depth comparison or learned occlusion confidence; it uses triangle-identity agreement from a reference mesh [1909.03471]. Other settings likewise require task-specific validity logic: overlap-only evaluation in low-overlap surround-view depth [2606.16960], static-region masking in generated video [2603.19048], or geometry-constrained search manifolds in RPC satellite imagery [2606.17564]. A plausible implication is that no PGCM is meaningful without a clearly specified valid-support set.

A fourth boundary concerns localizability versus agreement. The RPC study shows that very high projected 3D feature consistency can coexist with poor localization under matching inference [2606.17564]. Thus a descriptor can look geometrically consistent at the true 3D point while still failing to produce a sharp, unique response over the physically valid search manifold. PGCMs based only on similarity at correspondences may therefore overstate practical usefulness.

A fifth boundary concerns hallucinated geometry. The multiview robustness study demonstrates that learned reconstruction backbones can hallucinate dense geometry and cross-view support for unrelated scenes, repeated images, and random noise, while classical failure-aware verification can correlate far better with human judgments [2605.18754]. This suggests that a PGCM built entirely on learned residuals may inherit the failure modes of its backbone unless it explicitly accounts for registration failure, missing dense support, or narrow viewpoint coverage.

Finally, perspective consistency is not identical across domains. Inverse-depth mesh correction, MVS forward-backward reprojection, perspective-graph inpainting, ego-frame driving, static-background video audits, panorama stitching, and RPC satellite matching all preserve the same intuition—consistency under geometry-aware transfer—but use different camera models, visibility assumptions, and aggregation rules [1909.03471][2310.19583][2603.12013][2606.17564]. The most defensible general statement is therefore narrow: PGCM denotes a family of metrics derived from the question of whether geometry predicted or inferred in one view remains compatible with geometry in other views once the relevant camera model, valid-support set, and aggregation rule are made explicit.

Source: https://www.emergentmind.com/topics/perspective-geometry-consistency-metric-pgcm