View-Consistency Benchmark

• View-Consistency Benchmark is a suite of evaluation metrics designed to assess consistency across multiple data and image views in both distributed systems and generative modeling.
• It employs defined measures like k-atomicity, Δ-atomicity, and feature-based similarity to quantify staleness and geometric integrity.
• Empirical evaluations highlight trade-offs in performance and robustness, informing practical system tuning and model design improvements.

A view-consistency benchmark is an evaluation methodology and metric suite designed to quantify consistency across multiple client or system "views" of data or generated content. In distributed systems, it characterizes staleness and anomalies in key-value store replicas as observed by clients. In computer vision and generative modeling, it quantifies geometric and appearance coherence across synthesized multi-view images or 3D reconstructions. Recent benchmarks combine formal metrics, systematic data curation, and rigorous evaluation protocols to provide actionable, minimally intrusive, and generalizable measures of operational or generative consistency.

1. Formal Definitions and Consistency Metrics

View-consistency benchmarking is grounded in rigorously defined, application-specific metrics. In distributed storage, client-centric metrics such as k-atomicity and Δ-atomicity formalize version-based and time-based staleness, respectively. For a key-value store execution $E$:

• k-atomicity: $E$ is $k$-atomic if all reads return one of the $k$ most recent writes in a total order extending real-time ("happens-before") relationships. The minimal $k$ quantifies version staleness (Rahman et al., 2012).
• Δ-atomicity: $E$ is Δ-atomic if, after shifting each read's start time back by Δ, the execution is linearizable. No read returns a value overwritten more than Δ time before its start, providing a time-bounded staleness measure.
• Per-read staleness score, $\chi(r)$: For each read $r$ returning value $v$, $$\chi(r) = \max(0,\, \text{start}(w'') - \text{end}(r))$$, where $w''$ is the earliest write of a newer value.

Within generative AI and 3D synthesis, view-consistency is measured by:

• Feature-based view consistency (MEt3R): Given images $I_1, I_2$ from differing views, dense 3D correspondences are inferred (e.g., via DUSt3R), features are warped to overlapping regions, and cosine similarity is averaged. The symmetric metric is $1 - \frac{1}{2}(S(I_1,I_2) + S(I_2,I_1))$, with $0$ indicating perfect consistency (Asim et al., 10 Jan 2025).
• Self-consistency (MVGBench): 3D reconstructions from disjoint sets of generated views (using Gaussian Splatting) are compared via Chamfer distance (geometry), mean depth error, and texture measures (conditional PSNR/SSIM/LPIPS), all without reliance on ground-truth 3D (Xie et al., 11 Jun 2025).
• Multi-view perceptual metrics: PAInpainter employs PSNR, SSIM, LPIPS, and FID for appearance and distributional assessment, augmenting these with cross-view cosine similarity of fused RGB and depth features for candidate selection (Cheng et al., 13 Oct 2025).

These metrics are designed to be robust to generative ambiguity, operational heterogeneity, or lack of ground-truth, and emphasize concrete client-observed or model-generated anomalies.

2. Benchmarking Methodologies and Algorithms

View-consistency benchmarks implement the rigorous measurement of consistency through tailored methodologies:

• End-to-end distributional benchmarking: In distributed storage, every client operation (get/put) is instrumented with high-resolution timestamps and results, using realistic workloads such as YCSB, with logs analyzed post-mortem to group and cluster operations per key and value, yielding per-read and global staleness statistics (Rahman et al., 2012).
• No-disruption principle: All operations are drawn from the real workload, avoiding injected probes or middleware, with per-operation overhead maintained below 5%, and all analysis performed offline to minimize measurement bias.
• Multi-view generative evaluation: For generative models, multi-view datasets (e.g., SPIn-NeRF, NeRFiller, RealEstate10K, CO3D, GSO) supply ground-truth or real-world image sequences. Evaluation involves splitting generated views, reconstructing and aligning 3D representations (Gaussian splatting), and comparing projections and features across disjoint subsets (Xie et al., 11 Jun 2025, Cheng et al., 13 Oct 2025, Asim et al., 10 Jan 2025).
• Adaptive neighbor sampling and cross-view propagation: In PAInpainter, a graph of view perspectives is constructed using transformer feature matchers, enabling adaptive sampling, content propagation, and candidate selection with consistency verification enforced via fused RGB/depth features (Cheng et al., 13 Oct 2025).

These procedures are explicitly designed to assess view consistency in minimally intrusive, scalable, and generalizable ways across systems and generative models.

3. Data, Datasets, and Evaluation Protocols

The reliability of view-consistency benchmarking critically depends on dataset choice and evaluation protocol:

Benchmark	Dataset(s) Used	Evaluation Focus
MVGBench	GSO, OmniObject3D, MVImgNet, CO3D	Geometry, texture, semantics
PAInpainter	SPIn-NeRF, NeRFiller	3D inpainting, multi-view
MEt3R	RealEstate10K (100 sequences)	Image generation, consistency
Key-Value SUT	YCSB	Distributed consistency

In generative settings, datasets are selected for diversity in camera pose, lighting, object category, and realism. Protocols include best-setup (native camera parameters), robustness analysis (perturbed elevations, lighting), and real-world generalization with alignment of predicted 3D models (Xie et al., 11 Jun 2025). Evaluations involve rendering, feature alignment, and per-metric normalization to facilitate fair and comparable scoring.

On real-world distributed stores, experiments are performed with high-fidelity clock synchronization and configurable failure injection (node, rack, network levels). This ensures benchmarks cover operationally plausible fault modes, key distributions, and client behaviors (Rahman et al., 2012).

4. Quantitative Results and Comparative Analysis

State-of-the-art benchmarks reveal critical trade-offs in current systems and models:

• On distributed key-value stores (Cassandra, 3-node replication, 80/20 hotspot YCSB workload), per-read staleness $\chi(r)$ was < 4 ms for most queries, with rare spikes up to 233 ms. The observed global Δ averaged 53 ms, markedly lower than the worst-case theoretical bounds (Rahman et al., 2012). This suggests practical deployments benefit from empirical benchmarking to tune SLOs and replication.
• Within PAInpainter, perspective-aware propagation and dual-feature checking yielded PSNR improvements up to 2.54 dB (NeRFiller: 29.51 dB PSNR, 0.94 SSIM, 0.08 LPIPS) and FID reductions versus baselines (Cheng et al., 13 Oct 2025). Ablation demonstrates that each protocol component (graph sampling, content propagation, consistency verification) progressively increases view consistency.
• MEt3R separates structural and appearance drift in generative models: DFM attains near-perfect consistency (MEt3R 0.026), while MV-LDM achieves a better quality-consistency trade-off (Asim et al., 10 Jan 2025). Ground-truth video lower bounds (MEt3R ≈ 0.022) establish empirical noise floors.
• MVGBench shows that traditional metrics penalize plausible generative samples overly harshly, and that models like ViFiGen (SV3D+CaPE+ConvNextV2) achieve state-of-the-art 3D consistency (Chamfer 3.15, cPSNR 28.93, cSSIM 0.897), yet all current methods degrade on real images and struggle under robustness perturbations (Xie et al., 11 Jun 2025).

5. Design Insights, Best Practices, and Applications

Extensive ablation and cross-model comparisons yield insights for future system and model design:

• Richer camera embeddings (Plücker RCN, CaPE) and stronger input encoders (ConvNextV2, DINOv2) lead to measurable gains in self-consistency scores (Xie et al., 11 Jun 2025).
• Multi-view synchronization—such as video-diffusion ST attention—enforces cross-view coherence, often making additional architectural complexity unnecessary.
• Increased data scale (50k–150k objects) is essential for generalization.
• Best practices in distributed benchmarking include lightweight instrumentation, log-based analysis, and avoidance of workload perturbation, enabling realistic SLO enforcement and operational tuning (Rahman et al., 2012).
• Feature-based consistency metrics (e.g., MEt3R) provide differentiable, pose-free, dataset-independent loss functions adaptable to closed-loop consistency training (Asim et al., 10 Jan 2025).
• Realism and robustness require evaluation under misalignments, varying lighting, and limited-view scenarios—current methods exhibit notable sensitivity to these factors.

6. Limitations and Future Challenges

View-consistency benchmarks, while foundational, are subject to a range of technical caveats:

• Metrics requiring high-quality 3D reconstruction (DUSt3R, Gaussian Splatting) can be compromised by texture-poor regions or misalignment, especially on real unconstrained data (Asim et al., 10 Jan 2025, Xie et al., 11 Jun 2025).
• Feature-extraction backbones can shift absolute values but are generally reliable for relative ranking.
• No current evaluation completely resolves generative ambiguity: valid but diverse samples may still be penalized.
• Synthetic–real generalization gaps remain significant, with performance drops measured across all major methods (Xie et al., 11 Jun 2025).
• Future efforts are directed at few-view generation, diverse scene categories (e.g., articulated, outdoors), more robust and scalable 3D fitting, and high-fidelity Gaussian splatting (Xie et al., 11 Jun 2025).

Systematic, multifaceted, and minimally biased view-consistency benchmarking is an area of continued research, essential for quantifying and advancing both distributed system robustness and generative model coherence.