Accurate evaluation of spatial simulation for camera‑controllable generation

Develop rigorous and unambiguous evaluation methodologies and benchmarks to accurately assess spatial simulation in camera‑controllable image generation, addressing the ambiguity of calibration‑based evaluators when generated images exhibit subtle geometric differences so that geometric consistency can be measured precisely.

Background

The paper currently evaluates camera‑controllable image generation by estimating pixel‑wise camera maps using an offline vision‑based calibration method and then computing angular errors. The authors note that, while this is the best available practice, the reported calibration errors can be ambiguous, especially when spatial differences between generated images are subtle.

They emphasize that precise evaluation of spatial simulation is essential to advance camera‑controllable generation and suggest using stronger camera understanding models as evaluators and designing benchmarks that better capture geometric consistency.

References

Accurately evaluating spatial simulation thus remains an open challenge and is crucial for advancing camera-controllable generation.

— Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation  (2510.08673 - Liao et al., 9 Oct 2025) in Limitation and Future Work (near end of paper)

Simulation and UR5e results support the control utility of this representation within the evaluated adaptation settings, while independent verification of hidden-scene geometry remains open.

— LIFD: Anchored Diffusion for 3D-Aware Scene Memory in Robotic Manipulation  (2609.19796 - Li et al., 17 Sep 2026) in Section Conclusion

EgoGenEval covers static indoor scenes, four camera-action families, and rollouts of up to three steps; dynamic scenes, wider motions, and longer horizons remain open.

— Beyond Visual Quality: Evaluating Physical Consistency under Ego-Motion with EgoGenEval  (2609.11172 - Long et al., 10 Sep 2026) in Section 5, Discussion and Limitations, paragraph “Limitations”