Cross-scene generalization of the latent refiner

Develop a universal neural refiner that generalizes across scenes rather than being trained on a single image in a per-scene setting, to improve the practicality of physically based latent rendering.

Background

The latent rendering pipeline separates physically based latent components from residual latent components. A feed-forward neural refiner predicts the residuals using the rendered latent and auxiliary geometric buffers such as surface normals and depth. In the presented method, the refiner is trained independently for each scene using only one ground-truth rendering.

Although the refiner generalizes to intra-scene changes such as camera, lighting, and object edits, the paper identifies its per-scene training and lack of cross-scene generalization as a limitation. A universal refiner operating across different scenes would make the approach more practical and broadly applicable.

References

For simplicity, we train this neural refiner on a per-scene basis using a single ground truth rendering. We leave the generalization of the refiner across scenes for future work.

— Physically Based Rendering in the Latent Space  (2609.21054 - Radovanovic et al., 17 Sep 2026) in Section 4.3, subsection “Latent refinement”; reiterated in Section 5, subsection “Limitations and future work”