Generalizing ray-distortion rendering to visualize deformations on diverse neural surface representations

Develop a rendering method that visualizes deformations or fracture on neural surface representations by generalizing ray-distortion approaches (e.g., non-linear sphere tracing for deformed signed distance fields) to a broad range of surface formats, so that deformation visualization integrates with existing neural rendering pipelines across representations such as neural implicits, point clouds, Neural Radiance Fields, and Gaussian splats.

Background

The paper introduces a compact neural displacement field that maps a coarse base mesh to a high-fidelity surface represented in various non-mesh formats (e.g., neural implicits, point clouds, NeRFs, Gaussian splats). This enables robust geometry processing tasks, efficient mesh extraction, and transmission-friendly workflows for streaming applications.

While the method supports computation and mesh-based tasks, the authors highlight a challenge in visualizing deformations or fractures directly within neural rendering pipelines. Sampling point clouds on the fine surface can display deformations but does not leverage neural rendering. Prior work suggests ray-distortion techniques for deformed SDFs, yet extending such techniques to the diversity of modern surface representations remains unresolved.