Joint optimization without co-adaptation

Determine whether joint end-to-end optimization schemes for the ZeroDiff prior-construction mapping and diffusion denoiser can retain the performance gains observed on some datasets while suppressing co-adaptation that harms zero-shot transfer to unobserved locations.

Background

ZeroDiff ordinarily trains the informed-prior construction module and diffusion denoiser in two stages, preventing gradients from the diffusion loss from flowing into the prior components. The appendix compares this gradient-isolated procedure with joint end-to-end optimization, in which the diffusion loss also updates the dynamics model and prior construction through a differentiable denormalization–renormalization chain.

Joint optimization improves performance on the Solar, Temp, and Streamflow datasets but reduces performance on Methane. The paper attributes this negative result to co-adaptation between the prior and the denoiser on observed locations, which may not transfer to unobserved locations when the exogenous-to-target relationship is weak or underdetermined. The unresolved problem is therefore to design joint schemes that preserve potential optimization gains without sacrificing cross-location zero-shot generalization.