Stable post-training for joint audio-video generators

Establish a stable post-training procedure for a joint audio-video generator with modality-specific backbones and a bidirectional interaction module.

Background

The paper explains that joint audio-video post-training must simultaneously address visual quality, audio quality, text consistency, cross-modal semantics, and temporal synchronization. These objectives may conflict, and a single global reward cannot reliably attribute improvements or regressions to the audio stream, video stream, or cross-modal interaction module.

The authors note that existing audiovisual preference-optimization methods demonstrate feasibility but do not resolve how to perform stable post-training for architectures that retain modality-specific backbones while incorporating bidirectional cross-modal interaction. The problem is therefore directly relevant to improving the reliability and controllability of reinforcement-learning or preference-alignment procedures for native joint audio-video generation.