Explain the performance gap between attention biasing and classifier-free guidance

Determine whether the performance gap between attention biasing and classifier-free guidance arises because attention biasing modifies pre-softmax weights, thereby maintaining a normalized attention distribution over the same tokens and remaining closer to the training regime.

Background

The paper compares an inference-time attention-biasing method with classifier-free guidance (CFG) for improving stylization in autoregressive diffusion-based motion generation. The reported experiments show that attention biasing performs better than CFG across the evaluation metrics while requiring less computational overhead.

The authors offer a mechanistic explanation for this observed performance difference: attention biasing changes the pre-softmax attention weights while preserving a normalized distribution over the same tokens, whereas CFG combines separate predictions and may produce outputs outside the training distribution. This explanation is explicitly presented as a conjecture rather than an established result, leaving its validity unresolved.

References

We conjecture that this performance gap arises because the bias modifies the pre-softmax weights, so attention stays a normalized distribution over the same tokens and thus closer to the training regime.

— Walk Like This: Stylized Locomotion for Physics-Based Characters through Example-Driven Diffusion  (2610.18147 - Aurand et al., 13 Oct 2026) in Section 5.2, “Inference Techniques,” subsection “Attention Bias”