Explain when pre-merge compression is preferable to post-merge token merging

Derive a principled explanation of when input-level pre-merge compression should be preferred over post-hoc semantic token merging for constructing high-compression visual tokenizers.

Background

The paper compares two strategies for adapting pretrained semantic encoders to a 32× token grid. Input-level pre-merge compression resizes the image before semantic encoding, whereas post-hoc compression merges already extracted semantic tokens. Experiments show that pre-merge compression produces better generation quality, but the authors do not establish the general conditions under which one strategy is preferable. A principled account would clarify how compression interacts with semantic abstraction, spatial information, and reconstruction or generation performance.

References

In addition, while we observe that semantic latents can accelerate diffusion training and that input-level pre-merge compression outperforms post-hoc token merging, we do not yet have a complete understanding of which properties of semantic encoders are most important for generation. A deeper analysis of semantic encoder selection, as well as a principled explanation of when pre-merge or post-merge compression is preferable, remains an important direction for future work.

— DC-SAE: Deep Compression Semantic Autoencoder for Faster Diffusion Convergence  (2609.39222 - Huang et al., 30 Sep 2026) in Appendix, subsection “Limitations and Future Work”