CompactFusion: Accelerated Diffusion Inference

Updated 7 July 2026

CompactFusion is a communication-centric framework that compresses activation residuals to reduce data transfer during multi-GPU diffusion inference.
It employs delta compression with error feedback, exploiting temporal redundancy to maintain generation fidelity while significantly decreasing latency.
The approach integrates seamlessly into existing pipelines with minimal code changes, outperforming overlap-based methods in both performance and accuracy.

Searching arXiv for “CompactFusion” and closely related papers to ground the article in current literature. CompactFusion is a communication-centric acceleration framework for parallel diffusion inference that reduces inter-device traffic by transmitting compressed step-wise activation residuals rather than full activations at every denoising step. It is designed for multi-accelerator serving of diffusion Transformers, where sequence, patch, pipeline, or tensor-style parallelism can become communication-bound even when compute is abundant. The framework is presented as a thin wrapper around communication primitives, is training-free, and is reported to integrate into xDiT and distrifuser with minimal code changes while preserving generation quality more effectively than overlap-based methods that rely on stale activations (Luo et al., 23 Jul 2025).

1. Problem formulation and architectural scope

CompactFusion addresses a specific systems bottleneck in diffusion serving: the exchange of large activations across devices at every diffusion step. In the reported setting, models such as FLUX.1-dev and CogVideoX-2B require multi-GPU execution for practical latency, but standard parallel inference sends large tensors repeatedly, so communication becomes a dominant fraction of runtime. For FLUX.1, standard patch parallelism can send approximately $60$ GB of activations per image per GPU and can spend more than $45\%$ of inference time on communication on $4\times$ L20 over PCIe (Luo et al., 23 Jul 2025).

The framework is motivated by an empirical observation: adjacent diffusion steps produce highly similar activations. CompactFusion therefore treats temporal redundancy, rather than scheduler overlap, as the primary object of optimization. This places it in contrast with displaced parallelism schemes that reuse stale activations to overlap communication with computation. Those methods can hide some latency, but they do not reduce transferred volume and can degrade fidelity because the communicated state is outdated (Luo et al., 23 Jul 2025).

CompactFusion is evaluated in image and video diffusion settings, including FLUX.1-dev at $1024\times 1024$ with $28$ DPM-Solver steps and CogVideoX-2B with $50$ steps, under sequence parallelism with Ring Attention and related communication patterns. The reported hardware includes $4\times$ H20 with NVLink at $366$ GB/s, $4\times$ L20 with PCIe at $17.13$ GB/s, and $45\%$ 0A40 with traffic control to emulate slower Ethernet-like links (Luo et al., 23 Jul 2025).

2. Residual Compression and error-feedback dynamics

The mathematical abstraction treats a layerwise activation trajectory as

$45\%$ 1

where $45\%$ 2 is the true activation at diffusion step $45\%$ 3. Under compression, each device instead maintains a reconstructed state $45\%$ 4. CompactFusion first computes

$45\%$ 5

then defines the transmitted object as the residual

$45\%$ 6

rather than $45\%$ 7 itself. The receiver reconstructs

$45\%$ 8

where $45\%$ 9 is the decompressed residual (Luo et al., 23 Jul 2025).

The compression primitive is formulated as a $4\times$ 0-compressor,

$4\times$ 1

The central claim is that residuals are much lower-energy than full activations, so the same compressor incurs much lower absolute error when applied to $4\times$ 2 than when applied to $4\times$ 3. The analysis assumes local stability,

$4\times$ 4

and a temporal redundancy condition,

$4\times$ 5

with $4\times$ 6 and $4\times$ 7 bounding activation energy (Luo et al., 23 Jul 2025).

A critical component is error feedback. Without it, residual-only transmission leads to cumulative drift, with total error growing like $4\times$ 8. CompactFusion instead defines each residual relative to the reconstructed previous state and feeds previous compression error back into the next residual computation. Under the stated assumptions, the steady-state error bound for residual compression is lower than the bound for naive activation compression by a factor proportional to $4\times$ 9, up to a stability-dependent multiplicative term. This formalizes the intuition that temporal redundancy can be converted into communication savings without destabilizing the denoising trajectory (Luo et al., 23 Jul 2025).

3. Compression codecs, implementation variants, and integration path

CompactFusion is a framework rather than a single codec. The reported implementation supports low-bit quantization, structured sparsity, and low-rank residual compression. The main deployed variants are summarized below.

Variant	Mechanism	Reported compression
Compact-1bit	1-bit residual quantization	$1024\times 1024$ 0
Compact-2bit	2-bit residual quantization	$1024\times 1024$ 1
Compact-Lowrank	Rank-32 INT4 low-rank residual compression on FLUX.1	$1024\times 1024$ 2

For quantization, a residual tensor $1024\times 1024$ 3 is represented as

$1024\times 1024$ 4

where $1024\times 1024$ 5 is an elementwise low-bit code and $1024\times 1024$ 6 is a rank-1 scaling matrix estimated from per-row and per-column magnitude statistics. The 1-bit version uses $1024\times 1024$ 7, while the 2-bit version uses a four-level codebook. These quantizers are implemented in fused Triton kernels (Luo et al., 23 Jul 2025).

For extreme compression ratios, Compact-Lowrank approximates $1024\times 1024$ 8 with subspace iteration and then quantizes $1024\times 1024$ 9 and $28$0 with INT4. A reported design conclusion is that, under a fixed bandwidth budget, increasing effective rank matters more than refining the low-rank factors with additional subspace-iteration steps. The residuals are empirically high-rank enough that wider low-rank coverage is preferable to a very precise low-rank approximation of too small rank (Luo et al., 23 Jul 2025).

Operationally, CompactFusion sits around communication primitives such as ring send/recv, all-gather, or all-to-all. It does not change scheduler logic, model weights, or partitioning strategy. In the reported xDiT and distrifuser integration, it requires no pipeline rework and fewer than $28$1 lines of core code change. A one-step warmup without compression is used to initialize the base state (Luo et al., 23 Jul 2025).

4. Measured latency, fidelity, and robustness

On FLUX.1-dev with $28$2L20 under sequence parallelism with Ring Attention, the reported single-GPU baseline is $28$3 s per image. Standard Ring Attention sequence parallelism reduces this to $28$4 s, while overlap-based baselines report $28$5 s for DistriFusion and $28$6 s for PipeFusion. Compact-1bit reaches $28$7 s and Compact-2bit $28$8 s, corresponding to $28$9 and $50$0 speedup over the single-GPU baseline, respectively. Compact-Lowrank is slower on this interconnect at $50$1 s because its compression overhead is harder to hide when bandwidth is relatively plentiful (Luo et al., 23 Jul 2025).

Quality results are reported against the original uncompressed model outputs. On the same FLUX.1 $50$2L20 setting, DistriFusion records PSNR $50$3, LPIPS $50$4, and FID $50$5. PipeFusion records PSNR $50$6, LPIPS $50$7, and FID $50$8. Compact-1bit records PSNR $50$9, LPIPS $4\times$ 0, and FID $4\times$ 1, while Compact-2bit records PSNR $4\times$ 2, LPIPS $4\times$ 3, and FID $4\times$ 4. Compact-Lowrank, despite its $4\times$ 5 compression, records PSNR $4\times$ 6, LPIPS $4\times$ 7, and FID $4\times$ 8, which remains better than the reported FID of DistriFusion (Luo et al., 23 Jul 2025).

On CogVideoX-2B, the same pattern is reported. With $4\times$ 9 GPUs on L20, single-GPU latency is $366$0 s, while Compact-1bit and Compact-2bit both reach approximately $366$1 s, outperforming Ring Attention, Ulysses, and DistriFusion in latency. Quality relative to original samples improves markedly over the overlap-based baseline: DistriFusion reports SSIM $366$2, PSNR $366$3, and LPIPS $366$4, whereas Compact-2bit reports SSIM $366$5, PSNR $366$6, and LPIPS $366$7 (Luo et al., 23 Jul 2025).

The ablations isolate two decisive factors. First, error feedback is necessary: for Compact-1bit, removing it degrades PSNR from $366$8 to $366$9, LPIPS from $4\times$ 0 to $4\times$ 1, and FID from $4\times$ 2 to $4\times$ 3. Second, warmup length matters but only moderately: Compact-2bit improves from FID $4\times$ 4 with one warmup step to $4\times$ 5 with two warmup steps, but the one-step configuration already achieves the main latency-quality tradeoff (Luo et al., 23 Jul 2025).

5. Relation to prior acceleration strategies

CompactFusion is best understood as a shift from communication overlap to communication reduction. Overlap-based displaced parallelism exploits temporal redundancy indirectly by reusing stale activations; CompactFusion exploits the same redundancy directly by sending only incremental information. This distinction has practical consequences. Because the communicated object remains current rather than stale, quality degradation is smaller; because the transmitted volume is lower, the method remains effective on slower links where overlap alone ceases to help (Luo et al., 23 Jul 2025).

This also clarifies a common misconception: CompactFusion is not primarily a model-compression method. It does not reduce parameter count, change the forward-pass semantics, or require retraining. Its compression target is the communicated activation stream in multi-device inference. A plausible implication is that its deployment boundary is the communication layer rather than the model-definition layer; accordingly, the paper frames it as compatible with sequence parallelism, patch parallelism, and other activation-exchange regimes (Luo et al., 23 Jul 2025).

The slow-network results are central to that positioning. On $4\times$ 6A40 under bandwidth-constrained emulation, prior sequence-parallel variants eventually become slower than single-GPU inference as communication deteriorates, whereas CompactFusion continues to deliver speedups. The reported headline is up to $4\times$ 7 speedup over a prior overlap-based method in such settings, specifically because residual transmission makes communication-heavy strategies like sequence parallelism viable on links that would otherwise be prohibitive (Luo et al., 23 Jul 2025).

6. Broader usage of the term and open questions

Outside parallel diffusion serving, the phrase “compact fusion” appears more loosely across several research areas. It describes an ultra-compact multi-modality image-fusion network with $4\times$ 8 trainable parameters and $4\times$ 9 KB model size in MMDRFuse (Deng et al., 2024), decision-level weighted-box fusion for occlusion-robust object detection on devices with less than $17.13$0 MB SRAM (Cheng et al., 1 Jun 2026), heterogeneous multi-LLM capability transfer into $17.13$1–$17.13$2B targets in FuseChat-3.0 (Yang et al., 6 Mar 2025), continuous decomposition for infrared-visible fusion in Conti-Fuse (Li et al., 2024), and, in a different domain entirely, a compact stellarator–tokamak hybrid that preserves tokamak-like volume and transport while adding external rotational transform (Henneberg et al., 2024). This suggests that “CompactFusion” is polysemous across contemporary literature, whereas the proper-noun system name refers specifically to residual compression for diffusion inference (Luo et al., 23 Jul 2025).

Several limitations remain explicit. Compression overhead is not always fully hidden on very fast interconnects; low-rank compression is especially disadvantaged when bandwidth is ample. The theoretical analysis depends on local stability and error-independence assumptions that are empirically motivated rather than exact. The method also requires at least one uncompressed warmup step. More broadly, its advantage relies on strong temporal redundancy in the denoising trajectory; a plausible implication is that architectures or schedulers with much larger inter-step activation change would narrow the margin between residual and full-activation compression (Luo et al., 23 Jul 2025).

Within its stated scope, CompactFusion defines a systems template for communication-aware diffusion serving: preserve the existing inference pipeline, compress only the information that changes from one step to the next, and stabilize the resulting recurrent reconstruction with lightweight error feedback. Its main significance lies in showing that, for diffusion models, temporal redundancy can be exploited not merely to hide communication, but to reduce it substantially while maintaining high fidelity (Luo et al., 23 Jul 2025).