Efficient Learned Data Compression via Dual-Stream Feature Decoupling

Abstract: While Learned Data Compression (LDC) has achieved superior compression ratios, balancing precise probability modeling with system efficiency remains challenging. Crucially, uniform single-stream architectures struggle to simultaneously capture micro-syntactic and macro-semantic features, necessitating deep serial stacking that exacerbates latency. Compounding this, heterogeneous systems are constrained by device speed mismatches, where throughput is capped by Amdahl's Law due to serial processing. To this end, we propose a Dual-Stream Multi-Scale Decoupler that disentangles local and global contexts to replace deep serial processing with shallow parallel streams, and incorporate a Hierarchical Gated Refiner for adaptive feature refinement and precise probability modeling. Furthermore, we design a Concurrent Stream-Parallel Pipeline, which overcomes systemic bottlenecks to achieve full-pipeline parallelism. Extensive experiments demonstrate that our method achieves state-of-the-art performance in both compression ratio and throughput, while maintaining the lowest latency and memory usage. The code is available at https://github.com/huidong-ma/FADE.

• The paper introduces FADE, which replaces deep serial prediction with dual-stream decoupling and hierarchical refinement to boost compression efficiency across diverse data types.
• It employs a parallel, concurrent pipeline design that overcomes GPU/CPU bottlenecks, significantly reducing latency and increasing throughput.
• Extensive benchmarks show FADE outperforms leading compressors by achieving higher compression ratios and system throughput compared to models like EDPC and MSDZip.

Efficient Learned Data Compression via Dual-Stream Feature Decoupling

Motivation and Problem Analysis

The exponential growth in heterogeneous data from domains such as text, multimedia, genomics, and scientific datasets intensifies the need for high-performance, scalable lossless data compression (LDC). Classical compressors (e.g., LZ77, Huffman, ANS, Gzip, zstd) do not effectively leverage complex statistical redundancies in high-dimensional modern data, especially long-range semantic dependencies. While modern LDC based on deep learning achieves significant gains in compression ratio (CR), these improvements are often hamstrung by inefficiency: deep serial single-stream models necessitate long computational paths, exacerbating both inference latency and GPU memory usage. Heterogeneous system deployment is further constrained by mismatched GPU (probability modeling) and CPU (entropy coding) speeds, causing throughput bottlenecks dictated by Amdahl's Law on serial sections.

Architectural Innovations: Dual-Stream Multi-Scale Decoupling and Hierarchical Refinement

The paper introduces FADE (efficient le\underline{a}rned \underline{d}ata compr\underline{e}ssion), with the following architectural advancements: Dual-Stream Multi-Scale Decoupler (DMD), Hierarchical Gated Refiner (HGR), and a Concurrent Stream-Parallel Pipeline (CSPP).

The DMD replaces the prevalent deep serial paradigm by explicit feature decoupling: a parallel, shallow architecture simultaneously processes macro-semantic (long-range, global) and micro-syntactic (local, N-gram-like) dependencies. The macro stream leverages a GeGLU-based rolling cache for global context, while the local branch employs a 1D CNN for strong local inductive bias. Fused by a content-adaptive router, the design eliminates the multi-scale interference and saliency dilution that impinge on single-stream MLPs, significantly reducing the required model depth.

Figure 1: Overview of the FADE pipeline, highlighting parallel disentanglement of input into local and global features, which are dynamically refined for final compression.

Information-theoretic and empirical analyses (e.g., mutual information decay profiles, self-similarity matrices, and saliency heatmaps) substantiate the sharp dichotomy in data dependencies. DMD precisely addresses these (Figure 2).

Figure 2: Visual evidence of dual dependency: fast micro-syntactic decay and persistent macro-semantic dependencies, with local/global streams specializing as intended.

Downstream of DMD, the HGR executes sample-adaptive, hierarchical refinement: (1) coarse-grained channel mixing with persistent memory via block matrix multiplication enables deep context/instance adaptation, and (2) fine-grained nonlinear refinement via GeGLU suppresses noise amplification. Content-aware gating ensures that channel interactions are selectively enhanced or dampened per-instance—essential for optimal probability assignment in highly nonstationary or bursty input data.

System-Level Optimization: Parallelization and Pipeline Design

FADE introduces the CSPP, a hybrid pipeline synergizing thread-safe temporal parallelism (ping-pong double-buffered producer-consumer queues) and data parallelism. This dual strategy (1) hides device heterogeneity and transfer latency, (2) addresses the irreducible sequential dependency in decompression by partitioning the input into $N$ independent sub-streams, and (3) enables decompression throughput to match compression throughput (Figure 3). This overcomes the typical asymmetry where strict autoregressive causality limits decompression acceleration.

Figure 3: Comparison of model execution strategies, with CSPP enabling parallel entanglement of probability estimation and coding, overcoming classic serial bottlenecks.

Empirical Results

Compression Ratio and Throughput

Extensive benchmarks across 7 domains (text, audio, images, video, floats, genomics, and heterogeneous) demonstrate that FADE achieves the highest average CR (3.716), outperforming recent strong baselines such as EDPC (3.667), MSDZip (3.573), and SEP (3.601). Notably, FADE yields improved CR on Enwik9 (6.288) and Silesia (5.400) datasets, domains representative of both semantic and format heterogeneity.

Figure 4: Pareto frontier of CR vs. throughput. FADE exhibits consistent superior trade-off.

Throughput assessment on Silesia confirms FADE attains the highest total throughput (4347 KB/min), with decompression TP only 10% below compression TP. Competing LDCs suffer from pronounced decompression stalls due to autoregressive sequential processing, never exceeding 2856 KB/min.

Ablation and Scalability Studies

Progressive ablations indicate each FADE innovation is critical: DMD alone raises CR to 4.086, HGR boosts adaptation and pushes CR to 5.400, while only CSPP maintains system-level throughput and low latency. Component loss trends, NLL analysis, and router activation dynamics empirically validate the correct specialization of streams and effective instance adaptation (Figures 6–8 in the Appendix).

Grid search over cache and FNR dimensions finds an optimal model scaling that balances CR and system throughput.

Theoretical and Practical Implications

FADE makes several bold claims supported by strong empirical evidence:

• Replacing deep serial prediction with shallow, parallel, dual-stream decoupling is both theoretically sound and practically crucial for maximizing compression efficiency.
• Full-pipeline parallelization, properly handled with concurrency and adaptive batching, eliminates the PR bottleneck of autoregressive modeling, yielding symmetric, acceleration in compression and decompression.
• Sample-adaptive channel mixing is necessary for deployment on non-stationary, multi-source data: static parameterization leads to significant performance degradation.

Practical implications are significant—FADE's architectural choices result in lower GPU memory usage and latency than state-of-the-art systems, even as FLOPs and parameters increase modestly. This demonstrates that model parallelism and architectural adaptation are more impactful than mere computational heaviness for system-level efficiency.

Limitations and Future Directions

Although FADE advances both modeling fidelity and system efficiency, the CR/TP improvements are obtained at the price of some increase in parameter count and FLOPs, compared to most LDCs except RNN-based DZip. However, this does not translate into increased inference-time latency or hardware cost. Remaining questions pertain to (1) optimal assignment of sub-streams for maximally non-stationary or adversarial data, (2) portability to non-autoregressive frameworks, and (3) extension to lossy codecs and multimodal joint compression.

Conclusion

FADE establishes a new state-of-the-art in learned universal data compression by unifying parallel feature decoupling, adaptive refinement, and advanced pipeline concurrency. Its methodological advances enable simultaneous maximization of compression ratio, throughput, and hardware efficiency on multi-source, real-world data. This positions FADE as a compelling solution for practical, scalable deployment in bandwidth- and latency-constrained environments.