Hierarchical Compression (STC)

Updated 19 April 2026

Hierarchical Compression (STC) is a method that decomposes data into multi-layered representations for efficient, scalable encoding across diverse domains.
It employs multi-layer sparse ternary codes and transform hierarchies to progressively refine residuals and optimize rate–distortion trade-offs.
Its adaptable design enables progressive decoding and parallel processing, significantly reducing complexity in image, video, 3D, and tokenized data compression.

Hierarchical compression—often designated as STC (Sparse Ternary Coding, Scale-wise Transform Coding, Spatial–Temporal Conditional, or Streaming Token Compression, depending on the modality)—refers to a family of schemes that decompose a data source into a succession of representations at progressively finer scales or higher levels of detail. In contrast to flat or single-rate models, hierarchical schemes allocate bits or tokens in multiple layers or stages, encoding coarse features globally and refinements only where necessary. This enables superior rate–distortion trade-offs, scalable and progressive decoding, and significant reductions in complexity or context length across domains. Hierarchical compression, as realized in modern neural and traditional models, underpins state-of-the-art systems for image, video, 3D geometry, code, and tokenized sequence domains (Ferdowsi et al., 2017, Lu et al., 2024, Lu et al., 2023, Brand et al., 2023, Ameen et al., 30 Apr 2025, Xu et al., 28 May 2025, Ostby, 11 Jan 2026, Wang et al., 30 Nov 2025).

1. Formal Foundations: Multilayer Sparse Ternary Codes and Transform Hierarchies

The foundational formalism for hierarchical compression is the multi-layer extension of Sparse Ternary Codes (ML-STC) for universal vector quantization (Ferdowsi et al., 2017). In ML-STC, a signal $f\in\mathbb{R}^n$ is decomposed across $L$ layers, each operating on the residual of the previous approximation:

At each layer $l$ , the input residual $r^{(l-1)}$ is projected via an analysis transform $W^{(l)}$ , ternarized by thresholding ( $\phi_{\tau^{(l)}}$ ), reweighted ( $\beta^{(l)}$ ), and entropy-coded.
The decoded code $a^{(l)}$ is synthesized using $B^{(l)}$ (typically $W^{(l)T}$ ), producing a reconstruction and new residual.
The process recurses: $L$ 0.
The total rate is $L$ 1, and distortion composes accordingly.

This recursive, residual-refining structure generalizes to learned autoencoder hierarchies (neural image/video coding (Brand et al., 2023, Lu et al., 2023, Lu et al., 2024)), 3D Gaussian Splatting compression (Xu et al., 28 May 2025), and structured token-based representation (code and video LLMs (Ostby, 11 Jan 2026, Wang et al., 30 Nov 2025)).

Hierarchical compression enables fine-grained control of the rate–distortion trade-off by assigning bits or tokens according to local signal complexity, exploiting spatial/temporal/statistical heterogeneity:

Rate–Distortion Fidelity: ML-STC and variants approach the Shannon lower bound at all bitrates by distributing quantization effort via many low-rate layers, thereby supporting water-filling optimality (Ferdowsi et al., 2017).
Progressive Decoding: Since each layer/scale contributes a refinement, partial decoding produces valid, albeit coarser, reconstructions; later layers refine only what remains uncertain—enabling scalability and graceful degradation under loss or bandwidth constraints (Lu et al., 2023, Lu et al., 2024).
Parallel and Local Computation: Layers can be computed in pipelined or parallel fashion, as dependencies flow primarily from coarser to finer scales, not across all positions or time steps (Lu et al., 2024).

3. Methodological Variants Across Modalities

Vector and Feature Compression (STC/ML-STC)

For high-dimensional vectors, ML-STC applies an eigenbasis projection at each layer, sparse ternarization with threshold $L$ 2, optimal per-coordinate reweighting, and entropy coding. Decoding aggregates per-layer reconstructions:

$L$ 3

Performance matches or surpasses binary hashing and locality-sensitive hashing, particularly at moderate-to-high rates (Ferdowsi et al., 2017):

Dataset	ML-STC PSNR Gain (vs. ITQ/LSH)	Rate Range
MNIST	2–3 dB	1–2 bpp
GIST-1M	2–3 dB	1–2 bpp

Learned Image Compression

Contemporary image codecs build multi-scale autoencoder hierarchies, where each latent space operates at different downsampling factors (Brand et al., 2023, Ameen et al., 30 Apr 2025):

Coarse scales (lower resolution, fewer channels) capture global structure at low bitrate.
Finer scales encode localized details only where needed (gated by spatial masks), enabling block-adaptive partitioning.
Bitrate per layer is controlled by entropy models conditioned hierarchically (hyperpriors).

LoC-LIC demonstrates a fivefold complexity reduction (1256→270 kMAC/Pixel) versus flat-architecture baselines at no significant performance loss (Ameen et al., 30 Apr 2025).

Video Compression: Multiscale Hierarchical VAEs

Deep Hierarchical Video Compression (DHVC) and its enhancements (Lu et al., 2023, Lu et al., 2024) compress each frame as a layered hierarchy:

Each scale’s latent is predicted using coarser spatial features and temporal cues from prior frames.
The resulting bitstreams per scale facilitate progressive decoding and robust streaming.

Method (PSNR anchor: HM-16.26)	BD-Rate Saving	Parameters	MACs/pixel	FPS (1080p)
DHVC 2.0	20–30%	92M	350k	20–33
DCVC-DC	—	50M	1400k	<2
VCT	—	750M	3000k	1.6

Hierarchical Token Compression (LLMs)

Hierarchical compression schemes for code bases (TREEFRAG/STC) (Ostby, 11 Jan 2026) and streaming VideoLLMs (Wang et al., 30 Nov 2025) exploit program structure and temporal redundancy, respectively:

TREEFRAG: Encodes a codebase as a rooted hierarchical tree, pruning fields at defined Levels of Detail (LOD), achieving ≥18:1 compression and preserving architectural context for LLMs.
Streaming Token Compression: Combines ViT feature caching across frames (STC-Cacher) and salience-driven pruning of visual tokens before LLM ingestion (STC-Pruner), reducing latency by up to 45% while retaining 99% of baseline accuracy.

4. Entropy Coding, Context Models, and Complexity Benefits

Hierarchical designs leverage conditional entropy models where each finer scale is conditioned on coarser decoded representations. For example:

In LoC-LIC, only small-channel high-res maps are processed at early stages, while channel depth is increased as resolution drops, optimizing computational loads (Ameen et al., 30 Apr 2025).
DHVC uses lightweight spatial–temporal context networks per scale, drastically reducing inference-time memory and operations (Lu et al., 2023, Lu et al., 2024).
In real-time VideoLLMs, STC-Cacher/Pruner operate without retraining and insert plug-and-play compression at both vision and context levels, directly translating to throughput gains (Wang et al., 30 Nov 2025).

5. Empirical Performance and Comparative Advantages

Hierarchical compression consistently outperforms flat methods on standard rate–distortion, complexity, and task-specific metrics:

ML-STC yields up to 3 dB PSNR improvement at moderate rates vs. binary hashing (Ferdowsi et al., 2017).
LoC-LIC realizes 5–10% BD-rate savings at ∼¼ complexity vs flat learned codecs (Ameen et al., 30 Apr 2025).
DHVC surpasses HEVC/VVC efficiency, achieves >10× throughput, and provides robustness to packet loss via progressive, scalable decoding (Lu et al., 2023, Lu et al., 2024).
TREEFRAG achieves 18:1–24:1 token compression with 94–97% LLM issue-scoping success, mitigating “lost-in-the-middle” effects (Ostby, 11 Jan 2026).
STC for streaming VideoLLMs realizes up to 45% LLM prefill latency reduction with 99% accuracy retention (Wang et al., 30 Nov 2025).

Method/Domain	Compression Benefit	Additional Advantages
ML-STC (vectors)	~2–3 dB PSNR gain	No codebook, fast similarity search
DHVC (video)	20–30% bit savings	Parallelism, packet loss resilience
LoC-LIC (image)	~5× complexity cut	Maintains state-of-the-art fidelity
TREEFRAG (code LLMs)	18–24× token reduction	Hierarchy-aware, mitigates token loss
STC (VideoLLMs)	24–45% latency reduction	Plug-and-play, preserves accuracy

6. Interpretability, Scalability, and Open Directions

Hierarchical compression architectures support interpretability and analysis of encoded representations:

ML-STC and SHTC maintain explicit residuals and transform bases, enabling theoretical analysis of rate allocation (Ferdowsi et al., 2017, Xu et al., 28 May 2025).
Sparse–coding layers (e.g., ISTA unfolding in SHTC) enable direct control of sparsity priors and step-sizes, beneficial for specific signal classes and minimizing parameter counts (Xu et al., 28 May 2025).
The ability to mask or gate high-resolution/fine-detail latents (as in (Brand et al., 2023)) facilitates spatially adaptive compression and efficient handling of heterogeneous or multi-domain inputs.

A plausible implication is that as task complexity and data heterogeneity rise, hierarchical compression frameworks will become increasingly central for efficient neural representation, distributed inference, and scalable downstream modeling.

7. Comparative Summary and Schematic Table

Hierarchical Method	Domain	Key Mechanism	Empirical Gains
ML-STC (Ferdowsi et al., 2017)	Vector	Layered sparse ternary codes	2–3 dB PSNR (1–2 bpp)
DHVC (Lu et al., 2024)	Video	Multiscale VAE w/ predictive coding	>20% BD-rate savings
LoC-LIC (Ameen et al., 30 Apr 2025)	Image	Hierarchical feature transforms	5× MACs cut
TREEFRAG (Ostby, 11 Jan 2026)	Code/LLM	Hierarchical AST pruning	≥18:1 token reduction
STC (Wang et al., 30 Nov 2025)	VideoLLM	Token caching & pruning	24–45% speedup

In sum, hierarchical compression (STC and related architectures) provides a general, principled, and practically validated framework for scalable, high-fidelity, and efficient representation across a spectrum of modalities. Its adoption in cutting-edge neural image/video codecs, code LLM compression, and 3D geometry transmission underscores its centrality in modern information representation and inference.