cuSZ-Hi: Open-Source GPU Scientific Compressor

• The paper introduces cuSZ-Hi, a fully open-source GPU-based lossy compressor that achieves up to 249% compression ratio improvement under fixed error bounds.
• It synergizes an advanced interpolation-based lossy decomposition with tailored open-source lossless pipelines, providing selectable modes for high compression or throughput.
• Experimental evaluations on diverse scientific datasets demonstrate superior rate-distortion performance and significantly reduce I/O bottlenecks in GPU-centric workflows.

Searching arXiv for the specified papers to ground the article in current preprint metadata. cuSZ-Hi is an open-source, GPU-based, scientific error-bounded lossy compressor designed to achieve high compression ratio while remaining low-latency and practical on modern GPU systems. It targets GPU-centered scientific workflows in domains such as cosmology, turbulence, molecular dynamics, seismic imaging, and quantum simulation, where raw data are produced at extremely high throughput and where I/O and storage become bottlenecks. Its defining feature is an optimized synergistic orchestration between interpolation-based lossy decomposition and open-source lossless encoding of quantized errors; under the reported evaluation, it achieves up to 249% compression-ratio improvement under the same error bound and up to 215% compression-ratio improvement under the same decompression PSNR relative to existing state-of-the-art scientific lossy compressors (Wu et al., 15 Jul 2025).

1. Problem setting and lineage

cuSZ-Hi addresses the standard error-bounded lossy compression problem for scientific arrays:

$$\max \ \frac{|X|}{|Z|} \quad \text{s.t.} \quad Z=\operatorname{Cmp}(X),\ X'=\operatorname{Dec}(Z),\ \left\|X-X'\right\|_{\infty}=\max |x_i-x_i'|\le \epsilon,$$

where $X$ is the input scientific data, $Z$ is the compressed output, $X'$ is the decompressed result, and $\epsilon$ is the user-specified absolute error bound. In the GPU setting, the optimization is specialized to a joint search over lossy decomposition and lossless encoding,

$$\operatorname{Dcp},\operatorname{Enc}=\arg\max_{\operatorname{Dcp},\operatorname{Enc}}\frac{|X|}{|\operatorname{Enc}(\operatorname{Dcp}(X))|}$$

subject to the same error bound and the throughput constraint

$$\operatorname{Speed}(\operatorname{Dcp},\operatorname{Enc}) \ge S_t.$$

The central objective is therefore not merely ratio maximization, but ratio maximization under bounded reconstruction error and without sacrificing GPU throughput too much (Wu et al., 15 Jul 2025).

The immediate antecedent is cuSZ-I, which introduced a GPU-customized interpolation-based predictor and layered lossless post-processing to improve the compression ratio and decompression data quality of GPU scientific lossy compression. cuSZ-I established that interpolation-based prediction could move GPU compressors closer to CPU interpolation compressors in rate-distortion behavior, but its strongest ratio enhancement relied on NVIDIA Bitcomp in lossless mode, a proprietary component. cuSZ-Hi was developed to fill the resulting gap: a fully open-source, high-ratio GPU compressor with carefully optimized lossy prediction and lossless encoding (Liu et al., 2023).

2. Framework architecture and selectable operation

cuSZ-Hi follows the standard scientific lossy compression pattern. The first stage performs lossy decomposition or prediction, converting floating-point data into quantized integer prediction errors. The second stage performs lossless encoding to further compress those integer errors through entropy reduction and redundancy removal. The framework is described as flexible, domain-irrelevant, and fully open-source, and its main claim is that the two stages are improved synergistically rather than independently (Wu et al., 15 Jul 2025).

Within the comparative GPU landscape summarized in the work, cuSZp2 uses direct value quantization plus simple encoding; cuSZ uses a Lorenzo predictor or interpolation predictor plus Huffman or Bitcomp; cuSZ-I uses an interpolation predictor plus Huffman plus NVIDIA Bitcomp; and cuSZ-Hi combines an improved interpolation predictor with open-source tailored lossless pipelines. This positioning is significant because earlier GPU compressors often prioritized throughput over compression quality, whereas cuSZ-Hi explicitly targets the ratio-speed trade-off.

A central architectural feature is the existence of two selectable modes. cuSZ-Hi-CR is compression-ratio preferred, whereas cuSZ-Hi-TP is throughput preferred. The framework is therefore not a single fixed pipeline but a selectable orchestration between a stronger-compression path and a faster path. A common misconception is to read cuSZ-Hi as a throughput-first design; the reported architecture instead indicates that it is a high-ratio design with an explicit fast variant for cases where encoding overhead must be reduced.

3. Interpolation-based lossy decomposition on GPUs

The lossy side retains the interpolation-based idea introduced in cuSZ-I but reorganizes it substantially. For 3D data, cuSZ-I used a chunk size of $33 \times 9 \times 9$ with anchor stride 8 in the GPU working region. cuSZ-Hi changes the data block size to $17^3$, the anchor stride to 16, and the interpolation hierarchy to 4 levels instead of 3. The stated reasons are more isotropic partitions, improved shared-memory and cache utilization, and substantially lower anchor overhead; the paper notes that anchor storage in cuSZ-I is sizable and that cuSZ-Hi reduces anchor storage to about $1/8$ of cuSZ-I while preserving prediction quality (Wu et al., 15 Jul 2025).

The interpolation procedure itself is redesigned as a parallelized multi-dimensional interpolation scheme. Rather than a purely sequential 1D interpolation order, interpolation progresses with increasing dimensionality,

$X$0

and interpolation across the same “color group” is fully parallelized on GPU threads. Prediction is isotropic across dimensions, avoiding dimension-order sensitivity. Final prediction uses the average of eligible interpolation predictions, and when multiple predictions are available only the highest-spline-order results are used. The reported example is explicit: if two cubic predictions are available, they are averaged; if one is cubic and the other is linear or quadratic, the cubic one is kept. This scheme is intended to exploit spatial correlation more effectively than the original sequential 1D spline interpolation scheme.

cuSZ-Hi also extends auto-tuning. It samples $X$1 of the data using blocks that match a full thread-block workload, tests multiple interpolation configurations on the sampled blocks, and selects the best configuration for each level based on minimum prediction error. The workload mapping is level-specific: Level 4 and Level 3 test all configurations within one thread block, Level 2 splits tests across 2 thread blocks, and Level 1 uses 6 thread blocks because of heavier computation. This design is explicitly intended to improve adaptability to diverse datasets while keeping tuning overhead lightweight.

4. Quantization-code reordering and lossless orchestration

After prediction and quantization, cuSZ-Hi treats the resulting integer stream as an object for further structural optimization. In interpolation compressors, quantization codes depend strongly on interpolation stride: larger strides typically lead to larger prediction errors, and the codes produced at different interpolation levels have different distributions. cuSZ-I flattened these codes in original order, mixing values from different interpolation levels. cuSZ-Hi instead reorders the quantization codes so that codes from the same interpolation stride are grouped together. The paper attributes better compressibility to this reordering because similar codes become adjacent, the resulting sequence becomes smoother, and outliers are concentrated near the beginning of the sequence. The paper also gives a 3D mapping equation for the reordered 1D index $X$2 with prefix terms by interpolation level, underscoring that the reordering is algorithmically defined rather than heuristic (Wu et al., 15 Jul 2025).

The lossless side is motivated by the observation that scientific lossy compressors often leave substantial redundancy in the integer output. The work distinguishes two kinds of redundancy: token entropy, arising from frequency imbalance of integer symbols, and bit-level redundancy, arising from repeated patterns in binary representations. It further reports that compressed outputs from other compressors, when benchmarked through NVIDIA Bitcomp, are often still compressible to only about $X$3–$X$4 of original size, indicating substantial remaining redundancy. Because Bitcomp is proprietary, cuSZ-Hi substitutes open-source pipelines tailored to the reordered code stream.

The two selectable lossless modes are as follows:

Mode	Pipeline	Stated preference
cuSZ-Hi-CR	Huffman $X$5 RRE4 $X$6 TCMS8 $X$7 RZE1	Compression-ratio preferred
cuSZ-Hi-TP	TCMS1 $X$8 BIT1 $X$9 RRE1	Throughput preferred

In cuSZ-Hi-CR, Huffman performs entropy coding based on symbol frequencies; RRE4 recursively removes repeated symbols using a bitmap marking identical predecessor symbols; TCMS8 transforms 64-bit two’s-complement to magnitude-sign form using

$Z$0

and RZE1 recursively removes zero symbols using bitmap-based marking. In cuSZ-Hi-TP, Huffman is removed to reduce overhead; TCMS1 operates on 1-byte symbols, BIT1 performs bit shuffle, and RRE1 performs one-byte repeated-symbol elimination. An additional implementation detail is that quantization codes are stored using a minimum one-byte width, uint8_t, while out-of-range outliers are stored separately. The stated rationale is that interpolation-based quantizers usually produce concentrated code distributions, with only a few large outliers.

5. Experimental characterization

The evaluation uses two GPU systems: NERSC Perlmutter with an NVIDIA A100 (80GB SXM) and a workstation with an NVIDIA RTX 6000 Ada (48GB). The reported memory bandwidths are 2039 GB/s base-1000 and 1900 GiB/s for the A100, and 960 GB/s base-1000 and 894 GiB/s for the RTX 6000 Ada. Compared baselines include cuSZ-L, cuSZ-I, cuSZ-IB, cuSZp2, cuZFP, and FZGPU; slower compressors such as cuSZx and MGARD-GPU are excluded from the main evaluation. The datasets span JHTDB turbulence data, Miranda hydrodynamics, Nyx cosmological hydrodynamics, QMCPack quantum simulation, RTM seismic imaging, and one additional scientific dataset of 37 files totaling 6.5 GiB; the corpus includes both 2D and 3D scientific fields, with totals ranging from about 612 MiB to 6.5 GiB (Wu et al., 15 Jul 2025).

The paper uses compression ratio, bit rate, and PSNR for evaluation. The bit rate is defined as

$Z$1

that is, average bits per element. PSNR is used for rate-distortion analysis, with same-PSNR comparisons supplementing same-error-bound comparisons. Reported compression-ratio results state that cuSZ-Hi achieves the best compression ratio in almost all test cases, that under large error bounds the compression ratio can exceed 300, that the improvement is greater than 200% in 6 of 18 cases and greater than 50% in 11 of 18 cases, and that when compared only with non-proprietary solutions the improvement ranges from 113% to 2786%. The strongest competitor in raw ratio is cuSZ-IB, but it depends on proprietary Bitcomp; cuSZ-Hi is reported to match or exceed it while remaining open-source.

The rate-distortion results similarly favor the compression-ratio-oriented mode. The reported conclusion is that cuSZ-Hi-CR gives the best rate-distortion among high-ratio compressors in most cases. On Miranda, at around PSNR 59 and 63, cuSZ-Hi-CR achieves about 140% and 120% compression-ratio improvement over cuSZ-IB. On QMCPack, at around PSNR 60 and 76, cuSZ-Hi-CR achieves about 215% and 80% compression-ratio improvement over cuSZ-IB. cuSZ-Hi-TP is described as remaining close in quality to cuSZ-Hi-CR while being faster.

Throughput results place cuSZ-Hi behind throughput-first compressors such as cuZFP, cuSZp2, and FZGPU, which is consistent with its design goal. Among compressors with strong ratio and quality, however, it is reported as competitive: cuSZ-Hi-CR has comparable or slightly lower throughput than cuSZ-I(B), with much better ratio, and the overhead is about $Z$2 on RTX 6000 Ada and $Z$3 on A100 in many cases. cuSZ-Hi-TP is often faster than cuSZ-I(B) and cuSZ-L; on RTX 6000 Ada it shows about 20%–40% speed improvement over cuSZ-L and can be up to $Z$4 faster than cuSZ-I(B).

The ablation study attributes gains to multiple cumulative components rather than any single dominant change. Revising data partition and anchor stride gives 6%–60% compression improvement over cuSZ-IB; quantization-code reordering adds another 6%–55%; auto-tuned multi-dimensional interpolation improves compression ratio by 8%–19%; and the optimized lossless pipeline contributes an additional 16%–50%. This directly supports the claim that both lossy prediction and lossless encoding are essential to the final ratio.

6. Open-source significance, comparative position, and limitations

A major contribution of cuSZ-Hi is its fully open-source design. This matters because cuSZ-I’s strongest ratio depended on NVIDIA Bitcomp, and proprietary dependencies limit portability, reproducibility, and customization. cuSZ-Hi replaces that dependency with open-source pipelines based on Huffman, RRE, TCMS, and bit shuffling, and is explicitly positioned as a more deployable and extensible option for scientific workflows (Wu et al., 15 Jul 2025).

In the GPU scientific compression landscape, cuSZ-Hi occupies an intermediate position between high-throughput but lower-ratio GPU compressors such as cuSZp2, cuZFP, and FZGPU, and high-ratio but proprietary or CPU-based approaches. The paper characterizes it as the first GPU-based high-ratio, high-quality, open-source scientific error-bounded lossy compressor in this line of work. That characterization is best understood relative to cuSZ-I, which had already demonstrated the value of GPU-optimized interpolation but still relied on proprietary Bitcomp for its strongest ratio gains (Liu et al., 2023).

The principal strengths reported for cuSZ-Hi are strong compression ratio, good rate-distortion behavior, an open-source and flexible design, two practical modes for different goals, a GPU-aware prediction and encoding design, and an ablation-backed design rationale. The principal limitations are equally explicit: it is still slower than aggressively throughput-oriented compressors such as cuZFP, cuSZp2, and FZGPU; future work is needed to reduce interpolation overhead further; better lossless encoders may still be possible; and the paper suggests future auto-selection across compressor archetypes and pipelines. A common misconception is therefore that cuSZ-Hi replaces throughput-first compressors outright. The reported evidence instead places it as a high-ratio, open-source GPU compressor that closes much of the gap to proprietary high-ratio GPU compression while preserving competitive throughput for practical scientific GPU workflows.