SZ3: Modular Framework for Scientific Compression

Updated 10 July 2026

SZ3 is a modular framework for composing prediction-based, error-bounded lossy compressors, defined by five reusable stages.
It decomposes compression into preprocessing, prediction, quantization, encoding, and lossless coding to tailor performance to dataset structures.
Empirical results show significant improvements in compression ratios, throughput, and energy trade-offs across diverse scientific applications.

SZ3 is a modular framework for composing prediction-based, error-bounded lossy compressors for scientific data. Rather than fixing a single codec, it decomposes compression into reusable stages—preprocessing, prediction, quantization, entropy coding, and a lossless backend—so that the pipeline can be specialized to dataset structure, error model, and performance target. In later literature, SZ3 is repeatedly treated as a state-of-the-art scientific compressor, both as a baseline for new compressors and as an off-the-shelf base codec to which additional guarantees, such as spectral, topological, or streaming access, are attached (Liang et al., 2021).

1. Modular framework and architectural scope

The defining contribution of SZ3 is its five-module abstraction. A compressor instance is assembled from a Preprocessor, Predictor, Quantizer, Encoder, and Lossless compressor. The framework uses template-based composition, datatype abstraction, and a multidimensional iterator so that module implementations can be reused across datatypes and dimensions without duplicating compressor logic (Liang et al., 2021).

Stage	Function	Examples described in the SZ3 paper
Preprocessor	transforms data or configuration before prediction	logarithmic transform, transposition
Predictor	estimates each value from context	Lorenzo, regression, pattern-based, interpolation
Quantizer	enforces lossy error control on residuals	linear-scaling quantizer, Unpred-aware Quantizer
Encoder	entropy-codes quantization symbols	Huffman, fixed Huffman
Lossless backend	compresses encoded bytestream	zstd

This abstraction is broad enough that the SZ3 paper explicitly presents FPZIP, SZ1.4, SZ2, SZ-Pastri, and cpSZ as expressible within the same architectural pattern. SZ3 is therefore not only a compressor but a construction framework for a family of prediction-based compressors (Liang et al., 2021).

The framework was motivated by the observation that no single predictive codec is uniformly optimal across scientific datasets. Some datasets favor low-error local predictors; others favor blockwise trend models or specialized preprocessing. SZ3 formalizes this as a composition problem rather than a monolithic implementation problem (Liang et al., 2021).

2. Prediction, quantization, and error control

SZ3 belongs to the class of prediction-based error-bounded compressors. Each value is predicted from previously available context, the prediction residual is quantized under a user error bound, and the quantized symbols are then entropy-coded and losslessly compressed. The standard guarantee is a pointwise absolute error bound, and the paper also describes support for pointwise relative error through preprocessing such as a logarithmic transform (Liang et al., 2021).

The standard linear-scaling quantizer in the SZ family uses equal-sized consecutive bins of width $2\epsilon$ , where $\epsilon$ is the user error bound. This is the mechanism by which the decompressed value remains within the prescribed bound. Values whose residuals fall outside the representable quantization range are treated as unpredictable values and stored separately. The paper defines the bit rate as

$\text{bit rate}=\frac{\text{bits}}{\text{cr}},$

where bits is the original precision and cr is the compression ratio (Liang et al., 2021).

SZ3’s predictor layer is intentionally plural rather than singular. The original paper describes Lorenzo, regression-based, pattern-based, and interpolation-based predictors, and emphasizes that the framework can automatically select the best-fit predictor for each data block using a designed error-estimation criterion. Later papers correspondingly describe SZ3 through different predictor configurations: as a modular scientific compressor using Lorenzo and regression predictors, or as an interpolation/spline-based predictor family. This variation is consistent with SZ3’s design, because the framework supports multiple predictor instantiations rather than one immutable predictive rule (Liang et al., 2021).

3. Canonical composed pipelines and early empirical results

The SZ3 paper evaluates several concrete pipelines assembled from the framework. SZ3-LR reproduces the familiar Lorenzo-plus-regression style. SZ3-Interp uses interpolation-based predictors. SZ3-Truncation is a speed-oriented design that bypasses most of the predictive machinery. SZ3-Pastri specializes to GAMESS electronic-repulsion-integral data, and SZ3-APS specializes to Advanced Photon Source data (Liang et al., 2021).

On GAMESS at absolute error bound $10^{-10}$ , SZ3-Pastri reports compression ratios of 10.76 on $ff|ff$ , 10.06 on $ff|dd$ , and 10.71 on $dd|dd$ , exceeding both SZ-Pastri and SZ-Pastri-with-zstd under the same setting. On APS, when the absolute error bound is below $0.5$, SZ3-APS improves compression ratio by 18% on chip pillar and 12% on flat chip over the second-best method. Across broader scientific datasets, SZ3-Interp outperforms SZ3-LR on most datasets; on Miranda at PSNR 90, it reaches compression ratio 47 versus 30 for SZ3-LR. At the opposite end of the trade-off, SZ3-Truncation reaches about 1 GB/s compression throughput, roughly 4× the second-best compressor, while SZ3-Interp remains above 100 MB/s on all tested cases (Liang et al., 2021).

These results establish the original encyclopedic identity of SZ3: it is a framework in which rate–distortion behavior can be shifted by module choice, rather than a single point on the compression-speed-quality surface (Liang et al., 2021).

4. Compression ratio, energy, and access-pattern trade-offs

Later systems work places SZ3 near the high-compression-ratio / higher-cost end of the scientific-compression trade-off space. An energy study using SZ3 v3.1.8 reports, for example, that on NYX the compressor ratio reaches 102105.50 at $\epsilon=1E{-1}$ , 545.13 at $\epsilon=1E{-3}$ , and 13.72 at $\epsilon$ 0; on S3D the corresponding values are 4055.78, 308.79, and 51.11. The same study also reports that when tightening the error bound from $\epsilon$ 1 to $\epsilon$ 2, serial energy consumption can increase by as much as $\epsilon$ 3 for SZ3. The paper characterizes SZ3 and QoZ as the compressors that often achieve higher compression ratios at increased energy cost, while also noting that SZ3 scales well in multithreaded environments (Wilkins et al., 2024).

This operational profile has motivated streaming alternatives. STZ is a hierarchical streaming framework that partially reuses SZ3—specifically, it applies SZ3 to the coarsest level and then uses its own hierarchical prediction on finer levels—but it restructures compression to support both progressive decompression and random-access decompression. STZ reports similar rate-distortion performance to SZ3 and up to 6.7× faster decompression, with serial examples such as Miranda at 11.7 s for STZ versus 63.1 s for SZ3. The comparison is informative because it isolates a feature that vanilla SZ3 does not natively provide: direct support for hierarchical access patterns (Wang et al., 1 Sep 2025).

5. Limits on out-of-domain data and semantics-sensitive tasks

Several later papers use SZ3 precisely to show where scientific-array predictors cease to transfer cleanly. In storage compression for pre-trained machine-learning models, SZ3 is treated as a state-of-the-art error-bounded scientific compressor but performs modestly because model parameters are described as trend-cluttered rather than smooth. On a 900-model sampled Hugging Face dataset totaling 575.88 GB, and applied only to floating-point layers under error bounds matched to the paper’s own method, SZ3 yields Overall AD = 0.18% and Overall CR = 1.16; moreover, 76.78% of models achieve only CR = 1.1× (Su et al., 2024).

A similar mismatch appears in particle-data clustering. For a HACC dataset with more than one billion 3D particles, plain SZ3 requires a relative error bound around $\epsilon$ 4 to preserve Friends-of-Friends cluster membership directly, and at that setting the compression ratio is only 1.42. The paper’s point is not that SZ3 violates its contract, but that pointwise coordinate-wise error bounds do not guarantee preservation of downstream cluster connectivity (Ren et al., 20 Apr 2026).

In federated learning, a gradient-aware error-bounded lossy compressor is evaluated using the same quantizer, entropy coder, and lossless backend as SZ3, but with a new predictor specialized to temporal and kernel-structure regularities in gradients. It reports up to 1.53× higher compression ratios than SZ3 with lower accuracy loss, supporting the interpretation that for gradient tensors the limiting factor is often predictor mismatch rather than the rest of the SZ3-style pipeline (Ye et al., 7 Nov 2025).

6. SZ3 in later research: baseline, augmentation target, and ecosystem component

SZ3 is repeatedly used as the principal rule-based baseline in later scientific-compression work. An attention-based learned compressor with guaranteed blockwise error bounds reports up to 8× higher compression ratio than SZ3 on S3D, up to 3× on E3SM, and up to 2× on XGC. A latent-diffusion framework for spatiotemporal data reports up to 4× higher compression on E3SM, 10× on S3D, and 5× on JHTDB at the same reconstruction error. Neural representations for weather and climate data report at least 10 times more compression ratio than SZ3 at the same weighted RMSE and weighted MAE, while ECNR reports an average +5.87 dB PSNR gain over SZ3 at similar compression ratios on four time-varying volumetric datasets (Li et al., 2024, Li et al., 2 Jul 2025, Huang et al., 2022, Tang et al., 2023).

At the same time, many papers do not attempt to replace SZ3, but to wrap it with new guarantees. FFCz augments off-the-shelf compressors such as SZ3 with joint spatial- and frequency-domain error control. On Nyx hiRes baryon density, the reported compression ratios are 44,307.1 for plain spatial-only SZ3, 218.8 for trial-and-error tightening until both spatial and frequency requirements happen to hold, and 34,957.7 for SZ3 + FFCz. A topology-preserving framework augments SZ3 so that

$\epsilon$ 5

thereby preserving the persistence-simplified contour tree while respecting the pointwise bound. For 2D tensor fields, TFZ enhances SZ3 so that topology errors drop to 0, whereas vanilla SZ3 at similar compressed size still produces, for example, 85, 582, 2,144, and 16,831 wrong cells on four symmetric datasets, and tens of millions on some asymmetric datasets (Ren et al., 4 Jan 2026, Gorski et al., 19 Feb 2025, Gorski et al., 12 Aug 2025).

Taken together, these later roles define SZ3’s current encyclopedic status. It is simultaneously a foundational modular framework, a high-quality scientific baseline, and an infrastructure component that other systems augment when they need guarantees or access patterns beyond pointwise error-bounded compression.