DASPack: Efficient Compression for DAS Data

• DASPack is a DAS-specific compression framework that enforces a controlled per-datum error bound via fixed-accuracy quantization and reversible processing.
• It employs a pipeline using wavelet transforms, low-order linear predictive coding, and entropy coding to optimize throughput and compression ratios.
• Validated on 15 diverse datasets, DASPack reliably supports the long-term storage and transport of massive distributed acoustic sensing data streams.

Searching arXiv for the DASPack paper and closely related DAS compression work. Distributed Acoustic Sensing (DAS) generates dense spatiotemporal measurements by turning telecom fiber into arrays of strain and strain‑rate sensors, but the resulting data volumes impose severe constraints on transfer, storage, and long-term retention. DASPack is a high-performance, open-source compression tool designed specifically for DAS data, with a compression framework tailored to DAS’s 2‑D spatiotemporal structure, heterogeneous sampling parameters, and domain requirements for controlled fidelity. It supports both controlled lossy and lossless compression by allowing users to select the maximum absolute deviation per datum via a fixed-accuracy quantization step before lossless compression, then applying a reversible transform/predictive/entropy pipeline. The system was validated on 15 datasets spanning multiple acquisition environments and is presented as a practical foundation for long-term, sustainable DAS data management in large-scale monitoring networks (Segui et al., 22 Jul 2025).

1. Definition and problem setting

DASPack is a compression framework and software implementation for distributed acoustic sensing data. Its defining property is controlled error: users choose a maximum absolute deviation per datum, and the remainder of the pipeline is exactly reversible, so the only information loss arises from the user’s quantization choice (Segui et al., 22 Jul 2025).

The problem it addresses is the scale of DAS acquisition. A single interrogator recording 5,000 channels at 1 kHz with 4‑byte floating points produces nearly 1.5 TB per day, and multi‑interrogator campaigns regularly exceed hundreds of terabytes per month. Across the 15 datasets evaluated in the paper, sampling rates span 50–10,000 Hz, channel counts from a few hundred to over 7,000, and gauge lengths 2–10 m; data are stored as 16‑bit integers, 32‑bit integers, or 32‑bit floats, and include both nanostrain and nanostrain‑rate signals (Segui et al., 22 Jul 2025).

This scale is central to the software’s design. DASPack is not a general-purpose codec repurposed for sensor data, but a DAS-specific compressor built around the data’s 2‑D time×distance organization and its domain requirement for explicit fidelity bounds. Relative to prior DAS compression methods, general-purpose codecs such as GZIP and Zstd, and image codecs such as JPEG2000, it is described as consistently achieving higher compression factors at comparable or higher throughput while uniquely offering an explicit, per‑datum error bound enforced by construction (Segui et al., 22 Jul 2025).

A useful distinction must be made between DASPack and compressed-domain sensing pipelines. Work on compressed-domain vibration detection and classification for DAS reduces data volume at acquisition via compressive sensing and performs inference directly on compressed measurements, achieving a 70% reduction in transmitted data with high detection and classification performance, but that approach is conceptually different from DASPack’s post-acquisition controlled compression framework (Shen et al., 2022). This suggests that DASPack belongs to the storage-and-transport layer of DAS workflows rather than the acquisition-layer compressive sensing paradigm.

2. Controlled fidelity model

The central scientific abstraction in DASPack is the separation of fidelity specification from bit-rate optimization. Controlled lossy mode begins with a single quantization step:

$$d_{\text{rounded}} = \operatorname{round}(d_{\text{orig}} / \Delta)\cdot \Delta.$$

This produces a strict $L_\infty$ guarantee:

$\max_i |x_i - \hat{x}_i| \le \epsilon,$

with

$\epsilon = \Delta/2.$

Because rounding to a grid of spacing $\Delta$ incurs at most half-step error, the relationship between $\Delta$ and $\epsilon$ is exact by construction, and no further lossy stage exists in the pipeline (Segui et al., 22 Jul 2025).

This design is significant for scientific workflows because the accuracy parameter is interpretable in the physical units of the signal. In strain data, the paper reports that $\Delta \approx 0.1$ n$\epsilon$ yields near‑perfect fidelity with approximately 6× compression, while $\Delta \approx 1$ n$L_\infty$0 yields approximately 9–10× compression with acceptable loss of low‑amplitude features. For strain‑rate data, the paper expresses $L_\infty$1 relative to raw int16 counts, noting that step 1 is lossless and step 10 reduces precision by one decimal digit (Segui et al., 22 Jul 2025).

Lossless mode is obtained by setting $L_\infty$2 equal to the native integer quantization, for example 1 count for int16 data. Under this condition, the transforms are exactly invertible with integer arithmetic and the reconstruction is bit‑for‑bit recoverable modulo container overhead. If $L_\infty$3 is smaller than the native quantization, the quantization step is effectively lossless (Segui et al., 22 Jul 2025).

The paper further states that small $L_\infty$4 yields near-uniform, uncorrelated quantization noise that preserves temporal and spatial bandwidth and resolution. A plausible implication is that DASPack’s controlled lossy mode is intended not merely for approximate visualization or archival compaction, but for workflows where bounded perturbations must remain interpretable in downstream scientific analysis.

3. Compression architecture

DASPack’s pipeline combines wavelet transforms, linear predictive coding, and entropy coding. The end‑to‑end process comprises integer mapping and optional quantization, tiling and mean removal, an optional wavelet transform in both dimensions, linear predictive coding on low-pass subbands, and entropy coding of residuals (Segui et al., 22 Jul 2025).

The initial stage converts raw data of type int16, int32, or float32 to signed integers using the chosen $L_\infty$5, thereby enforcing the absolute error bound. The same $L_\infty$6 is applied across all data to avoid discontinuities. The 2‑D time×distance array is then split into non‑overlapping tiles processed independently; square tiles of 2000 samples per side were used in benchmarks. Per‑channel temporal means are subtracted and stored as side information to improve prediction. Tiling enables parallelism and constrains linear predictive coding side‑information overhead (Segui et al., 22 Jul 2025).

The transform stage uses the biorthogonal LGT 5/3 wavelet implemented via the lifting scheme, yielding computational efficiency and exact integer‑to‑integer transforms as in JPEG2000. A single decomposition level was found sufficient; further levels did not improve compression. The transform separates approximation and detail bands in time and space without increasing sample count and allows perfect reconstruction (Segui et al., 22 Jul 2025).

Prediction is applied to the low‑pass wavelet subbands using low‑order linear predictive coding, first along time per channel and then along space on the resulting residuals. The residual computation follows

$L_\infty$7

Small orders are used to limit side information, typically $L_\infty$8 in time and approximately $L_\infty$9 in space. Coefficients are estimated to minimize mean squared error using the Levinson–Durbin recursion on unbiased autocorrelation with Toeplitz structure, then discretized and stored per tile to ensure exact reconstruction. Mean removal improves model fit and stability by eliminating low-frequency bias (Segui et al., 22 Jul 2025).

The final stage uses arithmetic coding via the Constriction library with per‑channel Gaussian modeling. Since residuals exhibit approximately Gaussian distributions with reduced variance, the model stores the channel’s mean and variance as parameters. To encode outliers efficiently, k‑exponential Golomb codes are used as a compact variable-length representation for large integers. The paper notes that these choices approach the Shannon limit when residuals are near-white (Segui et al., 22 Jul 2025).

4. Performance characteristics

The paper defines the compression factor as

$\max_i |x_i - \hat{x}_i| \le \epsilon,$0

and the source throughput as

$\max_i |x_i - \hat{x}_i| \le \epsilon,$1

where $\max_i |x_i - \hat{x}_i| \le \epsilon,$2 is the number of channels, $\max_i |x_i - \hat{x}_i| \le \epsilon,$3 the sampling rate, and $\max_i |x_i - \hat{x}_i| \le \epsilon,$4 the datum size. Real‑time operation is feasible when $\max_i |x_i - \hat{x}_i| \le \epsilon,$5 (Segui et al., 22 Jul 2025).

Across 15 datasets and multiple accuracies, DASPack achieves up to 3× compression in lossless mode for strain and strain‑rate data. In controlled lossy mode, the paper reports approximately 6× compression at near‑perfect fidelity and up to approximately 10× compression at acceptable degradation. Throughput is dataset-dependent: single-thread performance on an Apple M1 Pro CPU is typically 100–200 MB/s, reaching up to approximately 250 MB/s on strain and approximately 125 MB/s on strain‑rate, while multi-threaded execution reaches approximately 750 MB/s with 8 threads (Segui et al., 22 Jul 2025).

The benchmarks used 5‑minute excerpts per dataset, single‑thread runs on an Apple M1 Pro CPU, square tiles of 2000 samples per side, two LPC coefficients, and 32‑bit integer operations. Figure 1 in the paper compares dataset operating points against compressor limits and shows all datasets below the DASPack curve, including the most demanding case, Zurisee at 2,496 channels and 10 kHz. JPEG2000, Zstd, and GZIP fail on the extreme, whereas DASPack, Zfp, and H5TurboPFor can handle all configurations in real time (Segui et al., 22 Jul 2025).

The paper’s fidelity evaluation is tied to the guaranteed bound rather than to aggregate distortion metrics. Formal RMSE, MAE, SNR, and PSNR are not reported. Instead, fidelity is assessed through the enforced $\max_i |x_i - \hat{x}_i| \le \epsilon,$6 bound together with visual comparisons and error maps for real events, including Athens ML 3.4, Alboran mbLg 3.1, Vinaroz quarry blast, and La Palma mbLg 4.9. Quantization at 0.1 n$\max_i |x_i - \hat{x}_i| \le \epsilon,$7 yields negligible errors; at 1–4 n$\max_i |x_i - \hat{x}_i| \le \epsilon,$8, significant peaks are preserved with small residuals (Segui et al., 22 Jul 2025).

5. Data dependence and comparative behavior

DASPack uses the same pipeline for strain and strain‑rate data, but the two signal types behave differently under compression. Strain, often represented as 32‑bit data, has larger dynamic range and stronger temporal coherence, so wavelet plus LPC remove slowly varying components effectively and yield higher compression factors. Strain‑rate, often 16‑bit, has narrower dynamic range and weaker coherence as a derivative of strain, which limits predictor power and overall compressibility. The paper emphasizes that this difference is a property of the data rather than the method (Segui et al., 22 Jul 2025).

The comparative baselines include GZIP, Zstd, JPEG2000, H5TurboPFor, and Zfp. GZIP and Zstd provide moderate compression with relatively low throughput, generally below 100 MB/s. H5TurboPFor and Zfp prioritize speed and often exceed 200 MB/s; H5TurboPFor maintains high compression for strain‑rate data, while Zfp underperforms on strain‑rate and is only moderately better on strain. JPEG2000 reaches higher compression at relaxed accuracy but is very slow, around 10–20 MB/s (Segui et al., 22 Jul 2025).

Within this comparison, DASPack is reported to deliver the strongest compression factors across datasets and accuracies while maintaining high throughput. At higher tolerances, both JPEG2000 and DASPack exceed 8×, and DASPack reaches approximately 10× for some datasets. The stated explanation is architectural: integer lifting wavelets align the data with LPC locality assumptions; low-order LPC removes dominant low-frequency components and flattens residual spectra; arithmetic coding with Gaussian modeling and robust tail handling approaches entropy limits. By contrast, dictionary coders ignore 2‑D structure, and codecs tuned to floats or images target different correlation patterns (Segui et al., 22 Jul 2025).

An objective misconception to avoid is that higher throughput alone explains DASPack’s advantage. The paper’s argument is instead that its gains arise from matching transform, predictor, and entropy model to DAS spatiotemporal redundancies. Another misconception is that lossy performance depends on a hidden approximate stage later in the pipeline; the paper explicitly states that no further lossy stage exists beyond the initial quantization (Segui et al., 22 Jul 2025).

6. Implementation, interfaces, and workflow integration

DASPack is implemented in Rust for speed and memory safety and is released as open source at https://github.com/asleix/daspack. Entropy coding uses the Constriction arithmetic coding library. Python bindings are provided, and data are stored in HDF5 format together with side information such as means, LPC coefficients, and entropy model parameters (Segui et al., 22 Jul 2025).

The implementation details are tightly coupled to reversibility and operational deployment. Integer arithmetic and lifting ensure exact reversibility after quantization. Per‑tile independence bounds memory footprint, supports real‑time streaming, and enables parallel multi‑thread operation. Side information is stored with tiles to guarantee exact reconstruction (Segui et al., 22 Jul 2025).

The principal user-configurable parameters are the quantization step $\max_i |x_i - \hat{x}_i| \le \epsilon,$9, predictor order $\epsilon = \Delta/2.$0, wavelet level, tile size, and number of threads. Recommended settings in the paper are concise: one level of 5/3 lifting in time and space, small predictor orders with two coefficients in time and approximately one in space, square tiles of 2000 samples per side, and multi-threading to reach approximately 750 MB/s on modern CPUs. Larger predictor orders and additional wavelet levels did not improve efficiency in tests, because overhead exceeded gains (Segui et al., 22 Jul 2025).

The storage format and bindings make DASPack a workflow component rather than merely a standalone codec. Because HDF5 side information is preserved, the compressor interoperates with DAS workflows end‑to‑end. This suggests that the software is intended for integration into acquisition, archival, and analysis pipelines where traceable metadata and exact decompression semantics are required.

7. Applications, limitations, and prospective developments

The 15 evaluated datasets span landslides in Switzerland, urban monitoring in Athens, glaciers at Rhonegletscher, volcanic zones in Iceland, Tenerife, and La Palma, lakes such as Zurisee, and submarine cables in Alboran, Gran Canaria, Valencia, and Vinaroz. Examples in the paper show that earthquakes and blasts are retained with high fidelity at moderate $\epsilon = \Delta/2.$1, and DASPack is presented as enabling practical storage savings without compromising critical scientific content (Segui et al., 22 Jul 2025).

The core trade-off is compression ratio versus fidelity. Smaller $\epsilon = \Delta/2.$2 preserves microseismic detail, while larger $\epsilon = \Delta/2.$3 attenuates low-amplitude features but increases $\epsilon = \Delta/2.$4. A second trade-off is CPU versus I/O: with 8 threads, compression can exceed typical HDD write speeds of approximately 200 MB/s, so SSDs or other fast storage are recommended to avoid I/O bottlenecks. A third is latency versus throughput: tile-based independent processing supports low latency and steady throughput for streaming (Segui et al., 22 Jul 2025).

The paper identifies several limitations and edge cases. Highly nonstationary signals within a tile can reduce LPC effectiveness. Very small tiles increase side-information overhead, whereas very large tiles mix disparate statistics. The default 2000×2000 tile size is presented as a balanced choice but one that should be tuned. Strain‑rate’s lower coherence intrinsically limits compression compared with strain. Additional wavelet levels and higher-order predictors did not improve compression on the tested data (Segui et al., 22 Jul 2025).

Future directions discussed in the paper include adaptive compression driven by event detection, so that event windows are stored at fine $\epsilon = \Delta/2.$5 and quiet periods at coarser $\epsilon = \Delta/2.$6, and combination with upstream sampling methods such as decimation, subsampling, or compressed sensing, with DASPack then applied losslessly downstream for further entropy-based reduction (Segui et al., 22 Jul 2025). In relation to compressed-domain sensing work that avoids reconstruction and operates directly on compressed measurements (Shen et al., 2022), this suggests a broader systems perspective in which DASPack can coexist with acquisition-time reduction methods rather than compete with them.