Compressed Computation Overview

• Compressed computation is a set of methods that integrate compression directly into algorithms, preserving key algebraic, statistical, and mechanistic properties.
• It encompasses direct compressed-domain operations, sparse recovery via low-rank or sufficient-statistic techniques, and resource-compressed execution in distributed and quantum systems.
• These methods reduce data movement and compute overhead while maintaining accuracy through tailored decompression, error controls, and optimized hardware strategies.

Compressed computation denotes a family of methods in which compression is part of the computational model rather than a purely storage-oriented preprocessing step. Taken together, the literature uses the term for several closely related ideas: computing directly on compressed graphs, matrices, arrays, and strings; reformulating expensive tasks as sparse recovery, low-rank completion, or sufficient-statistic estimation; reducing communication and data movement by compressing exchanged state; and compressing the computational substrate itself, as in matchgate-based quantum simulation or neuron-limited neural toy models (Francisco et al., 2017, Almeida et al., 2012, Hebenstreit et al., 2017, Newgas, 13 Jul 2025). Across these settings, the central question is not merely how much data can be compressed, but which algebraic, statistical, or mechanistic properties must be preserved so that useful computation remains possible.

1. Conceptual scope and recurrent design patterns

Taken together, the literature suggests that “compressed computation” is best understood as an umbrella term for several technical regimes rather than as a single formalism.

Regime	Core mechanism	Representative papers
Direct compressed-domain computation	Operate on compressed data without full decompression	(Francisco et al., 2017, Martel, 2022, Agarwal et al., 2024)
Sparse or low-rank reformulation	Replace full computation by recovery from structure	(Almeida et al., 2012, Khalesi et al., 2023, Xu et al., 2015)
Resource-compressed execution	Use fewer qubits, neurons, or communicated bytes	(Hebenstreit et al., 2017, Newgas, 13 Jul 2025, Kodali et al., 28 May 2026)

In the direct-computation line, the compressed object is treated as a data structure that exposes enough regularity to support arithmetic or querying with complexity tied to compressed size. This is explicit in computation-friendly graph compression, where the runtime for $A x^\top$ is made proportional to the differential representation rather than to the original edge count (Francisco et al., 2017), and in compressed matrix and array formats that preserve linear operations or bulk statistics directly in compressed form (Martel, 2022, Agarwal et al., 2024).

In the sparse-recovery line, compression is not primarily a bit-level encoding of the original object. Instead, the computation is rewritten so that only a sparse assignment matrix, a low-rank spectral matrix, or a finite set of sufficient statistics needs to be inferred or stored. Matrix completion for 2-D spectroscopy, compressed sensing for distributed computation, and transition-probability recovery for branching processes all follow this pattern (Almeida et al., 2012, Khalesi et al., 2023, Xu et al., 2015).

In the resource-compression line, what is compressed is the execution substrate: the number of qubits needed to simulate a many-body system, the number of nonlinear activations available to a neural network, or the number of communicated bytes in exascale eigensolvers. This usage is technically distinct from compressed storage, but it retains the same structural premise: exploit special algebraic form so that the effective compute budget is smaller than naive dimensional analysis would suggest (Hebenstreit et al., 2017, Newgas, 13 Jul 2025, Kodali et al., 28 May 2026).

2. Direct computation on compressed representations

A canonical formulation appears in graph algorithms. For WebGraph-style compression, each row $\mathbf v_i$ of the adjacency matrix is stored relative to an earlier similar row $\mathbf v_{r_i}$, and the product $A x^\top$ is evaluated incrementally through

$y_i = y_{r_i} + \sum_j A'_{ij}x_j.$

If $m'$ is the number of nonzeros in the differential matrix $A'$, the runtime is $\Theta(\mathrm{nnz}(A'))$, not proportional to the original edge count $m$ (Francisco et al., 2017). The same paper reports PageRank speedups such as 2.95× on eu-2015-hc, 2.05× on it-2004-hc, and 3.05× on uk-2014-hc, with Web graphs exhibiting compression factors such as 8.26× and 7.58×. The broader point is that a representation based on row similarity can support arithmetic directly because the dot product respects the differential encoding.

Lossless matrix compression pushes the same idea into numerical linear algebra. “Computable Compressed Matrices” stores positive-integer matrices as bitstrings while preserving element addressability and numerical meaning, so that one can, in principle, “compress, compute, and only decompress for reading” (Paixão et al., 2013). The Supreme Minimum method uses the bit-length of the maximum element uniformly across entries, while Variable Length Blocks stores each element with its own bit-length header. The paper’s synthetic study reports that VLB outperformed SM in 64,944 out of 65,536 tested Beta-parameter combinations, while SM is better when bit-length is constant across the matrix. The hardware motivation is explicit: RAM latency is described as roughly 150 CPU cycles, whereas L1 cache latency is 3 to 4 cycles.

Lossy array compression extends this compressed-domain view to floating-point tensors. The compressor blaz operates on $8\times 8$ blocks of binary64 matrices and supports fully compressed-domain addition and multiplication by a constant, while dot product and matrix multiplication use partial decompression (Martel, 2022). A block of 4096 bits is stored in 360 bits as 29 8-bit integers + 2 binary64 numbers, giving a compression rate of about 11.37. For $\mathbf v_i$0 matrices, the paper reports addition about 62× faster than uncompressed execution and more than 5000× faster than zfp, while matrix multiplication is “not much slower than uncompressed” and zfp is roughly 10× slower than blaz. PyBlaz generalizes the idea to arbitrary-dimensional arrays and supports negation, addition, scalar multiplication, dot product, mean, covariance, variance, $\mathbf v_i$1 norm, cosine similarity, SSIM, and an approximate Wasserstein distance directly on compressed data (Agarwal et al., 2024). Its error analysis is explicit: compression-induced error comes from binning and pruning, while most supported compressed-space operations add no further error beyond recomputation of bins after arithmetic and the approximation used for Wasserstein distance.

Compressed strings supply another mature example. For SLP-compressed texts, local subsequence recognition and partial semi-local LCS can be performed in

$\mathbf v_i$2

improving on the earlier $\mathbf v_i$3 bound, while global subsequence recognition admits an $\mathbf v_i$4 algorithm (0707.3407). The same literature also treats factorization itself as compressed computation: LZ-CICS computes both LZ77 and LZ78 in $\mathbf v_i$5 deterministic time, or $\mathbf v_i$6 randomized time, using $\mathbf v_i$7 bits of working space; with a compressed suffix tree already loaded, both factorizations can be computed in $\mathbf v_i$8 time using $\mathbf v_i$9 bits (Köppl et al., 2015).

These positive results coexist with strong lower bounds. Grammar boosting proves that random access on RePair, Greedy, LongestMatch, Sequential, and Bisection requires

$\mathbf v_{r_i}$0

time in $\mathbf v_{r_i}$1 space, while LZ78 requires $\mathbf v_{r_i}$2 (De et al., 2023). It also yields conditional lower bounds for compressed CFG parsing, including the statement that under the Combinatorial $\mathbf v_{r_i}$3-Clique Conjecture there is no algorithm running in

$\mathbf v_{r_i}$4

for all constants $\mathbf v_{r_i}$5 and $\mathbf v_{r_i}$6 on inputs compressed by several major grammar compressors. Compressed computation is therefore not a blanket claim that compression uniformly makes algorithms easier; the literature contains both constructive accelerations and provable barriers.

3. Sparse recovery, low-rank structure, and statistical compression

A second major lineage treats compression as an inferential principle. In 2-D spectroscopy, the unknown spectrum matrix $\mathbf v_{r_i}$7 is reconstructed from a random subset of entries by nuclear-norm minimization,

$\mathbf v_{r_i}$8

with Singular Value Thresholding used for large-scale recovery (Almeida et al., 2012). For the Fenna–Matthews–Olson example, the full $\mathbf v_{r_i}$9 spectrum required about 60 minutes on a laptop, whereas 1% sampling reproduced the major features qualitatively in about 2 minutes, and 8% sampling yielded a reconstruction with Frobenius error $A x^\top$0. The paper characterizes this as about a 30-fold speedup at 1% sampling.

Transition-probability computation for continuous-time branching processes follows a parallel logic. Transition probabilities are Fourier coefficients of the probability generating function, but a full FFT-style inversion requires $A x^\top$1 expensive PGF evaluations. By assuming the transition matrix is sparse in the spike basis and sampling only a reduced set of Fourier measurements, the compressed sensing generating-function method solves an $A x^\top$2-regularized recovery problem and reduces the number of required evaluations to about $A x^\top$3 (Xu et al., 2015). In the birth-death-shift example, the sampled PGF evaluations were reported as fewer than 2% of the full-grid computation for a representative $A x^\top$4 case, while maximum absolute errors were around $A x^\top$5 or smaller.

Multi-user distributed computation over the reals can likewise be expressed as sparse factorization. If $A x^\top$6 is the job matrix, $A x^\top$7 the decoding matrix, and $A x^\top$8 the computation matrix, exact recovery requires

$A x^\top$9

and the normalized computation cost is

$y_i = y_{r_i} + \sum_j A'_{ij}x_j.$0

The paper reformulates this as

$y_i = y_{r_i} + \sum_j A'_{ij}x_j.$1

so that designing a computation assignment becomes a sparse recovery problem (Khalesi et al., 2023). A simple random construction yields the upper bound $y_i = y_{r_i} + \sum_j A'_{ij}x_j.$2, while a Basis Pursuit guarantee is given under a Lambert-$y_i = y_{r_i} + \sum_j A'_{ij}x_j.$3 sparsity condition involving the sub-Gaussian parameter $y_i = y_{r_i} + \sum_j A'_{ij}x_j.$4.

Compressed optimization of device architectures makes the same move in an inverse-control setting. CODA linearizes a device map $y_i = y_{r_i} + \sum_j A'_{ij}x_j.$5 around a working point and replaces an $y_i = y_{r_i} + \sum_j A'_{ij}x_j.$6-sparse control objective by the convex surrogate

$y_i = y_{r_i} + \sum_j A'_{ij}x_j.$7

thereby favoring sparse and local voltage changes (Frees et al., 2014). The paper uses the $y_i = y_{r_i} + \sum_j A'_{ij}x_j.$8 magnitude of the control update, denoted $y_i = y_{r_i} + \sum_j A'_{ij}x_j.$9, as a benchmarking metric and reports that reducing the SiGe spacer height improves controllability by producing smaller $m'$0.

Compression also appears as a sufficient-statistic reduction in statistical estimation. “You Only Compress Once” groups identical feature vectors $m'$1 and stores

$m'$2

showing that OLS coefficients and their covariance matrices can be recovered exactly from compressed data, including under heteroskedasticity and clustered autocorrelation (Wong et al., 2021). The compressed weighted regression preserves

$m'$3

This is a distinct sense of compressed computation: the raw sample is discarded, but the algebra needed for estimation and inference is preserved exactly.

A further variant is robust subspace clustering from compressed random projections. Row Space Pursuit observes only $m'$4, with $m'$5, and directly targets the row space $m'$6 of the low-rank component rather than reconstructing the full data (Liu et al., 2018). The alternating optimization updates $m'$7 by taking top right singular vectors and updates the sparse corruption by shrinkage; the per-iteration cost is dominated by $m'$8. The paper argues that clustering can then be performed on the recovered row-space representation without constructing an $m'$9 affinity matrix.

4. Communication-aware compressed computation

In distributed systems, the main bottleneck is often not arithmetic but communication. Compressed coded distributed computing merges two previously separate ideas: combining intermediate results from the same Reduce function and coding across different tasks (Li et al., 2018). For linear reductions,

$A'$0

the paper gives the achievable load

$A'$1

to be compared with

$A'$2

and

$A'$3

The significance is exactness with reduced communication load: local sums are first formed within a task and are then coded so that one transmission serves multiple receivers and jobs.

A related but more information-theoretic line studies helper-based compression for function computation. “Applications of Common Information to Computing Functions” extends Gács–Körner–Witsenhausen common information to functional common information $A'$4, using a helper that sends a nest variable $A'$5 induced by the function and then allows the sources to encode only what remains ambiguous within each nest (Malak, 2021). The resulting rates take the form

$A'$6

with several refinements for permutation-invariant functions via bipartite graph decompositions. The conceptual shift is that the communicated object is not the raw source pair but the task-relevant common structure.

Exascale electronic-structure simulation brings the same principle to HPC. In fully relativistic NC-SOC DFT, the sparse generalized eigenproblem

$A'$7

is solved by residual-based Chebyshev filtered subspace iteration, which is explicitly tolerant to inexact matvecs (Kodali et al., 28 May 2026). This residual awareness enables FP32/TF32 filtering and block floating-point compressed MPI halo exchange with compression ratios over 4x, while retaining “double-precision robustness.” For the 732-atom Fe$A'$8GeTe$A'$9 system, the paper reports a reduction from 5503 s in the FP64/FP64 baseline to 2427 s for FP32(TF32)+BPV12, i.e. 2.27× faster, with an energy difference of about $\Theta(\mathrm{nnz}(A'))$0 Ha/atom. Here compression is not a property of the input alone; it is embedded in the iterative numerical method and justified by residual-based convergence control.

5. Scientific simulation, operator compression, and quantum execution

Several works compress not only data but operator families themselves. Compressed convolution starts from the observation that many convolution kernels are highly redundant and can be represented in an optimal eigenbasis (Elsner et al., 2013). If

$\Theta(\mathrm{nnz}(A'))$1

then only the basis convolutions

$\Theta(\mathrm{nnz}(A'))$2

need to be computed expensively; each desired output is then reconstructed by a cheap linear combination. The truncation error is the sum of discarded eigenvalues, and the output recovery is described as constant time per desired convolved output once the basis convolutions are available. For Planck $\Theta(\mathrm{nnz}(A'))$3, six detector beams can be compressed to three modes. For the Keck array, 2480 beams can be compressed to 8 modes, reducing convolution operations by a factor of about 310, and the paper describes cost reductions of two to three orders of magnitude with negligible loss of accuracy.

Compressed quantum computation compresses a whole circuit family rather than a data object. For nearest-neighbor matchgate circuits with computational-basis input and a single-qubit $\Theta(\mathrm{nnz}(A'))$4 measurement, an $\Theta(\mathrm{nnz}(A'))$5-qubit computation can be compressed to $\Theta(\mathrm{nnz}(A'))$6 qubits, and for the Ising model symmetry reduces this to essentially $\Theta(\mathrm{nnz}(A'))$7 qubits (Hebenstreit et al., 2017). The experimental demonstration on IBM Quantum Experience simulated a four-qubit Ising chain using only two physical qubits. The Hamiltonian is

$\Theta(\mathrm{nnz}(A'))$8

and the magnetization is recovered from a single-qubit $\Theta(\mathrm{nnz}(A'))$9 measurement after compressed evolution. The experiment sampled

$m$0

used $m$1 and $m$2, and estimated systematic error with validating circuits, reporting

$m$3

This line of work is highly specialized—it relies on matchgate structure and does not compress arbitrary quantum algorithms—but it is a clear instance in which a many-body computation is executed on a smaller quantum register while preserving the target observable.

Taken together, operator-compression and circuit-compression approaches show that compressed computation is not restricted to storing fewer bytes. It can instead mean diagonalizing a family of kernels into a low-dimensional mode basis, or identifying a circuit class whose algebra permits execution on a logarithmically smaller quantum device.

6. Neural-network meanings and the superposition dispute

A recent neural-network literature uses “compressed computation” in a more mechanistic sense: the bottleneck is the number of nonlinear units rather than memory or bytes. “Compressed Computation: Dense Circuits in a Toy Model of the Universal-AND Problem” studies a one-layer ReLU network

$m$4

where the hidden dimension $m$5 limits the number of nonlinear activations available for computing all pairwise ANDs of $m$6 sparse Boolean inputs (Newgas, 13 Jul 2025). The paper reports that training finds a fully dense solution in which every neuron contributes to every output, with incoming weights that cluster around two values. This circuit scales with dimension, trades off error rate against neuron efficiency, is robust to changes in sparsity and other key parameters, and extends naturally to other Boolean operations and Boolean circuits. The central claim is not that the network learns the previously studied sparse theoretical constructions, but that it prefers a dense reusable circuit basis.

A distinct 2026 paper scrutinizes a different “Compressed Computation” toy model that appeared to compute 100 ReLU functions with just 50 neurons (Bhagat et al., 12 Jun 2026). The authors show that the residual architecture is equivalent to a one-layer MLP trained on

$m$7

so the task includes a structured mixing term induced by the embedding. The residual model computes

$m$8

and rearrangement yields the equivalent one-layer objective

$m$9

The paper reports several convergent pieces of evidence that the apparent advantage is due to label mixing rather than to “computation in superposition”: the gain disappears when $8\times 8$0, a model trained with $8\times 8$1 reverts to the naive loss after fine-tuning on clean labels, performance scales with the magnitude of the mixing matrix, and the learned neuron directions concentrate in the 50-dimensional subspace associated with the top 50 eigenvalues of $8\times 8$2. A hand-designed semi-non-negative matrix factorization baseline derived solely from $8\times 8$3 reproduces the qualitative loss profile and improves on prior baselines, though it does not match the trained model. The paper’s conclusion is explicit: the CC model is “probably not a good toy model of computation in superposition.”

This clarification matters because it separates several notions that are easy to conflate. Neuron reuse under a hidden-width bottleneck, dense shared circuitry, storage superposition, and label mixing are not the same mechanism. A plausible implication is that future neural theories of compressed computation will need to distinguish carefully between genuine reuse of nonlinear computation and performance gains produced by architectural side effects such as induced correlations in the labels.