Linear Sketching: Algorithms & Applications

• Linear sketching is the use of linear maps to compress high-dimensional data while retaining essential structure for computation, estimation, or recovery.
• It underpins diverse applications from streaming algorithms and numerical linear algebra to graph sparsification, offering efficient dimensionality reduction and data analysis.
• Despite its versatility, linear sketching exhibits trade-offs such as null-space loss and model-sensitive limitations, affecting optimality in various settings.

Searching arXiv for foundational and directly relevant papers on linear sketching. Linear sketching is the use of a linear map to compress a high-dimensional object while retaining enough structure for computation, estimation, or recovery. In the vector setting this takes the form $x \mapsto Sx$; in matrix problems it is typically $A \mapsto SA$; and in streaming or graph settings the sketch is maintained incrementally under updates because linearity allows the compressed state to be updated without revisiting the full input (Woodruff, 2014). Across the literature, the same idea appears under several closely related interpretations: oblivious subspace embeddings for numerical linear algebra, linear sketches for streaming and communication, sketch-and-project methods for linear systems, and compressed summaries for frequency vectors, graph incidences, and covariance matrices (Hosseini et al., 2018). The topic therefore spans both algorithm design and lower bounds: linear sketches can be optimal in some models, provably accelerate certain iterative solvers, and yet be fundamentally limited by null-space loss, adaptivity barriers, or communication lower bounds in others (Rebrova et al., 2019).

1. Formal definitions and problem classes

At its most general, a linear sketch maintains a compressed representation

$\mathrm{sketch}(x)=Sx,$

where $S$ is a matrix over the same ambient algebraic structure as the data. In streaming formulations, $x$ may lie in $\mathbb{Z}^n$, $\mathbb{F}_2^n$, or $\mathbb{Z}_p^n$, and updates modify coordinates by addition, XOR, or addition modulo $p$. The algorithm stores only $Sx_t$, not $A \mapsto SA$0 itself, and outputs either an exact value $A \mapsto SA$1 or an approximation whose quality is measured, for example, by an $A \mapsto SA$2-type mean-squared guarantee (Hosseini et al., 2018). This linearity is the reason sketches compose naturally with turnstile updates, distributed aggregation, and one-pass streaming.

In matrix problems, the dominant abstraction is left multiplication by a sketching matrix $A \mapsto SA$3, producing $A \mapsto SA$4. The central geometric notion is the $A \mapsto SA$5 $A \mapsto SA$6-subspace embedding: $A \mapsto SA$7 preserves $A \mapsto SA$8 for every $A \mapsto SA$9, equivalently $\mathrm{sketch}(x)=Sx,$0 when $\mathrm{sketch}(x)=Sx,$1 is an orthonormal basis for $\mathrm{sketch}(x)=Sx,$2 (Woodruff, 2014). This viewpoint unifies overdetermined least squares, low-rank approximation, preconditioning, and iterative Hessian sketching.

A third formulation, common in streaming and network measurement, represents a frequency vector $\mathrm{sketch}(x)=Sx,$3 through a sketch array $\mathrm{sketch}(x)=Sx,$4, usually with $\mathrm{sketch}(x)=Sx,$5. For Count-Min and Count Sketch, $\mathrm{sketch}(x)=Sx,$6 is a very sparse $\mathrm{sketch}(x)=Sx,$7 or $\mathrm{sketch}(x)=Sx,$8 matrix, each column has exactly $\mathrm{sketch}(x)=Sx,$9 nonzeros, and the full matrix is not stored explicitly because the corresponding column can be generated online from hash functions (Yuan et al., 15 Mar 2026). The same formalism underlies graph sketching when the sketched object is an incidence or Laplacian representation rather than a dense vector (Chen et al., 2022).

2. Embeddings, sketch families, and geometric effects

The classical sketch families differ mainly in the tradeoff among embedding dimension, multiplication cost, and structural guarantees. Dense Johnson–Lindenstrauss matrices and Gaussian sketches give $S$0 $S$1-subspace embeddings with $S$2, while sparse embedding matrices of CountSketch type achieve input-sparsity-time multiplication $S$3 at the cost of larger row dimension, such as $S$4 in the basic sparse-subspace-embedding results (Woodruff, 2014). Structured transforms such as the SRHT and ROS preserve the same subspaces more quickly than dense Gaussian multiplication and are therefore standard in high-dimensional least squares and Hessian sketching (Chen et al., 8 Sep 2025).

For sketched least squares and randomized second-order methods, these embeddings are used not only to approximate the objective but also to approximate curvature. In iterative Hessian sketching one replaces $S$5 or $S$6 by a sketched Hessian $S$7, solving a sequence of reduced systems whose conditioning is controlled by the embedding quality (Wang et al., 2021). In sequential sketch-and-solve schemes, one uses a sequence of sketch sizes $S$8, with increasing statistical precision, and refines the solution over a hierarchy of sketched subproblems until a short final refinement on the full data reaches ordinary least-squares prediction accuracy (Chen et al., 8 Sep 2025).

Linear sketches also modify geometry in more subtle ways. In the sketched Motzkin method for consistent, overdetermined systems $S$9, Gaussian sketching is inserted into the maximal-residual selection rule rather than used as a one-shot preprocessing step. The key fact is that $x$0 is Gaussian, so the maximal coordinate scales like $x$1, yielding a one-step contraction

$x$2

and hence a logarithmically accelerated global rate for Gaussian Sketched Motzkin relative to randomized Kaczmarz (Rebrova et al., 2019). This is an instance in which the sketch does not merely reduce dimension; it reshapes the residual vector in a way that strengthens a greedy update.

The same geometric perspective also clarifies the limits of recovery. For a sketch $x$3, the decomposition $x$4 into range-space and null-space components shows that $x$5 captures only $x$6, while $x$7 is orthogonal to $x$8 and irrecoverable from $x$9 without additional prior structure (Yuan et al., 15 Mar 2026). This range–null decomposition is a precise statement of the compression–recovery tradeoff that appears throughout the subject.

3. Streaming, communication, and optimality

In streaming complexity, linear sketching is not merely a convenient design pattern; in some regimes it is essentially universal. For XOR streams over $\mathbb{Z}^n$0, if a Boolean function $\mathbb{Z}^n$1 can be computed by a $\mathbb{Z}^n$2-bit streaming algorithm on streams of length $\mathbb{Z}^n$3, then its exact randomized $\mathbb{Z}^n$4-sketch complexity is $\mathbb{Z}^n$5. For modular updates over $\mathbb{Z}^n$6, the analogous statement holds for streams of length $\mathbb{Z}^n$7, and an approximate version preserves $\mathbb{Z}^n$8-type error up to constants (Hosseini et al., 2018). The proof passes through many-player one-way communication, Fourier analysis, and Chang’s lemma, and it shows that sufficiently long modular-update streaming algorithms must behave like low-dimensional linear sketches.

The federated setting imposes an even stricter linearity constraint. In multi-round secure aggregation, each round reveals only the sum of local messages, so the server receives a per-round linear sketch of the round histogram and cannot retroactively apply a new sketch to earlier data. Under this model, the paper on federated heavy hitters proves that heavy hitter discovery and approximate histograms have distinct optimal communication laws: for $\mathbb{Z}^n$9-ApproxHH the per-user cost is $\mathbb{F}_2^n$0, while for $\mathbb{F}_2^n$1-ApproxHist it is $\mathbb{F}_2^n$2, establishing a separation between logarithmic and $\mathbb{F}_2^n$3-type round dependence under linear sketching (Gascon et al., 2023). Algorithmically, the heavy-hitter upper bound is achieved by local threshold subsampling combined with invertible Bloom lookup tables, which are themselves linear, mergeable sketches.

These optimality statements have a double significance. First, they justify why lower bounds for linear sketches are often treated as lower bounds for more general streaming algorithms in long modular-update models. Second, they show that the exact operational constraints—turnstile updates, modular arithmetic, secure aggregation, per-round separation—change what “optimal linear sketching” means. The paradigm is therefore model-sensitive rather than purely algebraic (Hosseini et al., 2018).

4. Sketching in numerical linear algebra and iterative solvers

The most established use of linear sketching in numerical linear algebra is to reduce the dimension of an overdetermined problem while preserving the objective sufficiently well to solve the smaller instance. For $\mathbb{F}_2^n$4 regression, sketch-and-solve on $\mathbb{F}_2^n$5 and $\mathbb{F}_2^n$6 produces a $\mathbb{F}_2^n$7-approximate solution in input-sparsity time $\mathbb{F}_2^n$8 when $\mathbb{F}_2^n$9 is an appropriate subspace embedding (Woodruff, 2014). Preconditioning variants use a constant-accuracy sketch to build a matrix $\mathbb{Z}_p^n$0 such that $\mathbb{Z}_p^n$1 has small condition number, after which conjugate-gradient-type methods obtain machine-precision solutions with only logarithmically many passes over the full data (Woodruff, 2014).

For statistical regression, sketching changes not only computation but also uncertainty. In sketched linear regression with $\mathbb{Z}_p^n$2, a projector-based analysis expresses the estimator through the oblique projector $\mathbb{Z}_p^n$3 and the bias projector $\mathbb{Z}_p^n$4. Conditional on the sketch, $\mathbb{Z}_p^n$5, and under rank preservation the estimator remains unbiased but its covariance dominates that of ordinary least squares through the excess term involving $\mathbb{Z}_p^n$6 (Chi et al., 2018). This gives an exact bias–variance decomposition under minimal assumptions on the sketching matrix and shows that even rank-preserving sketches introduce algorithm-induced variance.

Several later works push sketching from one-shot approximation into iterative optimization. Distributed iterative Hessian sketching for Markov-parameter estimation replaces $\mathbb{Z}_p^n$7 by independent sketched Hessians $\mathbb{Z}_p^n$8 across workers, yielding geometric convergence with rate $\mathbb{Z}_p^n$9 under sub-Gaussian or ROS sketches and sketch sizes proportional to $p$0 or $p$1 (Wang et al., 2021). Sequential least-squares estimators with fast randomized sketching combine sketch-and-solve and iterative sketching in a multilevel schedule and obtain a final prediction error matching OLS up to $p$2, while empirical runtime improves over PCG and iterative double sketching (Chen et al., 8 Sep 2025).

For linear systems, the sketch-and-project framework interprets an iteration as projection onto the sketched equation $p$3. Structured sketching extends this by letting the sketch depend on residual history, which yields short recurrences, orthogonal residuals, and finite termination in exact arithmetic. In SPD and symmetric-indefinite settings these residual-history sketches recover CG-like behavior with low memory, while for general nonsymmetric systems they can be combined with inner PLSS solves (Brust et al., 2024). A complementary extension is augmented Lagrangian sketching: penalty sketching and ALS replace the hard sketched constraint by a penalized or augmented-Lagrangian term, recovering randomized Kaczmarz-, coordinate-, Gaussian-pursuit-, and block-type methods as special cases while retaining global linear convergence and $p$4 Cesàro rates under broad sketch distributions (Morshed, 2022).

5. Recovery, inversion, and learned priors

When the sketch is viewed as a compressed code $p$5, exact recovery is obstructed by the null space of $p$6. In compressed sensing this obstruction is mitigated by sparsity and RIP; in streaming sketches such as Count-Min or Count Sketch, however, the sensing matrices are extremely sparse and the paper on generative linear sketching argues that their RIP behavior is poor, with sketch matrices such as CM, CS, PR, and NZE exhibiting large empirical “RIP distance” relative to dense Gaussian, Bernoulli, or Fourier constructions (Yuan et al., 15 Mar 2026). This motivates a recovery view in which the sketch is treated as an ill-posed inverse problem rather than as a sufficient statistic for exact reconstruction.

The proposed resolution is to supply a prior rich enough to parameterize the missing null-space degrees of freedom. In that framework, for a sketching matrix $p$7 of rank $p$8, there exists a latent variable $p$9 and a mapping $Sx_t$0 such that $Sx_t$1 uniquely determines $Sx_t$2. The FLORE framework instantiates this idea with Count-Min on the data plane and an invertible flow-based generative model on the control plane, trained by measurement consistency, reconstruction, invertibility, orthogonality, and sparsity losses; the reported gains are up to $Sx_t$3 lower per-element error and up to $Sx_t$4 speedup over learning-based baselines on several real and synthetic datasets (Yuan et al., 15 Mar 2026). The conceptual shift is that linear sketching need not be judged solely by worst-case recoverability from linear algebra; it can also serve as a compressed observation model for posterior inference.

At the same time, recovery from sketches is not always best served by linear sketches. For inner-product estimation, weighted Minwise Hashing is shown to match the classical linear-sketch guarantee $Sx_t$5 in dense regimes while improving it to

$Sx_t$6

for sparse vectors with limited overlap $Sx_t$7, thereby outperforming CountSketch- and JL-type sketches on the overlap-sensitive instances that arise in sparse data and dataset search (Bessa et al., 2023). This is a direct example of a task in which nonlinear sketching can exploit structure that worst-case linear projections cannot.

6. Applications, limits, and open directions

Linear sketching has produced fast algorithms in domains far removed from least squares. In graph algorithms, a survey-level picture treats graph sparsification as a sketching problem on incidence or Laplacian representations, with subspace-embedding ideas appearing in spectral sparsification and cut preservation (Woodruff, 2014). For weighted graphs, incidence sketches permit $Sx_t$8-cut sparsification using $Sx_t$9 measurements and $A \mapsto SA$00-spectral sparsification using $A \mapsto SA$01 measurements, while the latter is accompanied by an $A \mapsto SA$02 lower bound for some $A \mapsto SA$03-spectral sparsifier within the incidence-sketch model (Chen et al., 2022). The same paper shows optimality, up to $A \mapsto SA$04, for spanner computation in bounded-weight-ratio graphs under incidence sketches.

In combinatorial optimization, the modular subset sum problem admits a near-linear $A \mapsto SA$05 algorithm by simulating the textbook dynamic program with linear sketches that identify the sparse differences between successive reachable-sum sets. The sketching device is a two-coordinate linear map over interval indicators after a random modular permutation, sufficient to isolate and recover changed coordinates of the DP state (Axiotis et al., 2018). This is one of the clearest examples of sketching used not as a one-shot numerical primitive but as a mechanism for exact sparse recovery inside an offline dynamic program.

In federated decision-making, sketching is also used to compress second-order information rather than raw frequencies. FSCLB maintains local and server-side covariance sketches via SCFD, reducing per-round determinant computation from $A \mapsto SA$06 to $A \mapsto SA$07 and communication from $A \mapsto SA$08 to $A \mapsto SA$09, with regret

$A \mapsto SA$10

which simplifies to $A \mapsto SA$11 when the sketch size exceeds the covariance rank (Yang et al., 1 May 2026). This illustrates a recurring modern theme: matrix sketching is increasingly embedded inside online, distributed, and asynchronous decision pipelines.

The limits of linear sketching are equally central. Oblivious $A \mapsto SA$12-subspace embeddings require $A \mapsto SA$13 rows, spectral-norm approximation admits no nontrivial compression by linear sketches in the strongest model, adaptive-query reuse of a fixed sketch can fail even when the guarantee is valid for every fixed query, and several equivalence results for streaming still require long streams or modular-update restrictions rather than fully general turnstile models (Woodruff, 2014). For robust regression, the paper on oblivious sketching for $A \mapsto SA$14 and logistic regression shows that almost-linear dimension $A \mapsto SA$15 for $A \mapsto SA$16 and $A \mapsto SA$17 for logistic regression suffices for constant-factor approximation, but the $A \mapsto SA$18 tradeoff grows to $A \mapsto SA$19 or $A \mapsto SA$20, and the regularized logistic variant incurs an unavoidable $A \mapsto SA$21 factor for the CountMin-based sketch family (Munteanu et al., 2023).

Taken together, these developments portray linear sketching as a unifying but nonuniform paradigm. It is optimal in some communication and streaming models, nearly optimal for several classes of regression and linear-system solvers, and flexible enough to support graph sparsification, subset-sum acceleration, and federated bandits. Yet its performance is inseparable from the geometry of the target problem: null spaces, spectral tails, overlap structure, incidence locality, and adversarial adaptivity all determine whether a small linear sketch is the right abstraction or an intrinsically lossy one (Hosseini et al., 2018).