Sketched Bilinear Forms: Uniform Analysis

• Sketched bilinear forms are randomized bilinear expressions that approximate inner products by compressing vectors or matrices via sketching operators.
• The analysis employs advanced tools such as generic chaining, decoupling, and a double-tree method to achieve uniform control over deviations across paired sets.
• Applications in federated learning and contextual bandits leverage these geometric bounds, reducing dependence on ambient dimensions by relating performance to Gaussian widths and γ2-functionals.

Searching arXiv for papers on sketched bilinear forms and related randomized sketching bounds. Sketched bilinear forms are randomized bilinear expressions in which vectors or matrices are first compressed by sketching operators and then paired, with the aim of approximating an original bilinear quantity such as an inner product. In the formulation developed in "Beyond Johnson-Lindenstrauss: Uniform Bounds for Sketched Bilinear Forms" (Deb et al., 26 Sep 2025), the central object is the discrepancy between a sketched cross-inner-product and its unsketched counterpart, analyzed uniformly over pairs of sets rather than pointwise. The topic sits at the intersection of randomized linear algebra, high-dimensional probability, and statistical learning theory, because uniform control of such bilinear deviations underlies Johnson–Lindenstrauss guarantees, RIP-type statements, randomized sketching, and approximate linear algebra, while also appearing in sketched Federated Learning and contextual bandits (Deb et al., 26 Sep 2025).

1. Definition and scope

In the vector setting, let $U,V\subseteq \mathbb{R}^d$, and let a sketching operator be a random matrix $S\in\mathbb{R}^{b\times d}$, typically with i.i.d. sub-Gaussian rows satisfying $\mathbb{E}[S^\top S]=I_d$. The sketched bilinear form between $u\in U$ and $v\in V$ is

$(Su)^\top(Sv)\simeq u^\top v.$

This formulation generalizes the more familiar sketched norm or inner-product setting by emphasizing uniform control over two potentially distinct sets $U$ and $V$ rather than a single set or a single vector pair (Deb et al., 26 Sep 2025).

The paper also develops a matrix-indexed formulation. Let $\mathcal{M}\subseteq \mathbb{R}^{m\times n}$, $\mathcal{N}\subseteq \mathbb{R}^{n\times m}$, and let $S\in\mathbb{R}^{b\times d}$0 be a mean-zero, $S\in\mathbb{R}^{b\times d}$1-sub-Gaussian random vector. The associated random cross-inner-product process is

$S\in\mathbb{R}^{b\times d}$2

This matrix form subsumes vector sketches as a special case and is the basis for the paper’s uniform deviation theory (Deb et al., 26 Sep 2025).

A further extension treats a sum of $S\in\mathbb{R}^{b\times d}$3 independent sketches. If $S\in\mathbb{R}^{b\times d}$4 are i.i.d. copies of $S\in\mathbb{R}^{b\times d}$5, the quantity of interest becomes

$S\in\mathbb{R}^{b\times d}$6

This corresponds to settings in which independent sketching matrices are accumulated across iterations or rounds (Deb et al., 26 Sep 2025).

2. Geometric framework and complexity parameters

The analysis is expressed in terms of geometric complexities of the sets being sketched. For a set $S\in\mathbb{R}^{b\times d}$7 of matrices, the relevant radii are

$S\in\mathbb{R}^{b\times d}$8

The framework also uses Talagrand’s $S\in\mathbb{R}^{b\times d}$9-functional with respect to the operator-norm metric,

$\mathbb{E}[S^\top S]=I_d$0

with the infimum taken over admissible $\mathbb{E}[S^\top S]=I_d$1-coverings $\mathbb{E}[S^\top S]=I_d$2. Gaussian width $\mathbb{E}[S^\top S]=I_d$3 is also used throughout (Deb et al., 26 Sep 2025).

These quantities are not auxiliary technicalities but the basic parameters governing the deviation bounds. The stated motivation is that many existing uniform results for sketched inner products either do not apply or are not sharp on general sets, whereas the new framework derives uniform bounds in terms of geometric complexities of the associated sets (Deb et al., 26 Sep 2025). This suggests a shift from ambient-dimension control to geometry-sensitive control.

3. Uniform deviation theorem

The central result is Theorem 3.1 of (Deb et al., 26 Sep 2025), which gives a uniform bound for cross-inner-products. Under the assumptions that $\mathbb{E}[S^\top S]=I_d$4, $\mathbb{E}[S^\top S]=I_d$5, and $\mathbb{E}[S^\top S]=I_d$6 satisfies Assumption 2.1, define

$\mathbb{E}[S^\top S]=I_d$7

$\mathbb{E}[S^\top S]=I_d$8

and

$\mathbb{E}[S^\top S]=I_d$9

Then for all $u\in U$0,

$u\in U$1

where $u\in U$2 depend only on $u\in U$3 (Deb et al., 26 Sep 2025).

For vector sets $u\in U$4, the theorem is recovered by taking

$u\in U$5

In that specialization one obtains a high-probability bound of the form

$u\in U$6

as described in Propositions 5.1–5.2 (Deb et al., 26 Sep 2025).

The significance of this theorem is that it furnishes a unified supremum bound over pairs of structured sets. The paper explicitly states that this unified analysis recovers known results such as the J-L lemma as special cases, while extending RIP-type guarantees (Deb et al., 26 Sep 2025).

4. Analytical mechanisms: decomposition, decoupling, and chaining over pairs

The proof strategy combines several tools from high-dimensional probability. First, the quadratic fluctuation is decomposed as

$u\in U$7

where

$u\in U$8

This separates the off-diagonal chaos from the diagonal fluctuation (Deb et al., 26 Sep 2025).

A classical decoupling argument, stated as Lemma 3.2, controls the off-diagonal term: $u\in U$9 with $v\in V$0 an independent copy of $v\in V$1 (Deb et al., 26 Sep 2025). The diagonal term is treated through a new decoupling for Gaussian chaos, identified in the summary as Theorem 3.6 (Deb et al., 26 Sep 2025).

A distinctive feature is Theorem 3.4, described as “Double-Tree.” If a process $v\in V$2 is Lipschitz in each argument with probability $v\in V$3, then

$v\in V$4

This “double-chaining” enables separate $v\in V$5-control over the two indexing sets and is the mechanism by which the suprema over $v\in V$6 are reduced to geometric quantities attached to the individual sets (Deb et al., 26 Sep 2025).

A plausible implication is that sketched bilinear forms require genuinely two-sided complexity control: unlike one-set norm preservation, the indexing structure is intrinsically bipartite, and the analysis reflects that through separate chaining contributions for the two argument classes.

5. Multiple sketches and scaling with $v\in V$7

The extension to $v\in V$8 independent sketches appears as Theorem 4.1. Under the same setup, if $v\in V$9 are i.i.d. as $(Su)^\top(Sv)\simeq u^\top v.$0, then for every $(Su)^\top(Sv)\simeq u^\top v.$1,

$(Su)^\top(Sv)\simeq u^\top v.$2

The stated conclusion is that the main deviation grows only like $(Su)^\top(Sv)\simeq u^\top v.$3, matching a Hoeffding argument but now in the uniform supremum norm (Deb et al., 26 Sep 2025).

This $(Su)^\top(Sv)\simeq u^\top v.$4-behavior is one of the main structural claims of the paper. It is also the bridge to iterative or online procedures, because many modern analyses involve sketch matrices redrawn or reused over repeated rounds. The abstract explicitly states that the framework extends to the setting where the bilinear form involves a sum of $(Su)^\top(Sv)\simeq u^\top v.$5 independent sketching matrices and shows that the deviation scales as $(Su)^\top(Sv)\simeq u^\top v.$6 (Deb et al., 26 Sep 2025).

The same section gives two specializations:

Specialization	Statement
Johnson–Lindenstrauss (Prop 5.1)	for a finite set $(Su)^\top(Sv)\simeq u^\top v.$7 one gets $(Su)^\top(Sv)\simeq u^\top v.$8
RIP-type inner-product preservation (Prop 5.2)	for arbitrary $(Su)^\top(Sv)\simeq u^\top v.$9 of Gaussian widths $U$0, $U$1 suffices to ensure $U$2

These recoveries clarify the paper’s title. The work is positioned as going beyond Johnson–Lindenstrauss because the controlling parameter can be the geometric complexity of the sets rather than merely cardinality or ambient dimension (Deb et al., 26 Sep 2025).

6. Applications in federated learning and contextual bandits

One application is sketched Federated Learning, discussed in Section 5.3 and Theorem 5.3. The setup is as follows: $U$3 clients each run $U$4 steps of local GD on their loss $U$5, then sketch their total update $U$6 via a shared $U$7, send $U$8 to the server, which averages and broadcasts, and clients apply $U$9 (Deb et al., 26 Sep 2025).

The key analytical obstacle is a cross-term: $V$0 which must be controlled uniformly over a gradient set $V$1 and Hessian-eigenvectors $V$2. By Theorem 4.1 this is bounded by

$V$3

Under the PL condition $V$4, Hessian-spectral assumptions, and bounded gradients, if $V$5, then with probability $V$6,

$V$7

where $V$8 as width $V$9 (Deb et al., 26 Sep 2025). The summary notes that the first term is the vanilla PL-GD rate at rate $\mathcal{M}\subseteq \mathbb{R}^{m\times n}$0, while the second is a “floor” from sketch error $\mathcal{M}\subseteq \mathbb{R}^{m\times n}$1, and that $\mathcal{M}\subseteq \mathbb{R}^{m\times n}$2 depends on $\mathcal{M}\subseteq \mathbb{R}^{m\times n}$3 rather than ambient dimension $\mathcal{M}\subseteq \mathbb{R}^{m\times n}$4 (Deb et al., 26 Sep 2025).

A second application is sketched contextual bandits, in Section 6 and Theorem 6.4. The algorithm is “Sketched LinUCB” (Algorithm 1): at each round $\mathcal{M}\subseteq \mathbb{R}^{m\times n}$5, the learner draws a new sketch matrix $\mathcal{M}\subseteq \mathbb{R}^{m\times n}$6, solves least-squares in $\mathcal{M}\subseteq \mathbb{R}^{m\times n}$7-dimensional sketched space to get $\mathcal{M}\subseteq \mathbb{R}^{m\times n}$8, constructs a UCB confidence set in $\mathcal{M}\subseteq \mathbb{R}^{m\times n}$9, picks an optimistic sketched action $\mathcal{N}\subseteq \mathbb{R}^{n\times m}$0, and then plays the de-sketched action $\mathcal{N}\subseteq \mathbb{R}^{n\times m}$1 (Deb et al., 26 Sep 2025).

The regret decomposition is

$\mathcal{N}\subseteq \mathbb{R}^{n\times m}$2

with

$\mathcal{N}\subseteq \mathbb{R}^{n\times m}$3

as the sketched-space LinUCB regret in $\mathcal{N}\subseteq \mathbb{R}^{n\times m}$4, and

$\mathcal{N}\subseteq \mathbb{R}^{n\times m}$5

as a bias due to sketching. By Theorem 4.1 with $\mathcal{N}\subseteq \mathbb{R}^{n\times m}$6 and $\mathcal{N}\subseteq \mathbb{R}^{n\times m}$7, the bias term is controlled as

$\mathcal{N}\subseteq \mathbb{R}^{n\times m}$8

Choosing $\mathcal{N}\subseteq \mathbb{R}^{n\times m}$9 to balance $S\in\mathbb{R}^{b\times d}$00 and $S\in\mathbb{R}^{b\times d}$01 yields a regret bound of order

$S\in\mathbb{R}^{b\times d}$02

which, for suitably chosen $S\in\mathbb{R}^{b\times d}$03, can be smaller than the usual $S\in\mathbb{R}^{b\times d}$04. The paper further states that when $S\in\mathbb{R}^{b\times d}$05 have small Gaussian widths the regret depends on $S\in\mathbb{R}^{b\times d}$06 rather than $S\in\mathbb{R}^{b\times d}$07 (Deb et al., 26 Sep 2025).

7. Conceptual position and relation to adjacent results

The paper places sketched bilinear forms within a broader landscape that includes the Johnson–Lindenstrauss lemma, the Restricted Isometry Property, randomized sketching, and approximate linear algebra (Deb et al., 26 Sep 2025). Its explicit claim is that existing uniform bounds for sketched inner products of vectors or matrices underpin these areas, but many modern analyses involve sketched bilinear forms for which those bounds either do not apply or are not sharp on general sets (Deb et al., 26 Sep 2025).

Two clarifications follow from the framework. First, sketched bilinear forms are not limited to symmetric self-pairings such as $S\in\mathbb{R}^{b\times d}$08; they are fundamentally cross terms, indexed by two sets and often arising when distinct geometric objects interact, as in gradients paired with Hessian-eigenvectors or parameters paired with action sets. Second, the relevant sample or sketch dimension is not described solely by ambient dimension. The paper repeatedly emphasizes dependence on $S\in\mathbb{R}^{b\times d}$09-functionals or Gaussian widths, and, in applications, on $S\in\mathbb{R}^{b\times d}$10 or $S\in\mathbb{R}^{b\times d}$11, rather than on $S\in\mathbb{R}^{b\times d}$12 or $S\in\mathbb{R}^{b\times d}$13 alone (Deb et al., 26 Sep 2025).

This suggests a broader interpretation of the topic: sketched bilinear forms provide a probabilistic interface between compression and interaction structure. In that interpretation, the essential novelty is not sketching by itself but the uniform analysis of pair-indexed cross terms via generic chaining, decoupling, and geometric complexity measures.