Support-Basis Decomposition Overview

• Support-Basis Decomposition is a method that defines global objects via components with explicit support and basis roles, enabling clear geometric and combinatorial interpretations.
• It organizes positive spanning sets through simplicial and conical decompositions, precisely characterizing minimal positive bases and maximal negatively independent subsets.
• Applications range from efficient sparse–dense attention approximations and lossless multi-head attention reformulations to locally supported basis decompositions in neural field models.

Support-basis decomposition denotes a family of decomposition schemes in which a global object is represented through components that have explicit support and basis-like roles. In the foundational geometric formulation, it concerns the decomposition of a finite set $X\subset \mathbb R^d$ that positively spans $\mathbb R^d$ into minimal positive bases and maximal negatively independent subsets (Schoch, 2020). In later literature, closely related terminology is used for efficient attention algorithms, sparse additive decompositions after orthogonal basis transforms, decomposition of probability marginals via admissible support candidates, and neural architectures built from locally supported basis contributions (Aliakbarpour et al., 2 Oct 2025, Zhao, 2 Oct 2025, Ba et al., 2024, Matuschke, 2023, Moseley et al., 2021, Kumar et al., 24 May 2026).

1. Positively spanning sets and simplicial decomposition

For a finite set $X\subset \mathbb R^d$, the linear span and positive span are

$$\ell(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\in\mathbb R\Bigr\},\qquad \P(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\ge 0\Bigr\}.$$

A positive spanning set for a subspace $L\subset \mathbb R^d$ is a set $X$ with $\P(X)=L$. A positive basis is a positive spanning set that is inclusion-minimal. A simplex is a subset $S\subset X$ such that $0\in \P(S)$ and $\P(T)\subsetneq \P(S)$ for every proper subset $\mathbb R^d$0; equivalently, it is a minimal positive basis. Schoch denotes the family of simplices by $\mathbb R^d$1 and studies when a positively spanning set can be reconstructed from such components (Schoch, 2020).

The central structural condition is the factorisation condition,

$\mathbb R^d$2

where $\mathbb R^d$3. The main characterization states that $\mathbb R^d$4 is a positive basis of $\mathbb R^d$5 if and only if it is the union of at most $\mathbb R^d$6 simplices and satisfies the factorisation condition. In that case,

$\mathbb R^d$7

for some linear basis $\mathbb R^d$8, with each $\mathbb R^d$9 for a unique $X\subset \mathbb R^d$0, and the sets $X\subset \mathbb R^d$1 are pairwise disjoint. Each simplex is then of the form $X\subset \mathbb R^d$2 (Schoch, 2020).

This yields the standard cardinality bounds for positive bases in $X\subset \mathbb R^d$3: $X\subset \mathbb R^d$4 Equality $X\subset \mathbb R^d$5 occurs if and only if $X\subset \mathbb R^d$6 is a cross, namely a union of $X\subset \mathbb R^d$7 $X\subset \mathbb R^d$8-simplices derived from a linear basis, exemplified by $X\subset \mathbb R^d$9. In this setting, support-basis decomposition is not an approximation scheme but an exact classification of positive bases by their simplex structure.

2. Subbases, Boolean lattices, and conical decomposition

The same geometric framework has a second layer built from negatively independent subsets. A set $$\ell(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\in\mathbb R\Bigr\},\qquad \P(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\ge 0\Bigr\}.$$0 is negatively independent if $$\ell(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\in\mathbb R\Bigr\},\qquad \P(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\ge 0\Bigr\}.$$1 for every $$\ell(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\in\mathbb R\Bigr\},\qquad \P(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\ge 0\Bigr\}.$$2; equivalently, $$\ell(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\in\mathbb R\Bigr\},\qquad \P(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\ge 0\Bigr\}.$$3 is pointed. Maximal negatively independent subsets are collected in

$$\ell(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\in\mathbb R\Bigr\},\qquad \P(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\ge 0\Bigr\}.$$4

These subsets are the maximal pointed-cone frames carried by $$\ell(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\in\mathbb R\Bigr\},\qquad \P(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\ge 0\Bigr\}.$$5 (Schoch, 2020).

For any positively spanning set $$\ell(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\in\mathbb R\Bigr\},\qquad \P(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\ge 0\Bigr\}.$$6, the subbases

$$\ell(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\in\mathbb R\Bigr\},\qquad \P(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\ge 0\Bigr\}.$$7

form a Boolean lattice under inclusion. Moreover, the map $$\ell(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\in\mathbb R\Bigr\},\qquad \P(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\ge 0\Bigr\}.$$8 embeds $$\ell(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\in\mathbb R\Bigr\},\qquad \P(X)=\Bigl\{\sum_i a_i x_i:x_i\in X,\ a_i\ge 0\Bigr\}.$$9 into the Boolean algebra $L\subset \mathbb R^d$0, and if $L\subset \mathbb R^d$1 is a positive basis then this embedding is onto, so $L\subset \mathbb R^d$2. This identifies the combinatorics of positively spanning subcollections with the combinatorics of simplex families. A plausible implication is that the factorisation condition does not merely constrain intersections of spans; it organizes the full subbasis structure into a distributive combinatorial object.

Schoch’s second main theorem is conical rather than simplicial: any finite $L\subset \mathbb R^d$3 can be written as the union of at most $L\subset \mathbb R^d$4 maximal sets spanning pointed cones. If $L\subset \mathbb R^d$5 is a positive basis, these sets are tantamount to frames of the cones. The bound is sharp if and only if $L\subset \mathbb R^d$6 is a cross. More generally, there can be at most $L\subset \mathbb R^d$7 maximal subsets of $L\subset \mathbb R^d$8 spanning pointed cones when intersections of two of them do not span a set of full dimension (Schoch, 2020).

Low-dimensional examples make the extremal cases explicit. In $L\subset \mathbb R^d$9, a triangle with $X$0 is a single simplex, whereas the planar cross $X$1 attains $X$2 and $X$3. In $X$4, a tetrahedron with $X$5 is again a single simplex, while the standard cross $X$6 attains $X$7. These examples separate the simplex count, which is at most $X$8, from the pointed-cone count, which is at most $X$9.

3. Sparse–dense support decomposition in softmax attention

A distinct usage appears in efficient attention, where support-basis decomposition is introduced to approximate softmax attention beyond the bounded-entry assumption. The framework assumes that the entries of $\P(X)=L$0 are independent centered sub-Gaussian random variables with variance proxies $\P(X)=L$1, and uses this tail behavior to split each matrix into large and small entries at a threshold $\P(X)=L$2 (Aliakbarpour et al., 2 Oct 2025).

With

$\P(X)=L$3

the method defines the “support basis” of $\P(X)=L$4 by extracting the entries activated by $\P(X)=L$5: $\P(X)=L$6 The matrices $\P(X)=L$7 and $\P(X)=L$8 are disjoint and sum to $\P(X)=L$9. Under the stated tail bounds, $S\subset X$0 and $S\subset X$1 with high probability. The sparse part is handled exactly, while the dense bounded part is approximated by a degree-$S\subset X$2 Chebyshev polynomial $S\subset X$3 on $S\subset X$4, giving a low-rank factorization $S\subset X$5. Polynomial-kernel sketching with Count-Sketch or OSNAP reduces the cost of applying the approximation (Aliakbarpour et al., 2 Oct 2025).

The single-threshold algorithm builds $S\subset X$6 explicitly on its sparse support, constructs $S\subset X$7 so that $S\subset X$8, forms

$S\subset X$9

computes normalization terms from the row sums, and returns

$0\in \P(S)$0

which approximates $0\in \P(S)$1. The stated runtime is $0\in \P(S)$2 for the sparse construction plus $0\in \P(S)$3 for the Chebyshev factorization and $0\in \P(S)$4 for sketching, yielding overall sub-quadratic time $0\in \P(S)$5 for some $0\in \P(S)$6. The accuracy theorem states that if $0\in \P(S)$7 have at most $0\in \P(S)$8 entries exceeding $0\in \P(S)$9, then for any $\P(T)\subsetneq \P(S)$0 the algorithm returns $\P(T)\subsetneq \P(S)$1 in $\P(T)\subsetneq \P(S)$2 time such that

$\P(T)\subsetneq \P(S)$3

The multi-threshold extension removes all distributional assumptions by partitioning entry magnitudes into exponentially growing buckets $\P(T)\subsetneq \P(S)$4, producing $\P(T)\subsetneq \P(S)$5 disjoint blocks $\P(T)\subsetneq \P(S)$6. The support-basis and disjointness guarantee

$\P(T)\subsetneq \P(S)$7

The paper further states that combining these blockwise polynomial attentions gives a better global fit in $\P(T)\subsetneq \P(S)$8 norms, for $\P(T)\subsetneq \P(S)$9, than any single polynomial. Empirically, on $\mathbb R^d$00 Gaussian inputs, the method becomes faster than exact attention once $\mathbb R^d$01, and on LLaDA-8B-Instruct its degree-6 approximation is reported at approximately $\mathbb R^d$02 average accuracy versus approximately $\mathbb R^d$03 for exact attention, while the Alman–Song baseline fails at $\mathbb R^d$04 (Aliakbarpour et al., 2 Oct 2025).

4. Basis decomposition as an exact reformulation of multi-head attention

A second attention-related usage, presented as Basis Decomposition (BD), is exact rather than approximate. Its starting point is the matrix identity that if $\mathbb R^d$05 has rank $\mathbb R^d$06, then after selecting $\mathbb R^d$07 linearly independent rows to form a basis matrix $\mathbb R^d$08, the remaining rows are reconstructed by unique coefficients collected in $\mathbb R^d$09, and

$\mathbb R^d$10

in “row-first” form, or $\mathbb R^d$11 in “row-last” form (Zhao, 2 Oct 2025).

In multi-head attention, each head has matrices $\mathbb R^d$12, and the products $\mathbb R^d$13 and $\mathbb R^d$14 have rank at most $\mathbb R^d$15. Applying BD with $\mathbb R^d$16 yields per-head basis matrices and support coefficients: $\mathbb R^d$17 These are stacked across heads into shared $\mathbb R^d$18 and $\mathbb R^d$19. The corresponding transformed projections satisfy

$\mathbb R^d$20

so the scaled dot products, softmax weights, and reconstructed outputs are preserved exactly. The method is therefore described as the first lossless algorithmic reformulation of attention (Zhao, 2 Oct 2025).

The algorithm has an offline preparation stage and an inference stage. Preparation computes two BD decompositions for each $\mathbb R^d$21 and $\mathbb R^d$22, chooses the orientation with smaller average residual, and stores the resulting basis and coefficient matrices. Inference computes $\mathbb R^d$23, reconstructs $\mathbb R^d$24 and $\mathbb R^d$25 through the coefficient matrices, applies standard scaled-dot-product attention headwise, concatenates the outputs, and multiplies by $\mathbb R^d$26 to produce the final output. The stated savings factor in key/value projection FLOPs and parameters is

$\mathbb R^d$27

For $\mathbb R^d$28 and $\mathbb R^d$29, the reduction is $\mathbb R^d$30. On DeepSeek-V2-Lite (16B, FP16), the reported figures are 4 seconds of offline preparation, $\mathbb R^d$31 faster key/value projections in FP16, $\mathbb R^d$32 in BF16, $\mathbb R^d$33 smaller KV weights, and end-to-end perplexity changes of $\mathbb R^d$34 in FP32 and $\mathbb R^d$35 in FP16 (Zhao, 2 Oct 2025).

A common misconception is to treat the two attention papers as variants of the same algorithm. They are not. In the support-basis attention framework, the decomposition is sparse–dense and the result is a provably accurate approximation. In BD Attention, the decomposition is a rank-exact factorization of projection products and the result preserves every pairwise dot product and final projected output exactly.

5. Basis transforms and sparse additive function decompositions

In high-dimensional approximation, support-basis decomposition refers to finding an orthogonal transform under which an ANOVA or anchored decomposition becomes sparse. For a $\mathbb R^d$36 function $\mathbb R^d$37 on a box $\mathbb R^d$38, both ANOVA and anchored representations take the form

$\mathbb R^d$39

with the distinction lying in whether coordinates are integrated out or fixed at an anchor point. Sparsity is characterized through the function graph, whose vertices are $\mathbb R^d$40 and edges are $\mathbb R^d$41. Minimal additive decompositions use only cliques of this graph (Ba et al., 2024).

The key idea is that $\mathbb R^d$42 may not be sparse in the ambient coordinates, but $\mathbb R^d$43 may be sparse for some orthogonal $\mathbb R^d$44. The directional derivatives transform as

$\mathbb R^d$45

so choosing $\mathbb R^d$46 to annihilate most first and second directional derivatives yields a sparse graph $\mathbb R^d$47. The proposed algorithm has three steps. First, gradient samples are assembled into

$\mathbb R^d$48

and an SVD is used to minimize the number of active coordinates. Second, Hessian samples $\mathbb R^d$49 are approximately joint-block-diagonalized by a commutant-based procedure. Third, residual off-block edges are sparsified by minimizing

$\mathbb R^d$50

over $\mathbb R^d$51 by Riemannian gradient descent or the Landing method (Ba et al., 2024).

The resulting transformed representation is typically

$\mathbb R^d$52

with $\mathbb R^d$53 small and $\mathbb R^d$54 containing only the surviving mixed-derivative pairs. Because the ANOVA and anchored terms are minimal, the decomposition into at-most-two-variable summands is unique. The reported numerics include a synthetic $\mathbb R^d$55 example in which step 1 reduces the active dimension to $\mathbb R^d$56, step 2 identifies two connected components of sizes $\mathbb R^d$57 and $\mathbb R^d$58, and step 3 drives off-block entries to $\mathbb R^d$59 within $\mathbb R^d$60 Riemannian-gradient iterations, as well as a noisy $\mathbb R^d$61 case recovering $\mathbb R^d$62 with more than $\mathbb R^d$63 success over $\mathbb R^d$64 trials (Ba et al., 2024).

6. Admissible-support decomposition of probability marginals

In combinatorial optimization, the decomposition problem begins with a finite ground set $\mathbb R^d$65, a family of subsets $\mathbb R^d$66, a requirement function $\mathbb R^d$67, and a marginal vector $\mathbb R^d$68. The goal is to find a distribution $\mathbb R^d$69 on $\mathbb R^d$70 such that

$\mathbb R^d$71

or certify infeasibility. The associated feasible marginal set is

$\mathbb R^d$72

and every $\mathbb R^d$73 satisfies the covering inequalities

$\mathbb R^d$74

(Matuschke, 2023).

The decomposition algorithm is driven by admissible support candidates (ASCs). For residual marginals $\mathbb R^d$75 and residual requirements $\mathbb R^d$76, a set $\mathbb R^d$77 is an ASC if it satisfies three conditions: $\mathbb R^d$78, $\mathbb R^d$79 for every tight requirement set $\mathbb R^d$80, and $\mathbb R^d$81 for every non-dominated $\mathbb R^d$82 with $\mathbb R^d$83. Domination is defined by a partial order $\mathbb R^d$84 if either $\mathbb R^d$85 or

$\mathbb R^d$86

Algorithm 1 repeatedly selects an ASC $\mathbb R^d$87, chooses a peeling weight $\mathbb R^d$88, updates the distribution mass $\mathbb R^d$89, and reduces residual marginals and requirements. The invariants guarantee nonnegativity of residual marginals, preservation of the covering inequalities, and strict progress. Termination occurs after at most $\mathbb R^d$90 iterations, and the overall runtime is $\mathbb R^d$91 (Matuschke, 2023).

When a suitable ASC oracle exists, the framework yields an exact polyhedral characterization: $\mathbb R^d$92 The paper constructs ASCs for supermodular requirements, abstract networks with weak conservation, and Hoffman–Schwartz-type lattice polyhedra. It also characterizes balanced hypergraphs as precisely those systems $\mathbb R^d$93 for which every $\mathbb R^d$94 admits a perfect decomposition achieving

$\mathbb R^d$95

Here the “support” in support-basis decomposition is set-theoretic rather than geometric, and the basis role is played by the selected ASC subsets supporting the distribution.

7. Local support, partition-of-unity structure, and neural field decompositions

In scientific machine learning, the decomposition is spatial and local. Finite Basis PINNs (FBPINNs) represent the approximate solution of a differential equation on $\mathbb R^d$96 as

$\mathbb R^d$97

where $\mathbb R^d$98 is partitioned into overlapping subdomains $\mathbb R^d$99, $X\subset \mathbb R^d$00 is a smooth window with compact or near-compact support, $X\subset \mathbb R^d$01 rescales coordinates to a standard range such as $X\subset \mathbb R^d$02, and $X\subset \mathbb R^d$03 is a constraining operator that enforces boundary conditions exactly (Moseley et al., 2021). A strict partition of unity may be imposed by $X\subset \mathbb R^d$04, but the framework can also sum $X\subset \mathbb R^d$05 directly. The loss is the strong-form residual

$X\subset \mathbb R^d$06

with no boundary-loss term and no interface-matching penalty. The stated motivation is to mitigate spectral bias and reduce optimization stiffness by replacing one large network with many smaller local networks.

Courant adopts a related but attention-based local-support decomposition. It places latent anchors $X\subset \mathbb R^d$07 in the computational domain and computes decoder cross-attention weights

$X\subset \mathbb R^d$08

so that $X\subset \mathbb R^d$09 and $X\subset \mathbb R^d$10 for every head $X\subset \mathbb R^d$11 and point $X\subset \mathbb R^d$12. Because the decoder is affine in the latent values, the predicted field decomposes into local contributions $X\subset \mathbb R^d$13, and the model can be written in basis-expansion form

$X\subset \mathbb R^d$14

with $X\subset \mathbb R^d$15 induced by the attention weights and $X\subset \mathbb R^d$16 determined by the latent state (Kumar et al., 24 May 2026).

The two frameworks differ architecturally—FBPINNs use overlapping subdomain networks and window functions, whereas Courant uses geometry-anchored latent queries, shared random Fourier features, and an affine decoder—but both impose locality structurally rather than through an explicit sparsity regularizer. Courant is trained only with an $X\subset \mathbb R^d$17 prediction loss in physical space, yet its latents are reported to specialize geometrically: in steady 2D cylinder flow, the top-8 largest $X\subset \mathbb R^d$18 fields tile the boundary layer and wake; in transient vortex shedding, some anchors remain pinned to the cylinder wall while others track moving vortices, and the latent trajectory exhibits a PSD peak at the physical shedding frequency $X\subset \mathbb R^d$19 (Kumar et al., 24 May 2026). FBPINNs similarly use local normalization and local support to handle large and multi-scale domains without interface penalties (Moseley et al., 2021).

Across these usages, the term does not denote a single standardized formalism. In positive-span theory it classifies finite vector configurations via simplices and pointed cones; in attention it names either an approximate sparse–dense split or an exact rank factorization; in additive decomposition and marginal decomposition it organizes sparsity through basis transforms or admissible support candidates; and in neural PDE and surrogate models it refers to locally supported basis contributions assembled into a global field. A common pattern is the replacement of a monolithic global object by finitely many components whose supports are explicit and whose recombination is algebraically controlled.