Attention Bottleneck Theorem

• The paper formalizes two attention bottleneck frameworks—ACI, with a formal converse on posterior uncertainty reduction, and NVIB, which limits Transformer latent capacity.
• It introduces the JaKoB scaling law, showing that throughput is composed of a baseline linear verification term and a nonlinear √(JKB) information-leverage term.
• Both frameworks leverage KL penalties to enforce bottlenecks, revealing how attention acts as a scarce resource in screening protocols and Transformer cross-attention.

The expression Attention Bottleneck Theorem refers to two distinct but related information-theoretic constructions. In Attention-Constrained Inference (ACI), the theorem is a converse bound on how much posterior uncertainty can be reduced when many records can be screened cheaply but only a small subset can be verified deeply; the associated JaKoB scaling law states that throughput decomposes into a linear baseline and a nonlinear information-leverage term of order $\sqrt{J K B}$ (You, 9 Feb 2026). In nonparametric variational information bottleneck (NVIB) for Transformer encoder–decoder models, the paper does not name a formal “Attention Bottleneck Theorem,” but it gives the formal ingredients for an implied theorem: the NVIB objective induces a bottleneck on both the effective number of attention-accessible vectors and the information content of each vector (Henderson et al., 2022).

1. Two technical meanings of the bottleneck

Both uses of the term concern attention as a scarce inferential resource, but they operate at different levels of abstraction. In ACI, attention is a decoder-side budget constraint: $K$ records can be inspected at low cost, while at most $B$ can be verified. In NVIB, attention is a representation-capacity constraint inside a Transformer: the latent passed through cross-attention is regularized by a KL penalty that limits both cardinality and content.

Setting	Bottlenecked object	Central quantity
ACI	Verified records under budget $B$	Epistemic throughput $T(K,B)$
NVIB	Attention-accessible latent mixture $F$	$\mathbb{E}_x KL(q(F\mid x)\|p(F))$

This distinction matters because the ACI result is a formal theorem with converse and achievability statements, whereas the NVIB result is an implied theorem extracted from the objective, posterior family, and KL decomposition. A common misconception is to treat them as the same theorem about Transformer attention; the data instead support a narrower conclusion: they are parallel bottleneck formalisms with different state spaces, losses, and proof mechanisms (You, 9 Feb 2026, Henderson et al., 2022).

2. ACI: formal model, loss, and epistemic throughput

In ACI, one decision window contains two stages: screening and verification. Screening inspects $K$ records and computes a statistic $Z$ for each record; verification can follow up on at most $B$ of them, with $K$0. Each record has a binary latent type $K$1, where $K$2 denotes an informative record, and the prevalence is

$K$3

The Bayes-optimal screening score is

$K$4

Screening quality is measured by the mutual information

$K$5

which, for binary $K$6, can be written as

$K$7

In the local weak-screening logit model,

$K$8

with $K$9 small, $B$0, $B$1, $B$2, and continuous $B$3, Taylor expansion yields

$B$4

Verification reveals $B$5. If $B$6, then $B$7 carries information about the target $B$8; if $B$9, then $B$0 is independent of $B$1. The per-informative-record verification information is

$B$2

Under Bayes log-loss, posterior uncertainty governs population Bayes risk. In the window-level specialization with public artifacts, the Bayes-optimal expected log-loss after observing all screening outputs and published verification transcripts is

$B$3

where $B$4 if record $B$5 is not verified and $B$6 if it is verified. The paper defines epistemic throughput as

$B$7

Because screening is independent of $B$8, $B$9, hence

$T(K,B)$0

This gives the theorem its operative meaning: the bottleneck is not computational cost in general, but the entropy reduction obtainable when verification is the scarce stage (You, 9 Feb 2026).

3. The Attention Bottleneck Theorem and the JaKoB scaling law

The key ingredient is the selection enrichment lemma. For any selection rule $T(K,B)$1 based on $T(K,B)$2 with selection rate $T(K,B)$3,

$T(K,B)$4

The lemma is a data-processing/Pinsker-based bound on the maximum enrichment in informative records obtainable from screening quality $T(K,B)$5 when only an $T(K,B)$6 fraction can be verified.

The resulting Attention Bottleneck Theorem states that, under the paper’s assumptions, any policy that inspects $T(K,B)$7 records and verifies at most $T(K,B)$8 satisfies

$T(K,B)$9

The first term is the baseline contribution of random verification: on average, $F$0 of the $F$1 verified records are informative, and each contributes $F$2. The second term is the information-leverage term arising from screening-driven enrichment; it scales as $F$3 with explicit universal constant $F$4.

The theorem is called a JaKoB scaling law because throughput has a linear component in verification and prevalence and a square-root component in screening quality, screening breadth, and verification budget. The mechanism is explicit in the proof intuition: $F$5 Multiplying by $F$6 yields the converse bound.

A plausible implication is that the theorem formalizes a specifically decoder-side bottleneck. Cheap candidate generation, retrieval, or triage can scale aggressively, but the ceiling on reliable posterior update is controlled by the interaction of $F$7, $F$8, and $F$9, not by $\mathbb{E}_x KL(q(F\mid x)\|p(F))$0 alone (You, 9 Feb 2026).

4. Tightness, weak-screening achievability, and tail regimes

The square-root law is not only an upper bound. Under the weak-screening logit model and decoupled claims—independent $\mathbb{E}_x KL(q(F\mid x)\|p(F))$1 per record—the top-$\mathbb{E}_x KL(q(F\mid x)\|p(F))$2-by-score policy achieves

$\mathbb{E}_x KL(q(F\mid x)\|p(F))$3

where

$\mathbb{E}_x KL(q(F\mid x)\|p(F))$4

and $\mathbb{E}_x KL(q(F\mid x)\|p(F))$5 is the $\mathbb{E}_x KL(q(F\mid x)\|p(F))$6-quantile of $\mathbb{E}_x KL(q(F\mid x)\|p(F))$7.

In the joint limit $\mathbb{E}_x KL(q(F\mid x)\|p(F))$8 and $\mathbb{E}_x KL(q(F\mid x)\|p(F))$9 with fixed $K$0 and $K$1, the inner and outer bounds share the same $K$2 rate. This establishes the JaKoB law’s tightness up to constants.

The sparse-verification or haystack regime $K$3 introduces a sharp tail dependence. Rewriting the leverage term,

$K$4

Hence amplification depends on the upper-tail behavior of $K$5 through $K$6.

For heavy-tailed scores, the paper gives a Pareto$K$7 example with

$K$8

and standardized

$K$9

Then

$Z$0

so leverage is polynomial: $Z$1

For light-tailed scores, exemplified by $Z$2,

$Z$3

hence

$Z$4

The paper describes this as a sharp dichotomy: massive screening is highly effective only when the score distribution admits exploitable extremes.

The corresponding near-optimal policy is operationally simple: inspect $Z$5 records, compute screening statistics $Z$6, compute Bayes scores $Z$7 or a calibrated proxy, select the top $Z$8 indices by $Z$9, verify those $B$0 records, and publish $B$1 as artifacts. In the benchmark model, this policy is optimal, and in the weak-screening regime it achieves the JaKoB scaling. The paper also reports a finite-length simulation with $B$2, base rate $B$3, and logistic-score screening; across four screening strengths (AUC $B$4), the empirical throughput of the top-$B$5 policy tracks the exact benchmark boundary and remains below the converse ceiling and finite-pool oracle (You, 9 Feb 2026).

5. NVIB and the implied attention bottleneck in Transformers

The NVIB construction addresses a different problem: how to bottleneck the latent consumed by Transformer cross-attention. The paper formalizes the encoder representation as an exchangeable mixture distribution

$B$6

with a variable number of components $B$7 and exchangeable component indexing. To capture unbounded $B$8 and exchangeability, it uses Dirichlet processes (DPs). Attention is recast as Bayesian denoising: $B$9 and standard attention is recovered when $K$00 is a weighted mixture of impulses supported at encoder vectors.

The NVIB objective is an ELBO-style VIB objective with mixture-valued latent $K$01: $K$02 with

$K$03

Here $K$04 is a bounded factorized DP posterior and $K$05 is a bounded prior or a length-conditional prior. The paper uses

$K$06

to remove unwanted scaling with sequence length $K$07 and dimension $K$08.

The implied theorem has four components. First, the information bottleneck: $K$09 Second, $K$10 bottlenecks the number of effective attention vectors. With bounded FDP prior and posterior sharing component counts, $K$11 penalizes large $K$12 and sharply peaked Dirichlet weights, and it grows approximately linearly in $K$13. Under the DP occupancy analysis,

$K$14

so penalizing $K$15 reduces $K$16. Third, $K$17 bottlenecks per-vector information by penalizing deviations of $K$18 and $K$19 from the Gaussian prior. Fourth, an additivity bound gives

$K$20

ignoring weak dependencies introduced by normalization; this connects cardinality control and per-vector information control.

The paper’s corollaries are explicit. If $K$21, then $K$22 and the decoder receives the prior, so no information passes. If $K$23 grows with input length $K$24 while $K$25 remain fixed, the linear-in-$K$26 form of $K$27 and $K$28 forces $K$29 and per-component KLs down so that total capacity stays bounded per token.

Empirically, on Wikitext-103 subsets with short sentences; one-layer, one-head Transformers; no pretraining; averages over 5 seeds, the model exhibits the stated bottleneck behavior. With $K$30, $K$31, NVAE reaches BLEU $K$32, PPL $K$33, F-PPL $K$34, R-PPL $K$35, while retaining $K$36 of vectors on average. Stronger regularization, $K$37, $K$38, reduces $K$39, with degraded reconstruction but stronger compression. The paper also reports that a model trained on short sentences adapts the number of vectors to longer sentences, matching a hand-coded stride-0.5 baseline in vector usage (Henderson et al., 2022).

6. Connections, edge cases, and limitations

The two bottleneck constructions share an information-theoretic vocabulary, but their primitives differ. In ACI, the converse relies on the data processing inequality and Pinsker’s inequality, and under log-loss throughput equals mutual information. In NVIB, the upper bound is the standard VIB identity

$K$40

This suggests a common pattern: bottlenecks are enforced by converting a latent selection or representation problem into a KL-controlled capacity problem, but the meaning of the latent differs sharply between public verification artifacts and cross-attention mixtures (You, 9 Feb 2026, Henderson et al., 2022).

The ACI framework specifies several edge cases and caveats. If $K$41, then $K$42, top-$K$43 equals all, enrichment vanishes, and throughput is $K$44 capped by $K$45. If $K$46, the baseline vanishes, and the leverage term can dominate when $K$47 is large. The converse is robust to correlated records or global $K$48 because it uses only channel-level information and data processing. Miscalibration or adversarial manipulation reduces true $K$49 and may bias $K$50, diminishing leverage; the paper lists robust calibration, adversarial training, provenance checks as defenses.

The NVIB framework lists a different set of limitations. Training stability is complicated by two noise sources—Dirichlet/Gamma weights and Gaussian vectors—and the two KL parts are not trivially decoupled. The method uses bounded DPs with $K$51 tied to input length, approximate KL when only $K$52 is known, and Gamma reparameterization via blended approximations. The reported experiments are single-head and are presented as proof-of-concept on smaller partitions; broader tasks and human evaluation are identified as future work.

A final misconception is that “attention bottleneck” necessarily denotes a single scaling law. The data support a more precise conclusion. In ACI, the theorem is a theorem about epistemic throughput under scarce verification, with baseline

$K$53

and nonlinear leverage

$K$54

In NVIB, the bottleneck is about latent capacity in cross-attention, implemented by KL terms that suppress both effective cardinality and per-vector information. The shared term “attention bottleneck” therefore names a family of formal constraints on information flow rather than a single theorem with a single proof.