TK-SLT: Top-K Sparse Logits Transmission

• TK-SLT is a method that transmits only the highest K logits and their indices, reducing communication overhead while preserving key signal characteristics.
• It employs deterministic, randomized, or adaptive top-K sparsification to balance compression efficiency with model accuracy.
• TK-SLT is applied in split learning, federated learning, and speculative decoding to enhance inference throughput and reduce data transmission costs.

Searching arXiv for papers on Top-K sparse logits transmission and related top-k sparsification. Top‑K Sparse Logits Transmission (TK‑SLT) is a communication reduction technique in which a sender transmits only the largest $K$ entries of a logit vector together with their indices, rather than the full vector. In the most direct formulation, for logits $z\in\mathbb{R}^C$, deterministic TK‑SLT applies the operator $z \mapsto \tilde z$ with $\tilde z_i = \mathbf{1}_{\{i\in S_k(z)\}} z_i$, where $S_k(z)$ denotes the indices of the top‑$k$ entries by magnitude or value, depending on the application (Zheng et al., 2023). The method has emerged in several adjacent settings: split learning and vertical federated learning, where top‑$k$ sparsification is applied to cut‑layer activations and gradients (Zheng et al., 2023); communication‑efficient distributed learning, where top‑$k$ is analyzed as a compressor with contraction properties (Shi et al., 2019); and distributed speculative decoding for LLMs, where the draft model transmits only top‑$K$ token logits or probabilities to an edge verifier instead of the full vocabulary distribution (Zheng et al., 4 Sep 2025). Across these settings, TK‑SLT is best understood as a sparse transmission operator whose utility depends on the concentration of signal in a small number of coordinates, the communication model, and the downstream algorithm’s tolerance to truncation.

1. Formalization and operator view

In the generic TK‑SLT setting, the transmitted object is a logit vector $z\in\mathbb{R}^C$, where $z\in\mathbb{R}^C$0 is the number of classes, labels, or vocabulary items. The deterministic top‑$z\in\mathbb{R}^C$1 sparsification operator is the same one studied for split learning activations in "Reducing Communication for Split Learning by Randomized Top‑k Sparsification" (Zheng et al., 2023). For a vector $z\in\mathbb{R}^C$2, the index set of the $z\in\mathbb{R}^C$3 largest‑magnitude components is written as $z\in\mathbb{R}^C$4, and the sparsified vector is

$z\in\mathbb{R}^C$5

The paper explicitly notes that if $z\in\mathbb{R}^C$6 is identified with a logits vector, then this is exactly TK‑SLT (Zheng et al., 2023).

A practical transmission format is a pair of arrays containing the selected indices and values. In the split learning formulation, this appears as

$z\in\mathbb{R}^C$7

which maps directly to sparse logits transmission: the sender transmits the selected class or token indices together with their corresponding logits (Zheng et al., 2023). If the receiver reconstructs a dense vector, all omitted coordinates are set to zero before any subsequent computation.

The top‑$z\in\mathbb{R}^C$8 operator also admits a more abstract compressor interpretation. "Understanding Top‑k Sparsification in Distributed Deep Learning" defines $z\in\mathbb{R}^C$9 as the vector that keeps the $z \mapsto \tilde z$0 largest coordinates in absolute value and sets the remainder to zero (Shi et al., 2019). This perspective is important because it separates the algebraic properties of top‑$z \mapsto \tilde z$1 from the semantics of the vector being compressed. The paper observes that the Top‑$z \mapsto \tilde z$2 operator’s geometry and norm‑preservation arguments transfer from gradients to logits, provided the logit distribution has similar concentration properties (Shi et al., 2019). This suggests that TK‑SLT is not a separate mathematical primitive but a direct application of a broader sparsification operator to model outputs.

A further refinement arises when hard top‑$z \mapsto \tilde z$3 selection is treated as an optimization problem over the permutahedron. "Fast, Differentiable and Sparse Top‑k: a Convex Analysis Perspective" formulates the top‑$z \mapsto \tilde z$4 mask as

$z \mapsto \tilde z$5

where $z \mapsto \tilde z$6 is the permutahedron associated with the binary vector containing $z \mapsto \tilde z$7 ones and $z \mapsto \tilde z$8 zeros (Sander et al., 2023). This yields differentiable and sparse relaxations of top‑$z \mapsto \tilde z$9, which are relevant when TK‑SLT itself must be embedded into end‑to‑end trainable systems rather than used only as a non‑differentiable communication primitive.

2. Communication model and compression arithmetic

The communication advantage of TK‑SLT follows from replacing dense transmission of $\tilde z_i = \mathbf{1}_{\{i\in S_k(z)\}} z_i$0 values with sparse transmission of $\tilde z_i = \mathbf{1}_{\{i\in S_k(z)\}} z_i$1 values and $\tilde z_i = \mathbf{1}_{\{i\in S_k(z)\}} z_i$2 indices. In the split learning analysis, letting $\tilde z_i = \mathbf{1}_{\{i\in S_k(z)\}} z_i$3 denote the vector dimension, $\tilde z_i = \mathbf{1}_{\{i\in S_k(z)\}} z_i$4 the number of bits per value, and $\tilde z_i = \mathbf{1}_{\{i\in S_k(z)\}} z_i$5 the number of bits per index, the forward compressed size for deterministic top‑$\tilde z_i = \mathbf{1}_{\{i\in S_k(z)\}} z_i$6 is

$\tilde z_i = \mathbf{1}_{\{i\in S_k(z)\}} z_i$7

while the backward compressed size is $\tilde z_i = \mathbf{1}_{\{i\in S_k(z)\}} z_i$8 if the receiver already knows the index set from the forward pass (Zheng et al., 2023). The same arithmetic applies to TK‑SLT by replacing $\tilde z_i = \mathbf{1}_{\{i\in S_k(z)\}} z_i$9 with the logit dimension $S_k(z)$0: $S_k(z)$1 This formula makes explicit that sparse transmission cost depends not only on the sparsity ratio $S_k(z)$2 but also on the index overhead.

A closely related communication model appears in distributed speculative decoding for LLM inference. In "Communication‑Efficient Collaborative LLM Inference via Distributed Speculative Decoding," full distribution transmission for one token over vocabulary $S_k(z)$3 costs

$S_k(z)$4

and, with draft length $S_k(z)$5, the total uplink payload is

$S_k(z)$6

(Zheng et al., 4 Sep 2025). TK‑SLT replaces full transmission with sparse transmission of the top‑$S_k(z)$7 token raw probabilities and their indices, so the per‑distribution cost becomes

$S_k(z)$8

or approximately $S_k(z)$9 in latency models that neglect index bits relative to value precision (Zheng et al., 4 Sep 2025). The consequence is especially stark for vocabularies of size $k$0, where full FP16 transmission is about $k$1 kbit per token (Zheng et al., 4 Sep 2025).

The same principle extends to token trees in multi‑candidate speculative decoding. "Fast Collaborative Inference via Distributed Speculative Decoding" generalizes sparse transmission to what it terms Truncated Sparse Logits Transmission (TSLT), which includes Top‑$k$2 truncation as a special case (Zheng et al., 18 Dec 2025). There, communication scales with the number of internal node distributions in the token tree times the size of the sparse support, rather than the full vocabulary size at every node. This suggests that for hierarchical or multi‑branch generation algorithms, the advantage of TK‑SLT can compound with the branching structure rather than merely reducing a single softmax vector.

3. Training-time variants: deterministic, randomized, and adaptive top‑k

Deterministic TK‑SLT always selects the same type of subset: the currently largest $k$3 logits. This can create systematic undertraining of rarely selected coordinates. The split learning study addresses this by introducing randomized top‑$k$4 sparsification (RandTopk), which keeps exactly $k$5 non‑zero entries per vector but allows a small probability of sampling from outside the current top‑$k$6 set (Zheng et al., 2023). Given $k$7, $k$8, and $k$9, the per‑element selection probability at each draw is

$k$0

with sampling performed without replacement until $k$1 indices are selected (Zheng et al., 2023). The paper motivates this by noting that certain neurons or logits are almost never selected under deterministic top‑$k$2, and that randomization improves exploration and generalization while preserving exactly the same communication cost.

This randomized variant is directly relevant to TK‑SLT because the operator is unchanged: only the support selection rule differs. In the paper’s interpretation, one can use RandTopk during training and revert to deterministic top‑$k$3 during inference (Zheng et al., 2023). The empirical finding is that RandTopk usually outperforms deterministic top‑$k$4 at the same compressed size and can match or nearly match the uncompressed model even at compression ratios as high as about $k$5 on CIFAR‑100 (Zheng et al., 2023). A plausible implication is that TK‑SLT systems trained with sparse logits should not necessarily use a deterministic support during optimization, even if deterministic transmission is preferred at deployment.

Adaptivity can also be imposed over time rather than within a single vector. "Adaptive Top‑K in SGD for Communication‑Efficient Distributed Learning" treats the number of transmitted coordinates $k$6 as a per‑iteration decision variable under a total communication budget $k$7 (Ruan et al., 2022). The analysis yields a schedule in which larger $k$8 should be used in early and late training and smaller $k$9 in the middle phase, because the convergence upper bound depends on a product of a decreasing term $k$0 and an increasing discount factor (Ruan et al., 2022). Although derived for gradients, the paper explicitly suggests that the same control problem carries over to sparsified logits or related signals (Ruan et al., 2022). This suggests that a fixed $k$1 in TK‑SLT may be suboptimal whenever the value of more accurate sparse outputs changes over the course of training or adaptation.

A third form of adaptivity appears in communication‑aware federated LLM distillation. "Communication‑Aware Knowledge Distillation for Federated LLM Fine‑Tuning over Wireless Networks" defines the client‑side top‑$k$2 budget by the wireless channel capacity

$k$3

leading to

$k$4

where $k$5 is the per‑round transmission time, $k$6 is the allocated channel fraction, and $k$7 is the number of bits per transmitted index‑value pair (Zhang et al., 1 Sep 2025). Here, the sparsity level itself becomes a function of physical‑layer conditions rather than a purely algorithmic hyperparameter.

4. Statistical and optimization properties

The central statistical question for TK‑SLT is how much information is lost by truncating to the top‑$k$8 support. In the context of gradient sparsification, the simplest bound is

$k$9

which follows by comparison to random $K$0-sparsification (Shi et al., 2019). The same paper derives a tighter bound under bell‑shaped coordinate distributions: $K$1 equivalently with contraction coefficient

$K$2

(Shi et al., 2019). The paper explicitly states that if logits satisfy similar distributional assumptions—most entries near zero and the sorted normalized magnitude curve convex and below a line—then the same Top‑$K$3 bound applies to logits (Shi et al., 2019). This gives TK‑SLT a norm‑preservation interpretation: if a logit vector is sharply concentrated, top‑$K$4 preserves most of its $K$5 mass.

In split learning, the relevant argument is framed in terms of feature‑space geometry and margins. The last‑layer softmax is written as

$K$6

with $K$7 as the class margin (Zheng et al., 2023). The paper argues that structural cut‑layer reduction restricts representations to a fixed $K$8-dimensional hypersphere, whereas top‑$K$9 sparsification allows each input to occupy one of $z\in\mathbb{R}^C$0 supports, thereby expanding the effective feature space and increasing the achievable margin (Zheng et al., 2023). The quantitative expressions are heuristic rather than a full convergence theorem, but the intended conclusion is that top‑$z\in\mathbb{R}^C$1 sparsification can generalize better than simply narrowing the representation. This suggests that TK‑SLT should not be conflated with architectural dimensionality reduction: one preserves a combinatorial support space, the other removes it.

For hard top‑$z\in\mathbb{R}^C$2 inside differentiable systems, discontinuity is a separate concern. The convex analysis perspective formulates hard top‑$z\in\mathbb{R}^C$3 as a linear optimization problem over the permutahedron and then smooths it with a $z\in\mathbb{R}^C$4-norm regularizer (Sander et al., 2023). The resulting relaxed operator can be sparse and differentiable, and, for $z\in\mathbb{R}^C$5, supports magnitude‑based selection (Sander et al., 2023). A plausible implication is that if TK‑SLT is to be learned rather than merely applied, differentiable sparse top‑$z\in\mathbb{R}^C$6 relaxations provide a principled way to route gradients through the selection mechanism.

Finally, attention truncation offers a closely related but more specialized theory. "A Mathematical Theory of Top‑$z\in\mathbb{R}^C$7 Sparse Attention via Total Variation Distance" proves that for a softmax distribution $z\in\mathbb{R}^C$8 and its Top‑$z\in\mathbb{R}^C$9 truncation $z\in\mathbb{R}^C$00,

$z\in\mathbb{R}^C$01

so the approximation error is exactly the discarded softmax tail mass (Tzachristas et al., 8 Dec 2025). Although developed for sparse attention rather than logits transmission, the same identity is immediately relevant whenever TK‑SLT transmits logits and the receiver reconstructs a truncated softmax: the information loss is controlled by the omitted probability mass, not only by a coordinate count. This suggests that choosing $z\in\mathbb{R}^C$02 via tail mass thresholds may be more meaningful than choosing it as a fixed proportion of vocabulary size.

5. Applications across split learning, distillation, and distributed inference

TK‑SLT has appeared most explicitly in distributed speculative decoding for LLM inference. In AI‑RAN settings, a small draft model on a device proposes tokens while a large target model at the edge verifies them (Zheng et al., 4 Sep 2025). The bottleneck is uplink transmission of the draft model’s vocabulary distribution at each speculative step. TK‑SLT addresses this by allowing the draft model to apply softmax only to the top‑$z\in\mathbb{R}^C$03 logits and transmit only those token IDs and logits (Zheng et al., 4 Sep 2025). Because speculative decoding correctness depends only on using the proposal distribution actually used for sampling, the final token distribution remains equivalent to standalone LLM decoding, while the acceptance rate changes from $z\in\mathbb{R}^C$04 to

$z\in\mathbb{R}^C$05

where $z\in\mathbb{R}^C$06 is the top‑$z\in\mathbb{R}^C$07 truncated proposal (Zheng et al., 4 Sep 2025). The trade‑off is therefore not output quality but inference throughput.

The same family of systems is further analyzed through TSLT, where Top‑$z\in\mathbb{R}^C$08 is one truncation choice. The acceptance perturbation obeys

$z\in\mathbb{R}^C$09

where $z\in\mathbb{R}^C$10 is the truncated probability mass outside the candidate set (Zheng et al., 18 Dec 2025). This gives a direct operational criterion for TK‑SLT in speculative decoding: if the top‑$z\in\mathbb{R}^C$11 set captures almost all proposal mass, then acceptance rates and thus end‑to‑end latency remain essentially unchanged.

In split learning and vertical federated learning, the analogous object is not always a logit vector but a cut‑layer activation. Nevertheless, the authors explicitly state that top‑$z\in\mathbb{R}^C$12 sparsification of cut‑layer activations is mathematically and algorithmically the same as sparsifying logits, and that all results transfer if the cut layer is identified with the logits vector (Zheng et al., 2023). In that setting, both forward activations and backward gradients can be compressed; one implementation detail is that backward indices need not be resent if they match the forward sparsity pattern (Zheng et al., 2023). This is a practically important observation for any bidirectional TK‑SLT protocol.

Knowledge distillation provides a different perspective. "BiLD: Bi‑directional Logits Difference Loss for LLM Distillation" is not a communication paper but assumes that only top‑$z\in\mathbb{R}^C$13 teacher and student logits are used in the loss (Li et al., 2024). It argues that LLM logits have an extreme long‑tail distribution and that using only top‑$z\in\mathbb{R}^C$14 logits filters long‑tail noise while pairwise logit differences preserve ranking information (Li et al., 2024). Empirically, BiLD with only the top‑8 logits outperforms full‑logit KL in several teacher–student settings (Li et al., 2024). This suggests that, at least for task‑specific distillation, a sparse logit subset may be not merely a bandwidth compromise but a preferable target representation.

A contrary result appears in offline LLM pre‑training distillation. "Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMs" proves that naïve Top‑K KD yields biased gradients because the student learns a scaled version of the teacher’s Top‑K distribution, with zero mass outside the selected support (Anshumann et al., 21 Mar 2025). It shows that Top‑K distillation can be worse than cross‑entropy when $z\in\mathbb{R}^C$15 is small and much more poorly calibrated than full KD, whereas an importance‑sampling‑based sparse method preserves gradients in expectation (Anshumann et al., 21 Mar 2025). This is a genuine controversy around sparse logits: top‑$z\in\mathbb{R}^C$16 can be highly effective when the objective depends on head ranking or communication, yet biased when used as a direct surrogate for the full teacher distribution in KL‑based pre‑training.

OOD detection offers yet another usage pattern. "Out‑of‑Distribution Detection with Adaptive Top‑K Logits Integration" shows that, for ImageNet‑1K classifiers, a small model‑specific subset of top‑ranked logits beyond the maximum improves OOD detection relative to MaxLogit, MSP, and Energy (Shijo et al., 1 Jul 2025). The method ATLI uses the top‑1 standardized logit plus an adaptively chosen subset $z\in\mathbb{R}^C$17 of other top‑rank logits with learned signs: $z\in\mathbb{R}^C$18 (Shijo et al., 1 Jul 2025). The implication for TK‑SLT is that transmitting only a carefully chosen sparse subset of logits may preserve or even enhance uncertainty‑related information, whereas dense inclusion of all logits may inject noise.

6. Limitations, controversies, and design trade‑offs

The main limitation of TK‑SLT is that top‑$z\in\mathbb{R}^C$19 truncation is not uniformly benign across objectives. In speculative decoding, the final LLM distribution is unchanged and only acceptance rates are perturbed (Zheng et al., 4 Sep 2025). In split learning, top‑$z\in\mathbb{R}^C$20 acts on intermediate activations and may improve generalization relative to structural width reduction (Zheng et al., 2023). In offline distillation, however, pure Top‑K KL is biased and removes tail supervision, leading to poor calibration unless $z\in\mathbb{R}^C$21 is very large (Anshumann et al., 21 Mar 2025). These are not contradictory findings; they reflect different roles for the transmitted logits.

A second trade‑off concerns deterministic versus stochastic support selection. Deterministic top‑$z\in\mathbb{R}^C$22 minimizes immediate approximation error for a given vector, but it may starve some coordinates of updates. RandTopk addresses this by introducing a controlled probability of selecting non‑top‑$z\in\mathbb{R}^C$23 entries while keeping exact sparsity fixed (Zheng et al., 2023). This suggests that training‑time TK‑SLT may require stochasticity even if inference‑time transmission remains deterministic.

A third issue is sparsity pattern alignment across senders. Standard Top‑K preserves informative entries but leads to different supports on different nodes, making collective communication costly. "An All‑Reduce Compatible Top‑K Compressor" addresses this in distributed learning by using a shared random projection sketch to align row selections across workers, producing a globally contractive compressor with factor $z\in\mathbb{R}^C$24 and enabling index‑free All‑Reduce (Chen et al., 30 Oct 2025). This work is about gradients, but most of its construction is vector‑agnostic, and it explicitly suggests porting the same mechanism to high‑dimensional logits (Chen et al., 30 Oct 2025). A plausible implication is that large‑scale multi‑sender TK‑SLT systems may benefit from coordinated or sketch‑based support selection rather than independent local Top‑K.

Privacy is another nuanced dimension. Sparsifying logits or gradients does not automatically provide formal privacy. The split learning paper states that RandTopk provides no extra label privacy in SL, and notes that logits sparsification may reduce some information but comes with no formal guarantees (Zheng et al., 2023). In contrast, "Secure Aggregation with Top‑K Sparsification in Decentralized Federated Learning" develops an information‑theoretic secure aggregation protocol for sparse Top‑K vectors using masked values and permuted indices (Tang et al., 9 Jun 2026). Although formulated for gradients, the paper explicitly states that the same mechanism applies to logits: replace the gradient vector $z\in\mathbb{R}^C$25 with a quantized logits vector $z\in\mathbb{R}^C$26, apply Top‑$z\in\mathbb{R}^C$27, and use the same permutation and mask elimination machinery (Tang et al., 9 Jun 2026). This suggests that privacy‑preserving TK‑SLT is possible, but only with additional protocol machinery beyond sparsification itself.

Finally, the choice of $z\in\mathbb{R}^C$28 is rarely universal. In speculative decoding, the best throughput may occur at an intermediate $z\in\mathbb{R}^C$29, such as $z\in\mathbb{R}^C$30 rather than the smallest possible value, because acceptance rate and communication cost must be balanced jointly (Zheng et al., 4 Sep 2025). In OOD detection, performance improves when a small fraction of logits is added to the maximum but degrades when too many logits are included (Shijo et al., 1 Jul 2025). In distillation, top‑8 can suffice for BiLD (Li et al., 2024), whereas Top‑K KL may need hundreds of logits to approach full KD (Anshumann et al., 21 Mar 2025). Thus, $z\in\mathbb{R}^C$31 is not merely a compression knob; it is objective‑ and system‑dependent.

7. Research directions and broader significance

Several research directions follow naturally from the current literature. One is certified or tail‑mass‑controlled selection. The attention truncation theory shows that deterministic logit‑gap bounds and total variation certificates can determine the smallest $z\in\mathbb{R}^C$32 satisfying a target error budget $z\in\mathbb{R}^C$33 using only ordered logits (Tzachristas et al., 8 Dec 2025). This suggests that future TK‑SLT systems could replace fixed $z\in\mathbb{R}^C$34 with per‑example or per‑query certified sparsity.

A second direction is approximate top‑$z\in\mathbb{R}^C$35 selection. "Understanding Top‑k Sparsification in Distributed Deep Learning" proposes Gaussian$z\in\mathbb{R}^C$36, which estimates a threshold for approximate top‑$z\in\mathbb{R}^C$37 using mean and standard deviation under bell‑shaped coordinate distributions (Shi et al., 2019). The paper explicitly notes that if logits exhibit similar patterns, the same procedure can serve as an efficient approximate Top‑K selector for logits (Shi et al., 2019). This is especially relevant when $z\in\mathbb{R}^C$38 is very large and exact top‑$z\in\mathbb{R}^C$39 itself becomes a bottleneck.

A third direction is end‑to‑end differentiable sparse routing. The convex analysis framework of differentiable sparse top‑$z\in\mathbb{R}^C$40 provides exact sparsity together with gradient flow, and has been used for pruning, ViT fine‑tuning, and sparse mixture‑of‑experts routing (Sander et al., 2023). This suggests a path toward learnable TK‑SLT modules whose support selection is optimized jointly with the model rather than hard‑coded.

A fourth direction concerns adaptive and structured aggregation. Communication‑aware federated LLM distillation shows that adaptive Top‑K selection based on channel capacity and dimension‑wise confidence‑weighted sparse aggregation can reduce communication by approximately $z\in\mathbb{R}^C$41 while improving model accuracy relative to simpler baselines (Zhang et al., 1 Sep 2025). This implies that sparse logits transmission should be paired with an aggregation rule that treats missing coordinates as unknown rather than as literal zeros.

Across these developments, TK‑SLT occupies a broader methodological niche than a simple compression trick. It links communication efficiency, sparse approximation, distribution truncation, and support selection in a way that depends critically on the surrounding algorithm. In some settings it is a nearly lossless systems optimization; in others it changes the effective training objective and must be corrected, randomized, or supplemented. The current literature therefore supports a precise but non‑uniform characterization: Top‑K Sparse Logits Transmission is a general sparse communication primitive whose success depends on matching the top‑$z\in\mathbb{R}^C$42 operator, the support selection rule, and the reconstruction semantics to the statistical and systems properties of the task at hand (Zheng et al., 2023, Shi et al., 2019, Zheng et al., 4 Sep 2025).