TokenDance: Token-Centric Motion & LLM Serving

Updated 5 July 2026

TokenDance is a dual-modality framework that uses discrete tokenization via FSQ and Bidirectional Mamba for efficient music-to-dance generation, yielding competitive FID and BAS scores.
In multi-agent LLM serving, TokenDance employs collective KV cache reuse and Diff-Aware Storage to reduce redundancy and boost throughput across synchronized agents.
The paradigm transforms continuous signals into reusable token units, enabling refined control in motion generation and inference systems.

Searching arXiv for papers directly related to "TokenDance" and adjacent token-based motion/serving work. arxiv_search(query="TokenDance", max_results=10, sort_by="relevance") arxiv_search(query="music to dance token motion generation FSQ Mamba", max_results=10, sort_by="relevance") arxiv_search(query="multi-agent LLM serving KV cache sharing All-Gather TokenDance", max_results=10, sort_by="relevance") TokenDance is a term applied in recent arXiv literature to distinct but structurally related token-centric systems. In music-to-dance generation, it denotes a two-stage framework that discretizes both music and dance with Finite Scalar Quantization and generates dance tokens from music tokens with a Local-Global-Local Bidirectional Mamba architecture (Yang et al., 28 Mar 2026). In multi-agent LLM serving, it denotes a KV-cache–centric system that exploits synchronized All-Gather communication for collective KV Cache sharing across agents (Bian et al., 3 Apr 2026). Related papers use the term in a broader “TokenDance” or “TokenDance-like” sense for motion tokenization, music-conditioned pose generation, discrete motion planning, duet choreography, and token-based inference control, indicating a wider research pattern in which continuous signals or capacities are converted into reusable token-level abstractions (Li et al., 2021).

1. Scope, nomenclature, and major usages

The term has at least two established research usages. One is a generative modeling usage centered on music-to-dance generation; the other is a systems usage centered on LLM serving. Both place tokens at the center of representation and control, but they operate on different objects: motion and music in one case, KV caches and agent rounds in the other (Yang et al., 28 Mar 2026).

Usage	Core abstraction	Representative paper
Music-to-dance generation	Dual-modality tokens for music and dance	"TokenDance: Token-to-Token Music-to-Dance Generation with Bidirectional Mamba" (Yang et al., 28 Mar 2026)
Multi-agent LLM serving	Collective KV cache sharing across synchronized rounds	"TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing" (Bian et al., 3 Apr 2026)
Broader conceptual lineage	Tokenized motion, pose, or capacity abstractions	"Towards Tokenized Human Dynamics Representation" (Li et al., 2021)

A common misconception is that TokenDance names a single unified method. The literature instead uses the label for different systems that share a token-centric design philosophy. In motion generation, tokens are discrete motion or music units. In LLM serving, the optimization target is KV-cache redundancy induced by communication structure. A plausible implication is that the name has become shorthand for approaches that convert a difficult continuous optimization problem into operations over reusable, structured units.

2. Antecedents in tokenized motion representation

A key precursor is the proposal to tokenize long human dynamics through actons: recurring temporal patterns discovered in a fully unsupervised pipeline (Li et al., 2021). The input is a 3D skeleton sequence $\bm{x} \in \mathbb{R}^{T \times 3J}$ , a Transformer encoder produces frame-wise representations $\bm{z} \in \mathbb{R}^{T \times F}$ , and K-means assigns each frame to a cluster. Actons are then extracted as maximal contiguous segments of identical cluster labels. The paper defines a two-stage framework: a Temporal Alignment Network (TAN) for self-supervised frame-wise representation learning, followed by K-means lexicon building and segmentation (Li et al., 2021).

The motivation is that long, untrimmed human dynamics such as dancing do not admit stable clip-level natural-language labels. The proposed alternative is to discover reusable mid-level units with full sequence coverage. Evaluation uses Kendall’s Tau for temporal alignment and normalized mutual information and language entropy for lexicon quality. On AIST++, raw 3D skeleton distances yield $\tau = 0.44$ , whereas TAN features yield $\tau = 0.80$ with full augmentation; on AIST++, TAN reports NMI $0.79$ and $F_2 = 0.81$ , outperforming raw skeleton, TCN, and TCC baselines (Li et al., 2021).

This lineage matters because later TokenDance-style systems adopt the same central premise: tokenization is not merely compression, but a way to expose reusable structure for downstream classification, segmentation, composition, or generation. In that sense, actons function as motion analogues of subword units, even though the paper itself uses clustering rather than learned discrete codebooks (Li et al., 2021).

3. TokenDance in music-to-dance generation

In its explicit generative usage, TokenDance is a two-stage music-to-dance generation framework designed to address the limited coverage of existing 3D dance datasets through dual-modality tokenization and efficient token-level generation (Yang et al., 28 Mar 2026). The input is a genre label $g$ and a music sequence $M = \{m_0, m_1, \dots, m_T\}$ with $m_t \in \mathbb{R}^{35}$ , and the output is a dance sequence $D = \{d_0, d_1, \dots, d_T\}$ , where each $\bm{z} \in \mathbb{R}^{T \times F}$ 0 is represented as SMPL parameters $\bm{z} \in \mathbb{R}^{T \times F}$ 1 (Yang et al., 28 Mar 2026).

The first stage uses Finite Scalar Quantization (FSQ). Given a latent $\bm{z} \in \mathbb{R}^{T \times F}$ 2, quantization is written as

$\bm{z} \in \mathbb{R}^{T \times F}$ 3

TokenDance uses 4 channel groups with levels $\bm{z} \in \mathbb{R}^{T \times F}$ 4, giving an effective codebook size of

$\bm{z} \in \mathbb{R}^{T \times F}$ 5

Dance is factorized into upper- and lower-body components, and music is decomposed into a 20-dimensional semantic component and a 15-dimensional acoustic component, each with dedicated discrete representations. The dance tokenizer is trained with a reconstruction objective that includes parameter-space and forward-kinematics terms over positions, velocities, and accelerations; the music tokenizer is trained with reconstruction losses on semantic and acoustic streams (Yang et al., 28 Mar 2026).

The second stage is a Local-Global-Local (LGL) token-to-token generator built on Bidirectional Mamba. Two local scanners process acoustic and semantic music tokens separately, a 4-layer global scanner fuses them with genre-aware conditioning, and a final local stage predicts upper- and lower-body dance tokens. The model is non-autoregressive: it takes the full music token sequence and produces all dance token logits in a single forward pass (Yang et al., 28 Mar 2026).

Quantitatively, TokenDance reports strong performance across three datasets. On AIST++, it achieves FID $\bm{z} \in \mathbb{R}^{T \times F}$ 6, FID $\bm{z} \in \mathbb{R}^{T \times F}$ 7, DIV $\bm{z} \in \mathbb{R}^{T \times F}$ 8, DIV $\bm{z} \in \mathbb{R}^{T \times F}$ 9, and BAS $\tau = 0.44$ 0. On FineDance, it reports FID $\tau = 0.44$ 1, FID $\tau = 0.44$ 2, DIV $\tau = 0.44$ 3, DIV $\tau = 0.44$ 4, and BAS $\tau = 0.44$ 5. On PopDanceSet, it reports FID $\tau = 0.44$ 6, FID $\tau = 0.44$ 7, DIV $\tau = 0.44$ 8, DIV $\tau = 0.44$ 9, and BAS $\tau = 0.80$ 0 (Yang et al., 28 Mar 2026). The user study reports DS $\tau = 0.80$ 1, DQ $\tau = 0.80$ 2, and DC $\tau = 0.80$ 3, compared with Lodge at 3.71, 3.78, and 3.69 respectively (Yang et al., 28 Mar 2026).

Efficiency is a central part of the framework’s identity. At 1024 and 4096 timesteps, TokenDance reports inference latencies of 1.22s and 2.31s, compared with 5.46s and 14.72s for Bailando, 8.59s and 27.91s for EDGE, and 4.57s and 11.96s for Lodge (Yang et al., 28 Mar 2026). This places TokenDance within a class of token-based generators that explicitly trade continuous regression for structured discrete transduction.

Several adjacent works map naturally onto a broader TokenDance paradigm, although they differ in tokenizer design, conditioning interface, and decoder class. A useful comparison is between acton discovery, hierarchical motion tokenization, image-like pose tokenization, and discrete motion planning (Ghosh et al., 23 Jun 2025).

DuetGen addresses music-driven two-person dance generation through a two-stage solution that first encodes unified two-person motions into discrete tokens with a hierarchical VQ-VAE and then generates these tokens from music with two masked transformers (Ghosh et al., 23 Jun 2025). The motion representation combines both dancers in a single vector of dimension $\tau = 0.80$ 4, with A’s root in global coordinates and B’s root relative to A. The VQ-VAE uses top-level and bottom-level codebooks to separate high-level semantics from low-level details, and introduces reconstruction, velocity, forward-kinematics, and relative-distance losses. On DD100, the two-level hierarchical VQ-VAE reports MPJPE 36.32/36.26, MPJVE 10.86/11.84, and RDE 0.17 mm; in generation it reports FID 1.31, PFID 2.54, PFC 1.47, CF 83.2%, and BAS 0.215 (Ghosh et al., 23 Jun 2025).

A different line reframes music-driven 2D dance pose generation as multi-channel image synthesis (Zhang et al., 12 Dec 2025). Here a pose sequence is encoded as a tensor $\tau = 0.80$ 5, with one-hot coordinate channels and temporal stacking, then compressed by a pretrained image VAE and modeled with a DiT-style backbone conditioned on music tokens from a pretrained Jukebox-style encoder. The method introduces a time-shared temporal indexing scheme and reference-pose conditioning for identity/body-proportion preservation and long-horizon segment-and-stitch generation. On AIST++2D, the one-hot representation improves FID from 41.35 to 29.31, DIV from 3.15 to 8.39, and BAS from 0.2432 to 0.2715 relative to a raw 2D representation; time-shared indexing improves BAS from 0.2369 to 0.2715 and DIV from 4.47 to 8.39 (Zhang et al., 12 Dec 2025).

MoTok addresses a different bottleneck: the tension between semantic token planning and fine-grained kinematic control (Gu et al., 19 Mar 2026). Its Perception–Planning–Control framework combines condition feature extraction, discrete token generation, and diffusion-based motion synthesis. The tokenizer uses a single-layer VQ codebook of size $\tau = 0.80$ 6 and latent dimension $\tau = 0.80$ 7, but delegates high-fidelity recovery to a diffusion decoder. On HumanML3D, it reports trajectory error reduced from 0.72 cm to 0.08 cm and FID reduced from 0.083 to 0.029 relative to MaskControl, while using only one-sixth of the tokens; under stronger kinematic constraints, FID further reduces from 0.033 to 0.014 rather than degrading (Gu et al., 19 Mar 2026).

These works show that “TokenDance” is not tied to a single tokenization mechanism. Clustering-based actons, VQ-VAE codebooks, FSQ, and image-VAE latents all instantiate the same broader strategy: make choreography, pose, or motion structure explicit at a token level. A plausible implication is that the main research variable is no longer whether to tokenize, but which factorization best matches the downstream control problem.

5. TokenDance in multi-agent LLM serving

A distinct systems interpretation of TokenDance appears in multi-agent LLM serving, where the central problem is KV-cache redundancy induced by synchronized rounds and All-Gather communication (Bian et al., 3 Apr 2026). In these applications, at round $\tau = 0.80$ 8 each agent produces an output block $\tau = 0.80$ 9, the scheduler gathers all outputs,

$0.79$0

and the next prompt for agent $0.79$1 is

$0.79$2

where $0.79$3 is private history and $0.79$4 specifies layout (Bian et al., 3 Apr 2026). Because every agent sees the same logical set of shared outputs but at different absolute positions, conventional prefix caching misses most cross-agent overlap, while per-request position-independent caching pays reuse overhead separately for each agent (Bian et al., 3 Apr 2026).

TokenDance lifts the optimization unit from individual requests to the entire round. Its two core mechanisms are collective KV cache reuse and Diff-Aware Storage (Bian et al., 3 Apr 2026). The KV Collector groups compatible requests, performs one batched reuse analysis over the group, and designates a Master request with minimum deviation; the remaining requests are stored as Mirrors using block-sparse diffs relative to the master. Shared blocks are thus reused once per round rather than once per agent, and storage cost converges toward mirror size rather than dense per-agent caches (Bian et al., 3 Apr 2026).

Empirically, the paper reports pairwise block similarity of 91–97% for an 8-agent GenerativeAgents round. On GenerativeAgents, Qwen2.5-7B yields a compression ratio of $0.79$5, with mirror diff about 9% of a full cache; Qwen2.5-14B yields $0.79$6, with mirror diff about 6% (Bian et al., 3 Apr 2026). Across end-to-end workloads, TokenDance supports up to 2.7x more concurrent agents than vLLM with prefix caching under SLO requirement, reduces per-agent KV Cache storage by up to 17.5x, and achieves up to 1.9x prefill speedup over per-request position-independent caching (Bian et al., 3 Apr 2026).

The evaluation further reports that, under a 1500 ms SLO, TokenDance sustains all 10 agents at QPS = 10 on GenerativeAgents with Qwen2.5-7B, whereas vLLM with prefix caching and CacheBlend baselines exceed the SLO at 10 agents (Bian et al., 3 Apr 2026). Fused diff retrieval reduces restore latency from 0.59 ms to 0.43 ms for 10 agents at QPS = 1 in GenerativeAgents/7B, a 27% reduction (Bian et al., 3 Apr 2026). This usage of TokenDance is therefore not about sequence generation, but about exploiting communication patterns to eliminate redundant memory and reuse work in serving infrastructure.

Across these literatures, TokenDance names systems that operate on reusable, explicitly managed units rather than raw continuous streams. In motion generation those units are actons, FSQ codes, or latent grid cells; in serving they are shared KV blocks and mirror diffs. The common design move is abstraction: once the problem is expressed at token granularity, alignment, reuse, planning, or fairness can be imposed more directly.

This broader token-centric view also appears in inference control. Token pools model inference capacity as explicit entitlements in token throughput, KV cache, and concurrency, with priorities shaped by service class, SLO, burst intensity, and debt (Cunningham, 27 Feb 2026). The priority weight is

$0.79$7

with debt and burst updated by exponentially weighted moving averages. In experiment, token pools maintain bounded P99 latency for guaranteed workloads during overload by selectively throttling spot traffic, whereas a baseline without admission control experiences unbounded latency degradation across all workloads (Cunningham, 27 Feb 2026). Although this paper is not titled TokenDance, its details explicitly frame token pools as “a concrete realization of what a ‘TokenDance’–style control plane for LLM inference would look like” (Cunningham, 27 Feb 2026).

An important misconception is that tokenization is synonymous with a single discrete codebook. The literature shows a wider design space. FSQ avoids codebook collapse by scalar quantization over channels (Yang et al., 28 Mar 2026). Hierarchical VQ-VAE separates high-level semantics from low-level detail (Ghosh et al., 23 Jun 2025). Diffusion-based discrete tokenizers delegate fine reconstruction to a continuous decoder so that tokens can remain compact and semantically focused (Gu et al., 19 Mar 2026). Multi-channel image approaches treat pose sequences as image-like tensors and operate with continuous latent tokens rather than hard motion codewords (Zhang et al., 12 Dec 2025).

The main open questions follow directly from these divergences. In motion generation, the unresolved issues include dataset coverage, finer-grained style control, and the balance between semantic abstraction and faithful reconstruction (Yang et al., 28 Mar 2026). In dance generation from in-the-wild 2D data, hand poses, multi-person interactions, and rare or extreme motion patterns remain difficult (Zhang et al., 12 Dec 2025). In multi-agent serving, TokenDance’s benefits depend on synchronized All-Gather structure and high prompt overlap; the gains diminish when prompts are highly divergent or asynchronous (Bian et al., 3 Apr 2026). Taken together, these results suggest that TokenDance is best understood not as a single algorithm, but as an emerging family of token-first solutions whose effectiveness depends on whether the target domain exposes enough repeated structure for tokenization, reuse, or collective control to dominate the system design.