Skeleton-Founded Tokenization

• Skeleton-founded tokenization is a design principle where tokens are constructed based on explicit skeleton structures like hierarchies, kinematic trees, or body-part decompositions.
• It improves performance and fairness in systems such as generative recommendation, motion modeling, and rigging by controlling branching and preserving semantic regularities.
• Techniques like head-prefix inheritance, residual quantization, and graph-based encoding balance structural fidelity and efficiency to optimize token representations.

Skeleton-founded tokenization denotes a family of tokenization schemes in which discrete symbols are constructed with explicit dependence on a skeleton: a hierarchical item code in generative recommendation, a BVH kinematic tree in motion modeling, a body-part decomposition in skeleton sequence learning, or a skeleton tree and bone semantics in rigging. In all of these settings, the token inventory is not treated as an undifferentiated compression artifact; it is designed to preserve structural priors, constrain branching, and expose semantically useful regularities to downstream sequence models. Across the cited literature, the term is used in several non-identical ways. This suggests that skeleton-founded tokenization is better understood as a design principle—tokenization grounded in skeleton structure—than as a single canonical algorithm.

1. Definition, scope, and representational motivation

A common motivation for skeleton-founded tokenization is dissatisfaction with structurally blind tokenizers. In the recommendation setting, undifferentiated semantic IDs make tail items repeatedly compete with head items along multiple decoding steps (Yin et al., 16 May 2026). In motion modeling, per-joint quantization presupposes a fixed topology and therefore breaks on heterogeneous BVH skeletons (Xu et al., 6 Feb 2026). In rigging, purely geometric or naive traversal-based serialization struggles with complex or non-standard trees, often producing unstable ordering or invalid connectivity (Zhang et al., 16 Apr 2025). In skeleton sequence learning, treating raw joint arrays as a homogeneous tensor overlooks body-part semantics and short-term motion primitives (Yang et al., 6 Mar 2026, Gökay et al., 6 Aug 2025).

A broader tokenization argument in language modeling provides useful context. The abstract of "Tokens, the oft-overlooked appetizer: LLMs, the distributional hypothesis, and meaning" states that tokens act both as semantic primitives and as vehicles for conveying salient distributional patterns, and that tokenizer objectives can affect model cognition (Zimmerman et al., 2024). A plausible implication is that skeleton-founded tokenization applies the same representational logic to structured domains: if the units presented to the model already align with hierarchy, morphology, or kinematics, the learned distribution can be more faithful to the underlying task.

Domain	Skeleton foundation	Tokenization consequence
Generative recommendation	Head-item SID skeleton	Tail items inherit head prefixes
Motion modeling	BVH ontology and kinematic tree	Tokens become topology-invariant summaries
Skeleton sequence learning	Body-part layout or motion patches	Tokens encode coherent spatio-temporal units
Rigging and skeleton generation	Tree structure, parent-child relations, bone semantics	Grammar-constrained or bone-indexed sequences

The family resemblance across these systems is therefore structural rather than superficial. Each method uses the skeleton to determine what counts as a token, where branching may occur, and how much information is shared across related entities.

2. Head-prefix inheritance in generative recommenders

In "Echoes in Filter Bubble: Diagnosing and Curing Popularity Bias in Generative Recommenders," skeleton-founded tokenization (SKT) is one of the two components of Ghost, together with asymmetric unlikelihood optimization (AUO) (Yin et al., 16 May 2026). The paper identifies two causes of popularity bias in SID-based generative recommendation: a token-level optimization flaw under maximum likelihood, and the undifferentiated property of item tokenization. Under the latter, a tail item’s generation path encounters multiple head-dominated branching points, so local imbalance compounds geometrically.

SKT defines the head-item set as the skeleton. Head items are first assigned fixed-length semantic IDs of length $L^h$ by residual quantization. For a tail item $v'$, the tokenizer finds the closest head item $v^*$ by cosine similarity, inherits the first $L^h$ tokens of $\Omega_{v^*}$, and appends $L^t$ additional tail-specific residual tokens, yielding

$$\Omega_{v'} = (c_{v^*}^{(1)}, \dots, c_{v^*}^{(L^h)}, c_{v'}^{(1)}, \dots, c_{v'}^{(L^t)}).$$

Head items therefore define the shared scaffold, while tail items differ only after a controlled prefix.

This structural change alters the bias geometry. In the conventional setting, if $z$ unpredictable head-tail competition steps occur, the paper gives the bound

$$\mathcal{P}_\theta(v_{tail}\mid h_u) \le (\gamma_{min})^{-z}\cdot \prod_j \mathcal{P}_d(c_{tail}^{(j)}\mid h_u,c^{<j}),$$

where $\gamma_{min}>1$. Under SKT, the multi-step suppression is collapsed to a single predictable divergence at step $v'$0, where the head path emits EOS and the tail path emits its first tail-specific token:

$v'$1

The paper frames this as a reduction from $v'$2 suppression to $v'$3.

Ghost uses the same residual-quantization codebooks as standard SID tokenizers, with codebook size per level $v'$4. The implementation splits items into head and tail using the standard 2–8 split, with head length $v'$5 and tail additional length $v'$6. AUO then penalizes semantically similar but structurally divergent head SIDs for tail targets, using $v'$7, $v'$8, and $v'$9.

Empirically, Ghost is evaluated on Amazon "Musical Instruments," "Arts, Crafts and Sewing," and "Video Games." Across all three datasets, the paper reports that Tail HR@10 improves versus standard GRs by 63.91% on average and versus debiasing baselines by 28.39% on average, with a maximum of 57.24%. MGU drops by 55.76% versus standard GRs, and by 16.68% versus IFairLRS-RR on average. The overall HR/NDCG reduction relative to LC-Rec is 7.46% and 6.34% on average. The ablation isolates SKT directly: on Instruments, Ghost achieves Tail HR@10 $v^*$0, Tail NDCG@10 $v^*$1, MGU@10 $v^*$2, whereas removing SKT yields Tail HR@10 $v^*$3, Tail NDCG@10 $v^*$4, MGU@10 $v^*$5. Replacing SKT with longer SIDs but without inheritance (RQK-4-6) gives Tail HR@10 $v^*$6, Tail NDCG@10 $v^*$7, MGU@10 $v^*$8. In this usage, skeleton-founded tokenization is fundamentally a branching-control mechanism.

3. Skeleton-grounded but topology-agnostic motion tokenization

"NECromancer: Breathing Life into Skeletons via BVH Animation" uses skeleton-founded tokenization in a different sense: the tokenization is founded on the source skeleton at encode/decode time, but the discrete motion code itself is topology-agnostic (Xu et al., 6 Feb 2026). NEC consists of the Ontology-aware Skeletal Graph Encoder (OwO), the Topology-Agnostic Tokenizer (TAT), and the Unified BVH Universe (UvU).

The motion input is represented as $v^*$9. For the root joint,

$L^h$0

and for non-root joints,

$L^h$1

OwO constructs a directed graph from the BVH rest pose, using joint names, rest-pose offsets, and topology. Joint semantics are embedded with CLIP text embeddings of joint names, and the graph encoder is pretrained with geometric distance regression, LCA prediction, and semantic contrastive objectives. The resulting per-joint embeddings $L^h$2 condition both encoder and decoder.

TAT then fuses motion features with OwO embeddings, introduces a learnable virtual joint at each time step, performs spatio-temporal processing and temporal downsampling, and quantizes only the final virtual-joint summary with residual vector quantization. If $L^h$3 is the virtual-joint summary, then

$L^h$4

with the residual recursion

$L^h$5

Because only the virtual joint is quantized, the token sequence has shape $L^h$6 and is independent of the joint count $L^h$7.

This design separates structure from motion. The skeleton is used to interpret and reconstruct motion, but the discrete token space stores only a universal motion description. The default configuration uses $L^h$8 codebooks of size $L^h$9, an OwO with 8 graph attention blocks and 512 latents, and a TAT with 3 spatio-temporal blocks.

The UvU corpus contains 47,807 BVH sequences from HumanML3D, Objaverse-XL, and Truebones Zoo, spanning humans, quadrupeds, birds, and fantasy creatures. On reconstruction, NEC w/ RVQ reports MPJPE of $\Omega_{v^*}$0 on HumanML3D, Objaverse-XL, and Zoo, compared with TM2T at $\Omega_{v^*}$1. GeoDist is reported as $\Omega_{v^*}$2 degrees, consistently the best rotational accuracy. Structural prior ablations show that replacing OwO with learnable queries or name-only variants degrades MPJPE and GeoDist, and OwO pretraining with LCA, pairwise distance, and contrastive name losses improves reconstruction and retrieval. Here, skeleton-founded tokenization is not a rigid serialization of the tree; it is a factorization in which the skeleton conditions token semantics without being embedded into the token sequence itself.

4. Skeleton sequences as image patches and motion words

Skeleton-founded tokenization also appears in sequence representation learning, where the token boundary is derived from body-part structure or short motion segments rather than from a pre-existing symbolic vocabulary.

In "Skeleton-to-Image Encoding: Enabling Skeleton Representation Learning via Vision-Pretrained Models," a skeleton sequence $\Omega_{v^*}$3 is rearranged into an image-like tensor by partitioning the body into five semantic parts—spine, left arm, right arm, left leg, right leg—and ordering joints along each kinematic chain (Yang et al., 6 Mar 2026). Time defines the vertical axis, ordered joints define the horizontal axis, and the coordinate channels map directly to RGB:

$\Omega_{v^*}$4

After interpolation and normalization, the representation is resized to $\Omega_{v^*}$5. With ViT-B/16, patchification yields $\Omega_{v^*}$6 patches, and each token is

$\Omega_{v^*}$7

Because adjacent columns correspond to semantically coherent body parts and adjacent rows correspond to adjacent time steps, the visual patches function as skeleton-founded tokens. The paper reports that random masking at 75% gives the best performance. On NTU-60, S2I obtains 83.1% linear evaluation on C-sub and 91.0% fine-tuned accuracy; the 3-stream version reaches 85.8% and 93.1%. On NTU-120, S2I gives 75.0%/86.6% and 3s-S2I gives 78.9%/90.2%. Cross-format evaluation reports NTU-60 $\Omega_{v^*}$8 Toyota CV1 at 53.8% for 3s-S2I and NTU-60 $\Omega_{v^*}$9 NW-UCLA at 94.2%, without explicit joint alignment.

In "Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentation," the token is a quantized latent patch rather than an image patch (Gökay et al., 6 Aug 2025). A sequence-to-sequence temporal autoencoder with joint-disentangled embeddings maps input $L^t$0 into a latent sequence, which is then divided into non-overlapping patches of length $L^t$1:

$L^t$2

Each patch $L^t$3 is assigned to its nearest codeword in a codebook $L^t$4:

$L^t$5

and the codebook is updated with EMA using $L^t$6. The total loss is

$L^t$7

Patch size is set to one second of motion per dataset, such as $L^t$8 for LARa and $L^t$9 for BABEL. The resulting token sequence directly defines action segments. On HuGaDB, SMQ reports MoF 42.0, Edit 36.1, and F1@50 24.3; on LARa, MoF 37.4, Edit 39.4, F1@50 16.4; on BABEL subset-2, MoF 49.1, Edit 37.8, F1@50 27.4. Ablations show that frame-wise quantization ($$\Omega_{v'} = (c_{v^*}^{(1)}, \dots, c_{v^*}^{(L^h)}, c_{v'}^{(1)}, \dots, c_{v'}^{(L^t)}).$$0) over-segments, while one-second patches improve Edit and F1.

These two systems instantiate the same principle at different scales. S2I makes a token correspond to a semantically ordered spatio-temporal patch in an image grid; SMQ makes a token correspond to a prototypical short-term motion primitive in a latent space. In both cases, the skeleton determines the partition from which the token alphabet is induced.

5. Rigging and skeleton generation in 3D assets

Rigging papers use skeleton-founded tokenization most explicitly: the skeleton tree itself, or bone-indexed skinning attached to that tree, becomes the serialized object of prediction.

System	Tokenized object	Structural mechanism
UniRig	Skeleton tree	Grammar with templates, DFS branches, parent-first decoding
SkinTokens / TokenRig	Per-bone skinning columns	Bone-indexed discrete codes after skeleton tokens
Animator-centric SG	Semantic bone groups and joint-parent coordinates	GroupStart, CLS, density token, within-group DFS

"One Model to Rig Them All: Diverse Skeleton Rigging with UniRig" introduces Skeleton Tree Tokenization for automatic rigging (Zhang et al., 16 Apr 2025). The token vocabulary includes <bos>, <eos>, <cls>, type tags, <branch_token>, and discretized coordinate tokens. Joint coordinates are discretized into $$\Omega_{v'} = (c_{v^*}^{(1)}, \dots, c_{v^*}^{(L^h)}, c_{v'}^{(1)}, \dots, c_{v'}^{(L^t)}).$$1 bins:

$$\Omega_{v'} = (c_{v^*}^{(1)}, \dots, c_{v^*}^{(L^h)}, c_{v'}^{(1)}, \dots, c_{v'}^{(L^t)}).$$2

The grammar begins with <bos> <cls>, emits zero or more template segments and spring-bone segments, then encodes the remaining forest as DFS-discovered branches of the form parent coordinates followed by child coordinates. Illegal token classes are masked at decode time, children are ordered by $$\Omega_{v'} = (c_{v^*}^{(1)}, \dots, c_{v^*}^{(L^h)}, c_{v'}^{(1)}, \dots, c_{v'}^{(L^t)}).$$3, and detokenization uses a proximity merge rule to sew branches together into a connected, acyclic tree. The optimized serialization reduces sequence length from naive $$\Omega_{v'} = (c_{v^*}^{(1)}, \dots, c_{v^*}^{(L^h)}, c_{v'}^{(1)}, \dots, c_{v'}^{(L^t)}).$$4 tokens to $$\Omega_{v'} = (c_{v^*}^{(1)}, \dots, c_{v^*}^{(L^h)}, c_{v'}^{(1)}, \dots, c_{v'}^{(L^t)}).$$5 tokens, a reduction of about 27–30% in practice. The paper reports a 215% improvement in rigging accuracy and a 194% improvement in motion accuracy, with J2J on VRoid* of 0.0101 for UniRig versus 0.0934 for TA-Rig and 0.2484 for RigNet.

"Skin Tokens: A Learned Compact Representation for Unified Autoregressive Rigging" shifts the tokenization target from skeleton topology to skinning weights (Zhang et al., 4 Feb 2026). A per-bone skinning column $$\Omega_{v'} = (c_{v^*}^{(1)}, \dots, c_{v^*}^{(L^h)}, c_{v'}^{(1)}, \dots, c_{v'}^{(L^t)}).$$6 is encoded by an FSQ-CVAE conditioned on geometry and quantized channel-wise:

$$\Omega_{v'} = (c_{v^*}^{(1)}, \dots, c_{v^*}^{(L^h)}, c_{v'}^{(1)}, \dots, c_{v'}^{(L^t)}).$$7

The resulting SkinTokens are appended after skeleton tokens in a single autoregressive sequence. The representation uses codebook level sizes such as $$\Omega_{v'} = (c_{v^*}^{(1)}, \dots, c_{v^*}^{(L^h)}, c_{v'}^{(1)}, \dots, c_{v'}^{(L^t)}).$$8, corresponding to 64,000 combinations, with reported utilization of 86.2%. The FSQ-CVAE objective is

$$\Omega_{v'} = (c_{v^*}^{(1)}, \dots, c_{v^*}^{(L^h)}, c_{v'}^{(1)}, \dots, c_{v'}^{(L^t)}).$$9

where the Dice term is introduced to counter sparsity in skinning. With $z$0 tokens, the representation achieves 183.74$z$1 compression versus FP16. Quantitatively, the paper reports 98%–133% improvements in skinning accuracy over state-of-the-art regressors and 17%–22% bone prediction gains with RL. On ModelsResource, for example, skin L1 improves from 0.0573 for RigNet to 0.0163 for TokenRig + SkinTokens (6 tokens, RL), and Motion improves from 0.0789 to 0.0158.

"Animator-Centric Skeleton Generation on Objects with Fine-Grained Details" makes semantics-first grouping the basis of tokenization (Sun et al., 22 Apr 2026). A GraphTransformer trained on 10,000 manually labeled skeletons predicts per-joint semantic classes. Humanoid skeletons are then partitioned into ordered groups such as main, hair, cloth, and other; tetrapods use main and auxiliary groups. The sequence includes a BOS token, a GroupStart token per semantic group, a density control token, a CLS token for global category, and skeleton tokens encoding six discretized coordinates per joint: its $z$2 and its parent’s $z$3. Within each group, traversal is DFS with children sorted by $z$4. The decoder is OPT-350M with cross-attention to shape, density, CLS, and optionally main-bone embeddings. On a dataset of 82,633 rigged meshes, the paper reports Precision 0.745, Recall 0.731, Accuracy 0.729, F1 0.730, J2J 0.036, J2B 0.027, and B2B 0.025, outperforming MagicArticulate, Puppeteer, and UniRig as evaluated there. Ablations show that removing the density token increases J2J from 0.036 to 0.041, and that semantic grouping with full DFS ordering yields better alignment than naive global DFS, semantic grouping without local DFS, or semantic grouping with BFS.

Taken together, these rigging papers show three distinct but compatible interpretations of skeleton-founded tokenization: grammar-constrained tree serialization, bone-indexed deformation tokenization, and semantics-aware group serialization.

6. Recurring design principles, trade-offs, and unresolved issues

Across Ghost, NEC, UniRig, SkinTokens, and the animator-centric skeleton generator, several recurrent design motifs appear (Yin et al., 16 May 2026, Xu et al., 6 Feb 2026, Zhang et al., 16 Apr 2025, Zhang et al., 4 Feb 2026, Sun et al., 22 Apr 2026). First, tokenization is used to move uncertainty away from early decoding steps: Ghost forces tail items to share a head prefix; UniRig emits parents before children; the animator-centric model emits semantic groups in a fixed order. Second, most systems separate a shared scaffold from a residual or auxiliary component: head-prefix inheritance plus tail suffixes, OwO-conditioned motion plus topology-agnostic RVQ codes, skeleton tokens plus per-bone SkinTokens, or main bone groups plus auxiliary detail groups. Third, ordering is rarely arbitrary. DFS traversal, child sorting by spatial coordinates, and fixed group order all act as inductive biases that stabilize the autoregressive event space.

The principal trade-off is between structural fidelity and generic efficiency. Ghost reports only modest reductions in overall HR/NDCG relative to the strongest popularity-biased baseline, but the improvement in fairness is substantial (Yin et al., 16 May 2026). NEC achieves topology-invariant tokens, yet still depends on BVH rest pose, joint names, and ontology quality (Xu et al., 6 Feb 2026). UniRig shortens sequences by about 27–30%, but its guarantees depend on grammar and proximity merging (Zhang et al., 16 Apr 2025). SkinTokens achieve 183.74$z$5 compression, but the representation remains sensitive to token budget, Dice weighting, and OOD topology (Zhang et al., 4 Feb 2026). SMQ gains temporal coherence from one-second patches, but transitions are limited to patch boundaries and therefore temporally coarse (Gökay et al., 6 Aug 2025).

Several limitations recur. Hyperparameters that define the skeleton-token interface are highly consequential: Ghost is sensitive to $z$6 and $z$7; NEC reports trade-offs in RVQ depth and robustness; SMQ depends on patch size $z$8, EMA decay $z$9, and vocabulary size $$\mathcal{P}_\theta(v_{tail}\mid h_u) \le (\gamma_{min})^{-z}\cdot \prod_j \mathcal{P}_d(c_{tail}^{(j)}\mid h_u,c^{<j}),$$0; the animator-centric model currently offers only global, not region-level, density control (Yin et al., 16 May 2026, Xu et al., 6 Feb 2026, Gökay et al., 6 Aug 2025, Sun et al., 22 Apr 2026). Generalization also remains uneven under severe topology shift or dataset imbalance. NEC explicitly notes challenges for large embodiment gaps such as quadruped-to-human transfer, and the animator-centric framework identifies underrepresentation of vehicles and accessories as a weakness (Xu et al., 6 Feb 2026, Sun et al., 22 Apr 2026).

A broader interpretive thread links these results back to tokenization research more generally. If tokens function as semantic primitives and as carriers of salient distributional patterns, then skeleton-founded tokenization can be read as an attempt to redesign the model’s event space so that hierarchy, kinematics, and part semantics are already visible before learning begins (Zimmerman et al., 2024). On that reading, the contribution of skeleton-founded tokenization is not merely a better compression code. It is a representational intervention that changes what regularities the model can see, where competition occurs, and how meaning-like structure is partitioned into discrete units.