SkinTokens: Discrete Rigging Representation

• SkinTokens are learned discrete representations that encode sparse per-bone influences on mesh vertices, transforming traditional rigging techniques.
• They use an FSQ-CVAE architecture to compress dense per-vertex weights into compact token sequences, drastically reducing storage and simplifying processing.
• The unified TokenRig model integrates SkeletonTokens and SkinTokens in an autoregressive sequence, significantly enhancing skeletal accuracy and skinning quality.

Searching arXiv for the primary paper and closely related uses of the term "SkinTokens". SkinTokens are a learned, compact, and discrete representation for skinning weights introduced to address rigging as a representation problem rather than as an ill-posed dense regression problem. In the formulation of "Skin Tokens: A Learned Compact Representation for Unified Autoregressive Rigging" (Zhang et al., 4 Feb 2026), each bone’s sparse influence over a mesh is encoded as a fixed-length sequence of discrete symbols, allowing skinning to be reframed as token sequence prediction and enabling a single autoregressive model, TokenRig, to generate skeletal parameters and skin deformations jointly. The term “SkinTokens” also appears in other domains—including controllable skin-image synthesis, skin-tone categorization, and dermatological multi-task transformers—but its primary technical meaning in this context is the discrete encoding of per-bone skinning fields for unified rig generation (Sun et al., 27 Aug 2025).

1. Definition and problem setting

The SkinTokens framework is motivated by several limitations of existing automated rigging pipelines. A typical mesh has $N \gg 10^3$ vertices and $J \gg 10^2$ bones, so the skinning weight matrix $W \in \mathbb{R}^{N \times J}$ has more than $10^7$ entries yet only approximately $4N$ nonzero elements, with reported average sparsity of $2$–$10\%$. Directly regressing $W$ with MSE or BCE is dominated by the zero-weight majority, which yields noisy “bleeding” artifacts in animation. Existing methods also often depend on geometric descriptors such as geodesic distances or watertight meshes, and skeleton inference and skinning are commonly trained separately, so the skeleton is generated “blind” to how the surface will deform (Zhang et al., 4 Feb 2026).

SkinTokens are denoted $\mathcal{D}$ and defined as learned, fixed-length sequences of discrete symbols encoding the sparse influence of one bone across all mesh vertices. For each bone $j$, its raw per-vertex weights $J \gg 10^2$0 are encoded into a short token string $J \gg 10^2$1 of length $J \gg 10^2$2, where each token takes one of $J \gg 10^2$3 possible integer values. Instead of storing $J \gg 10^2$4 continuous floats, one bone can be represented by, for example, $J \gg 10^2$5 tokens, reducing storage from hundreds of kilobytes to a few dozen bytes and yielding more than $J \gg 10^2$6 compression. This reformulates rigging as generation of joint-position tokens followed by SkinToken sequences in one autoregressive pass.

This representation suggests that the central difficulty of skinning lies not only in prediction capacity but in the mismatch between sparse geometric structure and dense continuous output spaces. A plausible implication is that compact discreteness provides a stronger inductive bias for sparse articulation fields than dense per-vertex regression.

2. FSQ-CVAE representation learning

The learned SkinTokens are produced by an FSQ-CVAE architecture. A mesh encoder $J \gg 10^2$7, described as VecSet-based, embeds surface geometry into $J \gg 10^2$8 continuous shape features. A skin encoder $J \gg 10^2$9, where $W \in \mathbb{R}^{N \times J}$0 is the ground-truth weight vector for one bone, produces a latent vector $W \in \mathbb{R}^{N \times J}$1. Finite-Scalar Quantization (FSQ) discretizes each dimension of $W \in \mathbb{R}^{N \times J}$2 onto a fixed grid of $W \in \mathbb{R}^{N \times J}$3 levels, yielding discrete $W \in \mathbb{R}^{N \times J}$4. The decoder takes $W \in \mathbb{R}^{N \times J}$5 as input and reconstructs per-vertex weights $W \in \mathbb{R}^{N \times J}$6 via a sigmoid-output network. Nested, or “prefix,” dropout randomly truncates $W \in \mathbb{R}^{N \times J}$7 during training to encourage compactness (Zhang et al., 4 Feb 2026).

The model follows the familiar CVAE structure

$W \in \mathbb{R}^{N \times J}$8

with $W \in \mathbb{R}^{N \times J}$9 prior to quantization and prior $10^7$0. The per-bone objective is an ELBO with a sparsity-aware reconstruction term:

$10^7$1

Reconstruction uses a mixture of BCE, MSE, and Dice loss:

$10^7$2

The reported ablation shows that removing Dice loss causes IoU to drop by approximately $10^7$3, and specifically from $10^7$4 to $10^7$5 on VRoid.

FSQ maps each continuous latent dimension independently onto the nearest of $10^7$6 equally spaced levels, avoids vector-quantization codebooks and dead entries, and uses a Straight-Through Estimator in the backward pass. The architecture is described as naturally capturing sparsity because very few latent dimensions carry the signal of non-zero weights, while nested dropout enforces early dimensions to encode the active region.

3. Unified autoregressive rigging with TokenRig

TokenRig models the entire rig as one token sequence:

$10^7$7

Each joint or bone is serialized in a depth-first or chain-based order. A bone is prefixed with a <type> token, followed by three delta-coordinate tokens $10^7$8, each quantized to an integer grid. After the final bone, the model appends for each bone $10^7$9 its FSQ-CVAE token sequence $4N$0. The total sequence length is

$4N$1

This formulation makes skeleton generation and skinning parts of a single generative trajectory rather than two decoupled stages (Zhang et al., 4 Feb 2026).

The autoregressive backbone is a $4N$2B-parameter Qwen3 transformer with rotary RoPE and Grouped Query Attention. At each step $4N$3 it models

$4N$4

and is trained with standard cross-entropy on the concatenation of skeleton and skin tokens. The full likelihood is

$4N$5

The significance of this design is architectural rather than purely representational. Because SkeletonTokens and SkinTokens are predicted in one sequence, the model can learn dependencies between skeleton topology, joint placement, and deformation support. This directly addresses the previously noted limitation that skeletons are generated without regard to downstream deformation quality.

4. Reinforcement-learning refinement

A supervised TokenRig model may average across difficult out-of-distribution cases, such as ignoring tails or wings, because next-token loss does not impose explicit global geometric constraints. To address this, the framework introduces a reinforcement-learning stage using Group-Relative Proximal Policy Optimization (GRPO) with four non-differentiable rewards (Zhang et al., 4 Feb 2026).

The first reward is volumetric joint coverage:

$4N$6

where $4N$7 are voxel centers in the interior, $4N$8 are joint positions, and $4N$9.

The second reward is bone-mesh containment:

$2$0

where $2$1 points are sampled along each bone segment.

The third reward is skinning coverage and sparsity:

$2$2

with

$2$3

and $2$4.

The fourth reward is motion smoothness. Five random poses are sampled, LBS is applied to each edge $2$5, and the edge distortion is measured before defining

$2$6

These terms are combined as

$2$7

with $2$8 and $2$9, while invalid rigs receive $10\%$0. The GRPO optimization objective is

$10\%$1

where $10\%$2, $10\%$3, and $10\%$4.

This suggests that the RL stage is not used to learn rigging from scratch but to regularize an already trained sequence model toward geometric validity, sparse support, and stable deformations under motion.

5. Empirical performance and ablations

The reported evaluation separates skeletal accuracy from skinning quality. For skeletons, the metrics are Joint-to-Joint Chamfer (J2J), Joint-to-Bone (J2B), and Bone-to-Bone (B2B), all with lower values preferred. Relative to RigNet, MagicArticulate, Puppeteer, and UniRig, TokenRig reduces J2J by $10\%$5–$10\%$6. For skinning, the metrics are L1 Error, L1 Variance, Precision/Recall on active region IoU, and Motion Loss. TokenRig reduces L1 error from $10\%$7 to $10\%$8 on ModelsResource and from $10\%$9 to $W$0 on Articulation 2.0, corresponding to a $W$1–$W$2 relative improvement in skinning accuracy. Qualitatively, the paper reports semantically coherent skeletons for ears, horns, and tails; reduced “bleeding” onto disconnected parts; and improved auxiliary bones and tighter skinning after GRPO refinement (Zhang et al., 4 Feb 2026).

The discrete representation itself is also analyzed. Figure 1 reports FSQ-CVAE reconstruction IoU as a function of $W$3 and codebook size, and indicates that even with $W$4 tokens, IoU remains above $W$5. Table 2 reports a codebook configuration $W$6, giving $W$7 entries, $W$8 compression, and $W$9 utilization.

Ablations emphasize several components. Removing Dice loss lowers IoU from $\mathcal{D}$0 to $\mathcal{D}$1 on VRoid and reduces full-coverage mask accuracy by approximately $\mathcal{D}$2. Omitting non-uniform scaling, subtree dropping, or joint deletion increases J2J by $\mathcal{D}$3–$\mathcal{D}$4, indicating that structured perturbations matter for robustness. The GRPO stage causes only slight changes in J2J but improves out-of-distribution skeletons with tails and wings, and reduces Motion Loss by approximately $\mathcal{D}$5 in out-of-distribution cases.

These results indicate that the gains are distributed across both representation learning and sequence modeling: SkinTokens improve fidelity of the skinning field itself, while the unified TokenRig plus RL stack improves skeleton quality and robustness on complex assets.

6. Broader usage of the term and conceptual distinctions

The term “SkinTokens” is not unique to rigging. In "Controllable Skin Synthesis via Lesion-Focused Vector Autoregression Model" (Sun et al., 27 Aug 2025), SkinTokens are the flattened discrete latent codes produced by a multiscale lesion-focused VQVAE for skin-lesion image synthesis. There, the encoder maps a $\mathcal{D}$6 image into multiscale discrete token grids $\mathcal{D}$7, and flattening these grids in raster-scan order yields a token sequence used by a decoder-only VAR transformer conditioned on lesion type and radiomic measurements. In that setting, SkinTokens are image-code indices rather than bone-conditioned deformation encodings.

In "TrueSkin: Towards Fair and Accurate Skin Tone Recognition and Generation" (Lu, 13 Sep 2025), “SkinTokens” refers to six discrete skin-tone categories: dark, brown, tan, medium, light, and pale. These classes are used for recognition benchmarking, weighted cross-entropy training, fairness analysis, and prompt-conditioned generation. The token concept is categorical and semantic, not generative-latent in the rigging sense.

In "MT-TransUNet: Mediating Multi-Task Tokens in Transformers for Skin Lesion Segmentation and Classification" (Chen et al., 2021), the relevant tokenization consists of segmentation patch tokens and a learnable classification token inside a shared transformer encoder. The paper’s technical contribution is token mediation between segmentation and classification rather than a discrete latent vocabulary named SkinTokens in the same sense.

These usages share a family resemblance: each replaces raw high-dimensional structure with tokenized intermediates that a transformer or related model can process efficiently. However, their semantics differ sharply. In rigging, SkinTokens are per-bone, sparse deformation codes; in skin-image synthesis, they are VQ-style image latents; in skin-tone recognition, they are human-defined category labels; and in dermatological multi-task learning, they are transformer input tokens. A plausible implication is that “SkinTokens” functions as a cross-domain label for discrete structure, but only the rigging formulation defines it as a learned compact representation for unified autoregressive rig generation.

7. Pipeline role, limitations, and future directions

Within production-oriented rigging workflows, the paper describes several practical implications. Compact SkinTokens can be stored, previewed, and edited as discrete scripts. A single autoregressive pass yields both skeleton and skinning, removing the need for separate heatmap regressors or MST solves. The authors further note the potential to augment commercial tools such as Blender and Maya with a plug-in that decodes token sequences into rigs (Zhang et al., 4 Feb 2026).

The reported limitations and future directions are specific. The paper notes a residual gap versus continuous-latent VAEs in extremely complex skinning and points to exploration of hybrid continuous-discrete latents. It also identifies interactive or conditional rigging—such as user-specified template constraints or partial bone layouts—as a future direction, and proposes extending the RL stage with physics-based animation rewards that capture dynamic plausibility, including muscle simulation and collision.

Taken together, SkinTokens define a representation-centric approach to rigging. The central contribution is not merely discretization, but the coupling of a sparse per-bone code, an autoregressive sequence model over the entire rig, and a reward-based refinement stage operating on geometric and semantic validity. This establishes rigging as a generative token modeling problem in which skeleton topology, joint placement, and skin deformation are learned jointly rather than optimized in isolation.