GPC: Large-Scale Generative Pretraining for Transferable Motor Control

Abstract: Developing controllers capable of completing a wide range of tasks in a natural and life-like manner is a key challenge in enabling practical applications of physics-based character animation. In this work, we introduce Generative Pretrained Controllers (GPC), which leverage tokenization and next-token modeling to create general-purpose, reusable generative controllers from large-scale motion datasets. Our framework utilizes end-to-end reinforcement learning to jointly optimize a "motion vocabulary", modeled via Finite Scalar Quantization (FSQ), along with a corresponding control policy that can map the discrete codes to physics-based controls. After the "codebook" has been learned, the underlying structure of this large vocabulary is modeled by training a GPT-style autoregressive transformer, leading to a powerful generative controller that generates controls for a physically simulated character by performing next-token prediction. Once the generative controller has been trained, we propose a suite of adaptation techniques for finetuning the controller for new downstream tasks. Our proposed framework greatly simplifies the training process compared to previous tokenized methods, and achieves a 99.98% success rate in reproducing a vast corpus of motion clips. The generative controller exhibits a variety of natural emergent behaviors, such as responsive behaviors to perturbations and recovery behaviors after falling. This results in highly robust general purpose controllers for a variety of downstream applications.

• The paper introduces a novel framework that leverages FSQ to discretize skill representations, addressing issues like mode collapse in continuous latent models.
• It employs a transformer-based generative controller that autoregressively predicts skill tokens, ensuring temporally coherent and dynamic motor behavior.
• Parameter-efficient task adaptation via Conditional Low-Rank Adaptation enables effective transfer to downstream tasks with minimal additional parameters.

Generative Pretrained Controllers: Large-Scale Generative Pretraining for Transferable Motor Control

Problem Formulation and Context

The paper "GPC: Large-Scale Generative Pretraining for Transferable Motor Control" (2606.29148) introduces a unified framework for learning general-purpose, reusable motor skill controllers for physics-based character animation using large-scale motion datasets. The primary objective is to overcome the limitations of continuous latent representations—prone to mode collapse and instability—by leveraging discrete skill quantization and transformer-based sequence modeling. The resulting Generative Pretrained Controllers (GPC) achieve high fidelity motor imitation, emergent robustness, and efficient transfer to downstream tasks, significantly advancing generalization and adaptation in physical character control.

Skill Quantization via FSQ

A cornerstone of the framework is skill quantization using Finite Scalar Quantization (FSQ). Unlike VQ-VAE approaches relying on explicit codebooks and elaborate stabilization heuristics, FSQ discretizes encoder outputs into fixed scalar levels, removing codebook learning challenges and dead-code phenomena. The encoder maps target motion windows into continuous latents, which are rounded to discrete codes, forming a structured latent skill space optimized end-to-end with RL using a strict motion-tracking objective.

FSQ circumvents mode dropout and code collapse, enabling scalable training on hundreds of hours of data and supporting motion categories from routine locomotion to dynamic acrobatics (Figure 1).

Figure 1: Keyframes of the results produced by the FSQ tracking controller on agile motions from Bones.

Further, the FSQ latent space exhibits structured topology supporting skill retrieval, discrimination, and interpolation, as confirmed by t-SNE projections and paired retrieval experiments (Figure 2, supplementary).

Generative Controller Design

Building on the discrete skill representation, a transformer-based controller is trained to autoregressively model the distribution of grouped skill tokens. This design leverages next-token prediction, akin to GPT-style autoregression, where at each step the model samples grouped skill tokens conditioned on the character's state and prior tokens (Figure 3). This allows for rich, temporally coherent behavior synthesis.

Figure 3: Skill quantization and generative transformer controller training pipeline, mapping reference motions to discrete latents and decoding those to actions.

Unconditional sampling from the generative controller reveals a diverse repertoire of emergent behaviors, including highly dynamic skills (Figure 4).

Figure 4: Unconditional sampling of the generative controller. GPC produces a wide array of highly dynamic skills, such as jumping, leaping, and rolling.

Critically, GPC produces robust responses and recovery strategies to external perturbations without explicit training signals (Figures 4, 5).

Figure 5: GPC produces robust and natural recovery behaviors when subjected to external perturbations.

Figure 6: GPC exhibits diverse and versatile responses to force perturbations with a generative controller trained with Bones.

Parameter-Efficient Task Adaptation

To enable efficient transfer to new tasks without retraining the full generative model, the framework incorporates Conditional Low-Rank Adaptation (CoLA) for parameter-efficient fine-tuning. Lightweight adapters (modulation layers) are inserted into the pretrained controller, adding less than 1% parameters. Fine-tuning proceeds via task-specific RL objectives and guided exploration strategies such as Nucleus Sampling, maintaining the natural behaviors encoded in the base model (Figure 7).

Figure 7: CoLA can be used to efficient finetune GPC to new tasks. By only training adaptation layers during finetuning, the pretrained generative controller can effectively complete downstream tasks while preserving natural behaviors encoded in the original base model.

Supervised fine-tuning (SFT), when combined with RL, can further bias the controller toward stylized behaviors (Figure 8, 10).

Figure 8: Qualitative comparison of task completion with and without SFT (left). SFT enables stylistic control by biasing the controller towards a desired behavioral style, such as crouched walking (right).

Figure 9: Trajectories of head joint height during task completion for controllers adapted (left) without SFT and (right) with SFT on a crouch motion.

Experimental Results and Numerical Highlights

The FSQ-based tracking controller achieves a 99.98% success rate in reproducing over 600 hours of motion clips, outperforming VQ-VAE and distillation-based tracking controllers, with mean per-joint position errors comparable to continuous MLP baselines. Ablations confirm the superiority of end-to-end RL training for encoder optimization.

Comparison against CVAE priors reveals that GPC exhibits substantially greater behavioral diversity (see Figures 7, 8), robust survival under physical disturbances, and lower jerk/acceleration artifacts. GPC maintains high success rates and fine task completion, even under strong pushes, while CVAE policies collapse.

Figure 10: Comparison between downstream task behavior of GPC and CVAE under identical task conditions. GPC produces more diverse behaviors than CVAE.

Figure 11: Trajectory comparison. The CVAE baseline (left) collapses to similar behaviors, while GPC (right) produces diverse trajectories.

Nucleus Sampling configuration studies reveal that limiting token support to high-probability codes preserves behavioral robustness and diversity during adaptation (Figure 12).

Figure 12: Return Curve under different Nucleus Sampling configurations during task adaptation. (a) Varying the sampling temperature $T$ with a fixed nucleus threshold $p = 0.9$. (b) Varying the nucleus threshold $p$ with a fixed temperature $T = 1.0$.

Task adaptation for structured human-scene interaction tasks (barrier and platform navigation) demonstrates transferability and the retention of rich, natural motor skills.

Implications and Theoretical Perspective

The GPC framework demonstrates that discrete skill tokenization and next-token generative modeling unlock scalable, high-fidelity motor controllers able to generalize across a vast motion domain. By disentangling latent skill representation from physical control action, robust compositionality and adaptation are enabled. Discrete token spaces mitigate common hazards of continuous latent priors, notably mode collapse and off-manifold drift, and further support inductive bias in downstream RL or SFT.

These advances pave the way for general-purpose motor controllers applicable to animation, robotics, and embodied agents. The modularity and adaptation interface afford rapid finetuning for new tasks, environments, or stylistic constraints, without catastrophic forgetting. Theoretically, the work supports the thesis that structured discrete latent spaces and autoregressive generative modeling can match or exceed the expressiveness of continuous priors while providing superior stability and compositionality.

Prospects for Future Research

Future extension directions include multimodal conditioning (e.g., text commands or semantic tokens), hierarchical composition for long-horizon planning, and integration with scene-aware interaction. Application to real-world robotic systems would require improved sim-to-real transfer and integration with perception modules. Investigation into hybrid discrete-continuous latent spaces and compositional code blending may provide further gains for highly dynamic and interactive skill synthesis.

Conclusion

The paper presents a scalable, robust solution to generative pretraining and adaptation in physics-based motor control, leveraging discrete skill quantization and transformer-based next-token modeling. Empirical results validate strong transferability, diversity, and physical robustness. The approach markedly improves over prior continuous latent methods and defines a path forward for generalizable skill controllers across animation and robotic domains.