Pre-Control for Motor Control
- Pre-Control is a framework that decouples visual representation learning from policy optimization using self-supervised masked image modeling.
- It trains reinforcement learning controllers on frozen visual features, achieving up to 80% higher success rates compared to supervised methods.
- The approach generalizes across diverse tasks and robot embodiments by leveraging large egocentric image datasets in a benchmark of hand-designed control scenarios.
"Masked Visual Pre-training for Motor Control" designates a visuo-motor learning framework in which a visual encoder is first pre-trained by masked image modeling on large collections of real-world images and is then frozen while reinforcement-learning controllers are trained on top from pixel observations. The method, termed Masked Visual Pre-training for Motor Control (MVP), uses the same visual representation across diverse motor control tasks without task-specific fine-tuning of the encoder. The work positions large-scale, label-free pre-training on real-world images as a viable route for learning control from pixels, and introduces a benchmark suite of hand-designed tasks varying in movements, scenes, and robots. It reports that, without labels, state-estimation, or expert demonstrations, the resulting controllers consistently outperform supervised encoders by up to 80% absolute success rate and sometimes match oracle-state performance (Xiao et al., 2022).
1. Conceptual basis and scope
MVP is a pre-control framework in which representation learning precedes policy learning and is optimized independently of any single downstream control task. The core procedure is two-stage: first, visual representations are pre-trained by masked image modeling on natural images; second, the visual encoder is frozen and neural network controllers are trained on top with reinforcement learning from pixel observations plus proprioception. No task-specific fine-tuning of the encoder is performed, and the same encoder is reused across all motor control tasks (Xiao et al., 2022).
This design is technically significant because it separates visual representation learning from policy optimization. In the reported formulation, the visual encoder is not adapted by downstream RL gradients; instead, controller learning is restricted to lightweight policy and value networks operating on the frozen image embedding and the proprioceptive state. The paper states that, to the best of its knowledge, this is the first self-supervised model to exploit real-world images at scale for motor control (Xiao et al., 2022).
The work also frames the problem as learning motor control directly from pixels rather than relying on labels, state-estimation, or expert demonstrations. That framing is operationalized through a benchmark, PixMC, comprising hand-designed tasks spanning multiple task families, scenes, and robot embodiments (Xiao et al., 2022).
2. Visual pre-training method and objective
The visual backbone is a Vision Transformer, specifically ViT-Small following Dosovitskiy et al., with patch size , embedding dimension $384$, transformer depth $12$ blocks, $6$ attention heads, MLP expansion , and $4.6$ G FLOPs for inputs, approximately ResNet-50. Pre-training uses MAE-style masked image modeling with random patch masking and a high masking ratio, exemplified by masked patches. Only unmasked patches are passed to the encoder, with a heavy encoder and light decoder design, and minimal augmentations rather than object-centric cropping or handcrafted invariances (Xiao et al., 2022).
The reconstruction head is a lightweight MAE-style decoder that reconstructs pixel values for masked patches from encoder representations. If is an input image, $384$0 the set of patch indices, $384$1 the masked patches, and $384$2 the visible patches, with encoder $384$3 and decoder $384$4, the self-supervised loss is
$384$5
In words, the optimization target is the mean-squared error over masked patches between reconstructed pixels and ground-truth pixels (Xiao et al., 2022).
Several design choices are emphasized. High masking ratios around $384$6 and the heavy-encoder/light-decoder asymmetry are reported as important for efficient and effective MAE training. Minimal augmentation suffices, specifically center crops and color adjustments, and the method does not rely on strong augmentation invariances. A dummy classification token is included for downstream transfer compatibility, and the ViT classification token is later used as the downstream image embedding (Xiao et al., 2022).
3. Pre-training data and representation transfer
The pre-training datasets include a Human-Object Interaction (HOI) collection of $384$7K images and ImageNet. The HOI collection is assembled from EPIC-Kitchens with frames sampled at $384$8 fps, Something-Something with frames sampled at $384$9 fps, and approximately $12$0K images from 100 Days of Hands. MAE pre-training is run for $12$1 epochs on $12$2 GPUs for both HOI and ImageNet. For controlled comparisons, the supervised ViT-S baseline is trained on ImageNet for $12$3 epochs using the recipe of Xiao et al. (Xiao et al., 2022).
The paper assigns particular importance to the difference between object-centric image corpora and in-the-wild egocentric interaction imagery. For ImageNet MAE, the random cropping ratio is in $12$4; for HOI MAE, it is in $12$5 to accommodate wide egocentric aspect ratios. Beyond that, preprocessing is intentionally light (Xiao et al., 2022).
A central empirical claim is that HOI pre-training outperforms ImageNet MAE on $12$6 manipulation tasks. The stated explanation is that HOI contains first-person views of human-object interactions and manipulation affordances, making it closer to robot wrist-camera perspectives and more relevant to task cues such as shape, color, and grasping context. The paper accordingly argues that in-the-wild images from sources such as YouTube or egocentric videos lead to better visual representations for various manipulation tasks than ImageNet images (Xiao et al., 2022).
Transfer is unusually strict: the same MAE-pretrained ViT-S encoder is reused without any fine-tuning across reach, cabinet, pick, and relocate tasks for both the Franka and Kuka+Allegro setups. This establishes encoder reuse across multiple robots, scenes, and task families as a defining property of MVP rather than an incidental result (Xiao et al., 2022).
4. Downstream control pipeline and the PixMC benchmark
After pre-training, the encoder is frozen and used as a feature extractor for RL. The observation space consists of high-resolution wrist-mounted RGB images, by default at $12$7, together with proprioception in the form of joint positions and velocities. The ViT classification token is used as the image embedding and projected to $12$8 dimensions; the proprioceptive state is separately projected to $12$9 dimensions; the concatenation forms the controller input (Xiao et al., 2022).
The policy is a $6$0-layer MLP with hidden sizes $6$1 and SeLU activations, outputting delta joint angles for position control in joint space at $6$2 Hz. The critic uses the same architecture with separate weights. Policy optimization is performed with Proximal Policy Optimization. The return objective is
$6$3
and the PPO training objective includes the clipped surrogate, value loss, and entropy bonus:
$6$4
Here $6$5 and $6$6 is the advantage estimate (Xiao et al., 2022).
PixMC is implemented in NVIDIA IsaacGym, allowing many parallel environments and fast training. The benchmark spans $6$7 task families instantiated across both robots, yielding $6$8 tasks in total: Reach, Cabinet manipulation, Pick, and Relocate. The robot embodiments are a Franka arm with a parallel-jaw gripper and a Kuka LBR iiwa with an Allegro hand. Both setups use wrist-mounted cameras and position control in joint angle space at $6$9 Hz, and hand-engineered oracle state inputs are also provided for upper-bound comparisons (Xiao et al., 2022).
Training uses dense task-specific reward functions, but evaluation is based on reward-independent success metrics measuring sustained closeness to task goals; absolute success rate is the primary metric. Reported training times in IsaacGym are approximately 0 minutes for oracle-state models and approximately 1 hours for pixel-based models over about 2M environment steps on a single NVIDIA 2080 Ti. PPO is implemented with multi-GPU data parallelism when needed, and up to 3 environments per 2080 Ti are reported for 4 inputs (Xiao et al., 2022).
5. Empirical findings, ablations, and failure modes
The main result is that MVP significantly outperforms a supervised ImageNet-pretrained ViT on 5 tasks and matches it on the remaining task. The paper reports up to 6 absolute success-rate improvement over supervised encoders. On pick and relocate tasks for both robots, the supervised baseline is described as flat at or near zero success, while MVP attains high success and often rivals the oracle-state model. At convergence, MVP closely matches oracle-state performance on 7 of the 8 tasks (Xiao et al., 2022).
Sample-complexity comparisons show both faster learning and higher asymptotic performance for MVP relative to supervised ImageNet ViT and random frozen features. Randomly initialized frozen encoders yield zero or near-zero success on 9 tasks, which the paper interprets as evidence that meaningful pre-trained visual features are necessary for these pixel-based control problems (Xiao et al., 2022).
The ablation on pre-training data is especially decisive. HOI MAE exceeds ImageNet MAE on $4.6$0 tasks, supporting the claim that representation quality depends strongly on viewpoint and interaction alignment between pre-training images and downstream observations. By contrast, scaling the model from ViT-S to ViT-Base, at $4.6$1 G FLOPs, does not produce clear gains on pick tasks; the stated implication is that larger models may require more data or modified recipes for control (Xiao et al., 2022).
Ablations on training strategy identify encoder freezing as crucial. When the encoder is unfrozen and optimized end-to-end with RL, training is reported as unstable, with NaN losses unless the learning rate is reduced; even then, success remains at zero across seeds whether initialization is random or from MAE. The paper attributes this to instability of RL gradients disrupting pre-trained representations and to insufficient samples for training a large ViT end-to-end in an on-policy RL setting. Freezing therefore serves both representational preservation and optimization stability (Xiao et al., 2022).
Comparisons with other baselines further clarify the operating regime. Reimplemented CURL and RAD with PPO and ViT-S underperform MVP for a fixed number of steps and require more compute, specifically $4.6$2 GPUs versus $4.6$3 GPU for MVP. The paper notes that environment-only training might surpass MVP given enough data, but presents MVP as a strong baseline with lower compute. It also reports that the MAE encoder can outperform a strong CLIP visual encoder on these manipulation tasks, while MoCo-v3 achieves non-trivial performance on some tasks, indicating some generality of the broader pre-control idea (Xiao et al., 2022).
6. Generalization, limitations, and subsequent developments
MVP reports several transfer and robustness findings beyond the primary task scores. In distractor experiments on a blue $4.6$4 cm cube, the model maintains high success under color and shape changes, suggesting disentanglement of shape and color, but is more sensitive to scale variation, likely because monocular wrist-camera inputs induce scale ambiguity. When object scale changes, success is reported at approximately $4.6$5, corresponding to picking either the original object or the distractor. In a Kuka+Allegro pick task, retraining only the controller permits transfer to new YCB objects such as a can, mug, and banana with at least $4.6$6 success, indicating that the frozen visual features retain some object-geometry generality in dexterous settings (Xiao et al., 2022).
The paper also states several limitations. Performance depends on the availability and relevance of large unlabeled image corpora; static-image pre-training has limited dynamics understanding; monocular vision introduces scale ambiguity; end-to-end fine-tuning of large ViTs in RL remains unstable and sample-inefficient; and scaling to larger vision models without scaling data does not improve results. Sim-to-real transfer is described as promising in light of prior IsaacGym work, but it is not explicitly tested in the paper (Xiao et al., 2022).
Within the subsequent pre-control literature, MVP functions as an early reference point. Later work on spatiotemporal predictive pre-training argues that static-content objectives such as MAE neglect temporal motion clues in egocentric video and reports gains from jointly predicting masked current and future frames (Yang et al., 2024). A later contrastive action-image pre-training formulation argues that neither image-language supervision nor purely visual self-supervision provides paired vision-action signals at scale, and compares directly against MVP among other encoders on real dexterous manipulation tasks (Sharma et al., 15 Jun 2026). At the same time, comparative studies of pre-trained vision models for control emphasize that downstream policy learning method strongly affects which encoder appears best, and that RL-based comparisons can be highly variable (Hu et al., 2023). Related work revisiting learning-from-scratch baselines argues that frozen pre-trained encoders can be hindered by a significant domain gap between pre-training data and downstream control benchmarks, and that fine-tuning plus augmentation can partially alleviate this gap (Hansen et al., 2022).
Taken together, these later results do not negate MVP’s central contribution. They place it within a developing research program on pre-control for motor systems: large-scale pre-training of visual representations before policy learning, with careful attention to data distribution, control objective, temporal information, and the stability constraints imposed by downstream optimization.