---
title: 'CrafterVPT: Behavioral Foundation Model'
url: https://www.emergentmind.com/topics/craftervpt
type: topic
---

# CrafterVPT: Behavioral Foundation Model

Searching arXiv for the cited CrafterDojo paper and the original VPT/Crafter context papers.
CrafterVPT, also written **C-VPT**, is the **behavioral foundation model** in the broader **CrafterDojo** suite, introduced to provide a strong **unconditional policy prior** for the Crafter environment [2508.13530]. Within that suite, **CrafterVPT** serves as the behavioral prior, **CrafterCLIP** as the vision-language grounding component, and **CrafterSteve-1** as the instruction-following component. The model is positioned as a Crafter analogue of behavioral pretraining in Minecraft: rather than requiring higher-level agents to learn low-level motor behavior from scratch, it supplies a pretrained low-level controller that can be reused for instruction following, hierarchical planning, and reinforcement-learning fine-tuning [2508.13530].

## 1. Conceptual role within CrafterDojo

CrafterVPT was introduced because Crafter, despite being a lightweight, open-source, Minecraft-like environment, had lacked the kind of foundation-model ecosystem that had supported progress in Minecraft research [2508.13530]. The immediate motivation was threefold: to **provide a behavioral prior** for Crafter agents, to **enable hierarchical agents** in which a planner issues high-level instructions and a CrafterVPT-like controller handles primitive actions, and to **unlock prototyping-friendly embodied agent research** in a lightweight environment that avoids the engineering overhead of Minecraft [2508.13530].

This placement makes CrafterVPT more than a standalone imitation-learned policy. In the architecture of CrafterDojo, it is the initial behavioral layer on which later components depend. **CrafterCLIP** is trained on captions derived from the same CrafterPlay trajectories, and **CrafterSteve-1** is constructed by conditioning the CrafterVPT backbone on goal embeddings from CrafterCLIP, thereby converting the unconditional policy into an instruction-following agent [2508.13530]. The paper therefore treats CrafterVPT as the reusable low-level “motor system” of the suite.

A plausible implication is that CrafterVPT occupies the same systems role in Crafter that behavioral priors occupy in other hierarchical embodied-agent stacks: it amortizes low-level control and allows downstream modules to operate at a more semantic level. The paper itself stops short of presenting it as sufficient for planning-heavy tasks on its own [2508.13530].

## 2. Dataset construction and training regime

CrafterVPT is trained by **behavior cloning / imitation learning** on a synthetic expert dataset called **CrafterPlay** [2508.13530]. Because Crafter lacks large online human gameplay corpora, the dataset was built using an **Expert Behavior Generator** toolkit. The construction pipeline is explicitly described as: training an expert policy in **Craftax-Classic-Symbolic** using PPO-RNN, rolling out that expert to generate trajectories, and then using those trajectories as demonstration data for CrafterVPT [2508.13530].

The expert policy used for data generation was trained for **10B timesteps** in Craftax-Classic-Symbolic, following Craftax’s PPO-RNN protocol, with an added survival penalty to avoid collapse into poor survival behavior after all achievements were already unlocked [2508.13530]. The resulting **CrafterPlay** dataset contains **20,000 episodes**, about **180M timesteps** in the appendix, and an average length of approximately **9,012 steps per episode**. Its trajectories are written as
$$
\tau_i = \{(o_t,a_t)\}_{t=1}^{T},
$$
that is, observation-action trajectories with explicit action labels [2508.13530].

The availability of ground-truth actions is a central distinction from the original Minecraft VPT setup. CrafterVPT does **not require an Inverse Dynamics Model (IDM)** to infer actions from unlabeled video, because the synthetic CrafterPlay pipeline already provides action supervision [2508.13530]. This alters both the data-engineering pathway and the training objective: the system is not trying to reconstruct latent actions from videos, but directly fit a policy to labeled trajectories.

The paper also reports scaling observations for the upstream expert and the downstream imitation learner. Expert-policy performance continued improving until around **10B** timesteps, which justified using that endpoint for dataset generation. On the imitation side, training on larger CrafterPlay subsets improved performance and converged around **18,000 episodes** [2508.13530]. This suggests that CrafterVPT’s effectiveness is tied not only to architecture but also to the scale and quality of its synthetic demonstration corpus.

## 3. Model architecture and optimization objective

CrafterVPT uses the **identical architecture as VPT**:
$$
\begin{aligned}
\text{Image Encoder: } \hspace{0.5em} && x_t & = \text{ResNet}_\theta (o_t) \\
\text{TransformerXL: } \hspace{0.5em} && \tilde{x}_{1:t} & = \mathrm{TrXL}_\theta(x_{1:t}) \\
\text{Policy Head: } \hspace{0.5em} && a_t & \sim \pi_\theta (a_t \mid \tilde{x}_{1:t}).
\end{aligned}
$$
Here, \(o_t\) is the current observation frame, \(x_t\) is the image embedding produced by a **ResNet**, \(\tilde{x}_{1:t}\) is the contextualized history representation from **Transformer-XL**, \(a_t\) is the sampled action, and \(\pi_\theta\) is the policy parameterized by \(\theta\) [2508.13530]. In operational terms, this is a **pixel-based recurrent policy** combining an image encoder, a sequence model, and an action predictor.

Three model sizes are trained.

| Variant | Params | Recurrence layers |
|---|---:|---:|
| tiny | 3.2M | 2 |
| base | 15.9M | 4 |
| large | 64.3M | 4 |

The associated hidden dimensions are reported as follows: **tiny** uses \(d_\text{model}=128\), \(d_\text{ff}=512\), and 4 heads; **base** uses \(d_\text{model}=512\), \(d_\text{ff}=2048\), and 4 heads; **large** uses \(d_\text{model}=1024\), \(d_\text{ff}=4096\), and 8 heads [2508.13530]. The paper notes that the **largest CrafterVPT** corresponds roughly to the **smallest VPT** variant in the Minecraft literature [2508.13530].

The training objective is standard behavior cloning:
$$
\mathcal{L}_\text{cvpt} = \mathbb{E}_{(o_{1:t}, a_t) \sim \mathcal{D}_\text{play}} \left[ -\log \pi_\theta (a_t \mid o_{1:t}) \right].
$$
In this expression, \(\mathcal{D}_\text{play}\) is the CrafterPlay dataset, \(o_{1:t}\) is the observation history up to time \(t\), and \(-\log \pi_\theta(a_t \mid o_{1:t})\) is the negative log-likelihood of the expert action under the policy [2508.13530]. The objective therefore optimizes next-action likelihood under recurrent pixel-conditioned history, rather than reward maximization directly.

## 4. Preprocessing, no-op imbalance, and ablations

A major practical issue in CrafterVPT training is **noop filtering**. CrafterPlay contained about **60% no-op actions**, which the authors identify as problematic because excessive inactive steps degrade behavior cloning, while eliminating all no-ops would remove useful waiting behaviors [2508.13530]. The adopted scheme is selective: it removes **noop sequences shorter than 20 steps** and preserves longer no-op stretches, since longer idle periods can encode meaningful behaviors such as waiting for safety [2508.13530].

This preprocessing step reduces the no-op ratio to **4.6%** and improves CrafterVPT performance [2508.13530]. The appendix further quantifies the structure of the imbalance: **60%** of all recorded actions are no-op, **40.6%** of no-op actions occur during sleep periods, and sleep-related no-ops account for **24% of all recorded actions** [2508.13530]. These numbers matter because they show that the issue is not merely annotation redundancy; a substantial fraction of idle behavior is behaviorally meaningful.

The ablation is one of the clearest empirical findings in the paper. If CrafterVPT is trained **without** this filtering, its Crafter Score drops to **38.1**, about a **38% decrease** from the final model [2508.13530]. The paper explicitly interprets the chosen filtering strategy as a compromise between suppressing pathological class imbalance and preserving strategic waiting.

Other ablations reinforce the scaling picture. Performance improves as the expert policy used to generate CrafterPlay is trained longer, up to around **10B** timesteps, and it improves with more imitation data, converging around **18,000 episodes** [2508.13530]. Taken together, these results indicate that the behavioral prior is sensitive to dataset composition, expert competence, and action-frequency skew, not only to raw model size.

## 5. Evaluation protocol and quantitative performance

CrafterVPT is evaluated in the standard **Crafter benchmark** using two metrics: **Crafter Score**, defined as the geometric mean of success across the 22 achievements and scaled to \([0,100]\), and **Return**, defined as normalized cumulative reward [2508.13530]. The appendix gives the Crafter Score formula:
$$
S = \exp \left( \frac{1}{22} \sum_{i=1}^{22} \ln (1 + s_i) \right) - 1,
$$
where \(s_i\) is the success rate for achievement \(i\) [2508.13530].

The evaluation setup uses **100 episodes**, reported as **means and standard deviations over five 20-episode chunks**. Agents receive **pixel observations**, while the **expert policy** used for comparison is evaluated with **state-based observations** [2508.13530]. This setup is important when interpreting comparisons between imitation-learned pixel policies and the generator expert from which demonstrations were derived.

The main quantitative results are as follows.

| Method | Score (%) | Return (%) |
|---|---:|---:|
| Human \(\dagger\) | \(50.5 \pm 6.8\) | \(65.0 \pm 10.5\) |
| Expert Policy (CrafterPlay) | 97.5 | \(98.4 \pm 0.04\) |
| \(\Delta\)-IRIS \(\dagger\) | 9.3 | \(35.0 \pm 3.2\) |
| prior method \(\dagger\) | \(31.8 \pm 1.4\) | \(69.7 \pm 1.2\) |
| **C-VPT (tiny)** | \(52.9 \pm 4.0\) | \(66.8 \pm 0.1\) |
| **C-VPT (base)** | \(61.0 \pm 3.0\) | **\(71.8 \pm 0.1\)** |
| **C-VPT (large)** | **\(61.4 \pm 4.7\)** | \(71.3 \pm 0.1\) |

These results establish several points. First, CrafterVPT **beats all prior baselines** on both Crafter Score and Return. Second, the best Crafter Score is achieved by **C-VPT (large)** at **61.4**, while the best Return is achieved by **C-VPT (base)** at **71.8** [2508.13530]. Third, the paper highlights an improvement of **up to 29.6% in Crafter Score** over the strongest prior method cited [2508.13530].

This evaluation supports the central claim that CrafterVPT functions as a strong behavioral foundation for Crafter. At the same time, the gap to the **Expert Policy (CrafterPlay)** remains large, which is consistent with imitation from pixel observations being a compressed behavioral prior rather than a direct reproduction of the generator expert.

## 6. Emergent behaviors, downstream use, and limitations

The paper emphasizes that CrafterVPT learns more than achievement-specific traces. Reported emergent behaviors include **constructing shelters for protection**, **placing items to block hostile mobs**, **building bridges or paths over water/lava**, and **strategic waiting and sleeping for safety** [2508.13530]. These behaviors are presented as evidence that the model has acquired reusable survival and construction priors rather than merely memorizing task-specific action sequences.

Within downstream systems, CrafterVPT is used in two main ways. For long-horizon tasks, the pretrained **C-VPT (base)** is fine-tuned with **PPO + LoRA** using LoRA settings \(\alpha=16\), \(r=16\), and a KL regularization weight of **0.1**, yielding **C-VPT-RL** [2508.13530]. Separately, **CrafterSteve-1** conditions the CrafterVPT backbone on goal embeddings from **CrafterCLIP**, so that the pretrained low-level policy becomes an instruction-following controller [2508.13530]. In both cases, CrafterVPT functions as the reusable behavioral backbone.

The limitations discussed in the paper are correspondingly specific. First, **no-op imbalance** can harm learning, but overly aggressive filtering can remove useful waiting behavior [2508.13530]. Second, **behavior cloning only captures observed expert behavior**; the model is strong as a prior, but not sufficient for planning-heavy tasks by itself [2508.13530]. Third, the **unconditional policy** struggles in hierarchical experiments on harder sequential tasks, especially **T3: make wood pickaxe then obtain sapling** and **T4: obtain coal, craft wood pickaxe, then obtain stone** [2508.13530]. Fourth, the paper explicitly shows that CrafterVPT helps but **needs instruction-following and planning layers on top for multi-step goals** [2508.13530].

A common misconception would be to treat CrafterVPT as a complete general-purpose Crafter agent. The evidence presented does not support that interpretation. The model is instead a high-quality low-level prior whose principal significance lies in enabling broader embodied-agent pipelines in a lightweight environment [2508.13530]. In that sense, its contribution is architectural as well as empirical: it establishes the behavioral substrate on which the rest of CrafterDojo is built.

Source: https://www.emergentmind.com/topics/craftervpt