---
title: 'CrossBFM: Humanoid Behavior Distillation Framework'
url: https://www.emergentmind.com/topics/crossbfm
type: topic
---

# CrossBFM: Humanoid Behavior Distillation Framework

CrossBFM is a framework for distilling a shared latent behavior space across morphologically different humanoid robots. It preserves the latent coordinate system of a frozen source Behavior Foundation Model (BFM), uses frame-level retargeting to create cross-embodiment correspondence, and trains a robot-agnostic encoder to map target-robot observations into that shared space. Separate latent-conditioned trackers then convert the shared latent into whole-body control for each embodiment. The framework supports motion tracking, goal reaching, and reward optimization, and can additionally provide a common interface for flow-based generation of behavior latent trajectories [2609.38087].

The name is not used consistently across research domains. In earlier literature, “CrossBFM” has also been used informally or interpretively for cross-bifix-free codes, cross-band beamforming, cross-frequency EEG feature modeling, cross-subject Brain Foundation Model adaptation, and related mechanisms. These uses are not equivalent to the humanoid-control framework. The formal method explicitly titled “CrossBFM: Distilling a Shared Latent Behavior Space Across Humanoid Embodiments” is the humanoid robotics system described below [2609.38087].

## 1. Conceptual basis and motivation

Behavior Foundation Models provide a high-level control interface in which a latent vector represents a behavior rather than a sequence of joint commands. The latent may encode a reference motion to imitate, a target pose to reach, or a reward to maximize. A latent-conditioned policy interprets this vector together with the robot’s proprioceptive state and produces whole-body actions.

CrossBFM is based on Forward–Backward (FB) representations. For a policy $\pi$, its successor measure is

$$
M^{\pi}(X\mid s,a)=\sum_{t\geq 0}\gamma^t
\Pr(s_{t+1}\in X\mid s,a,\pi).
$$

For a reward function $r$, the corresponding value function can be expressed as

$$
Q_r^\pi(s,a)=\int_S M^\pi(\mathrm{d}s'\mid s,a)\,r(s').
$$

An FB representation approximates the successor measure through a forward map $F$, a backward map $B$, and a latent-conditioned policy $\pi_z$:

$$
M^{\pi_z}(\mathrm{d}s'\mid s,a)
\simeq F(s,a,z)^\top B(s')\,\rho(\mathrm{d}s'),
$$

$$
\pi_z(s)=\arg\max_a F(s,a,z)^\top z.
$$

The backward map converts states, goals, motions, or rewards into points in the behavior space. For a reward-conditioned behavior, the latent is

$$
z_r=\mathbb{E}_{s\sim\rho}[B(s)r(s)].
$$

CrossBFM treats this latent coordinate system as the transferable asset. Rather than independently training a complete FB representation for every robot, it freezes a source BFM and learns encoders for other robots that reproduce the source latent coordinates.

Independent BFM training has two principal disadvantages. First, a complete training run requires approximately $10^2$ GPU-hours per robot. Second, independently trained models produce embodiment-specific latent spaces: equivalent behaviors may occupy different coordinates on different robots. Consequently, a latent vector learned for one humanoid cannot automatically be interpreted by another humanoid’s policy.

The latent space is constrained to the sphere

$$
\mathcal Z=\{z\in\mathbb{R}^{d}:\|z\|=\sqrt d\},
$$

with $d=256$. Latent vectors are projected using

$$
\operatorname{proj}_{\mathcal Z}(v)=
\frac{\sqrt d\,v}{\max(\|v\|,\varepsilon)}.
$$

This spherical constraint makes latent magnitudes comparable and focuses distillation on behavioral direction.

## 2. Cross-embodiment distillation pipeline

CrossBFM consists of latent distillation followed by latent-conditioned control. An optional third stage trains a flow-based generator on the shared latent space.

The source BFM is trained on the Unitree G1. Its backward map $B_S$ is frozen, and the source latent label for a source observation $o_t^{G1}$ is

$$
z_t^\star=\operatorname{proj}_{\mathcal Z}\big(B_S(o_t^{G1})\big).
$$

A motion clip is retargeted independently to each target robot while preserving its temporal timeline and frame rate. Corresponding frames therefore describe the same underlying behavior:

$$
o_t^{G1}\longleftrightarrow o_t^{M3}
\longleftrightarrow o_t^{T1}
\longleftrightarrow o_t^{N1}.
$$

The retargeted target frame receives the source latent label $z_t^\star$. For a target embodiment $X$, the encoder solves a supervised regression problem,

$$
E_X(o_{1:N}^{X})\mapsto \hat z_{1:N},
\qquad \hat z_t\approx z_t^\star.
$$

The training objective is frame-wise cosine regression:

$$
\mathcal L_{\mathrm{align}}
=
\frac{1}{N}\sum_t
\left[1-\cos(\hat z_t,z_t^\star)\right].
$$

This replaces target-side FB training, simulator-based unsupervised reinforcement learning, adversarial objectives, and replay-buffer construction with supervised latent alignment. Retargeting functions as a correspondence oracle, so its accuracy and temporal alignment directly affect distillation quality.

### Robot-agnostic input representation

The encoder uses a canonical 163-dimensional representation per frame:

| Component | Dimension | Function |
|---|---:|---|
| Key-body positions and velocities | $24+24$ | Encode eight corresponding key bodies |
| Key-body presence mask | $8$ | Distinguish absent bodies from zero-valued states |
| Root state | $1+4+3$ | Height, quaternion, and linear velocity |
| Canonical joints and velocities | $33+33$ | Represent embodiment-independent joint slots |
| Joint presence mask | $33$ | Encode missing joints |

The eight key bodies are the left and right knees, feet, elbows, and arm tips. Missing joints are zero-padded and marked by masks. Robot lengths are normalized by median root height, and fixed name maps, sign conventions, and canonical joint ordering are used. Adding a robot requires configuration metadata rather than robot-specific network weights.

The encoder receives a temporal window of $T=64$ frames, approximately 2.13 seconds at 30 Hz. Its architecture comprises a linear projection from 163 dimensions to width $H=256$, sinusoidal positional encoding, four pre-LayerNorm Transformer layers, eight attention heads, GELU feed-forward blocks, a LayerNorm, and a linear output head producing a 256-dimensional latent. The output is projected onto $\mathcal Z$.

The default encoder uses bidirectional attention for offline processing. A causal variant uses an upper-triangular mask so that frame $t$ attends only to frames at or before $t$, enabling online deployment without future reference frames. The encoder contains no robot-specific learned parameters.

The unified encoder contains approximately 3.27 million parameters and trains in under one hour on a consumer GPU. Its purpose is not merely to approximate the source latent independently for each robot, but to enforce agreement among robots observing corresponding behaviors.

## 3. Latent-conditioned whole-body control

After distillation, CrossBFM trains a separate latent-conditioned tracker for each embodiment. The tracker receives

$$
[z_t\mid\text{proprioception}],
$$

rather than the complete reference trajectory or dense target joint configuration.

The actor’s proprioception includes base angular velocity, IMU roll and pitch, joint positions and velocities, and the previous action. It outputs joint-position targets applied as offsets from a default pose. The critic receives privileged future reference observations, root and key-body information, contacts, and randomization parameters during simulation; these inputs are unavailable to the actor at deployment.

Each tracker is trained independently in MuJoCo/mjlab using PPO with 4,096 parallel environments, 200 Hz physics, 50 Hz control, 10-second episodes, and 30,000 PPO iterations. Training uses approximately $2.9\times10^9$ environment steps and requires under 10 hours per robot on an RTX 4090. Actor and critic MLPs have widths $(512,256,128)$ and ELU activations. Principal PPO parameters include $\gamma=0.99$, $\lambda_{\mathrm{GAE}}=0.95$, clipping coefficient $0.2$, target KL $0.01$, five learning epochs, four minibatches, and entropy coefficient $0.005$.

The imitation reward includes joint, key-body, root-height, orientation, velocity, and regularization terms. Regularization includes action-rate, joint-limit, collision, torque, contact, and stability penalties.

The division of labor is therefore:

- **Distilled encoder**: maps a target robot’s observations or reference motion into the source latent coordinate system.
- **Latent-conditioned tracker**: maps the shared latent and current proprioception into robot-specific whole-body actions.
- **Retargeting**: supplies offline frame correspondence and training data.
- **Prompting mechanisms**: generate or infer latent behavior commands.

The source latent interface is shared, but execution remains embodiment-specific because each robot requires its own tracker.

## 4. Prompting modes and latent generation

CrossBFM transfers the three prompting modes associated with the source BFM.

### Motion tracking

For a retargeted reference on robot $X$, the encoder generates

$$
\hat z_t=E_X(o_{t-T+1:t}^{X}).
$$

The tracker receives this latent instead of a dense joint-space reference. On the source and target robots, the same latent coordinate system therefore describes corresponding behavior.

### Goal reaching

A target pose is placed at the end of an encoder window, producing a goal latent. Interpolation in latent space provides smooth transitions between poses. This avoids abrupt changes in joint targets that can generate large torques or early termination.

### Reward optimization

Target robots do not possess the source BFM’s state distribution $\rho$. CrossBFM estimates reward latents from target-side motion data. For cached target frames $o_i^X$,

$$
Z_i=E_X(o_{i-T+1:i}^{X}),
$$

and for an embodiment-appropriate reward $r_i$,

$$
z_{\mathrm{rew}}
=
\operatorname{proj}_{\mathcal Z}
\left(\sum_i\omega_i r_i Z_i\right),
$$

where

$$
\omega=\operatorname{softmax}(\tau r_{1:M}).
$$

Reward features are based on embodiment-robust quantities such as root height and velocity, uprightness, angular velocity, arm-tip height, and knee height. Joint-specific reward terms are discarded because joint configurations differ across robots.

Absolute reward thresholds must be adapted to robot scale. CrossBFM uses marginal or quantile transport,

$$
\theta^{(r)}
=
\Phi_r^{-1}\big(\Phi_S(\theta)\big),
$$

where $\Phi_S$ and $\Phi_r$ are empirical cumulative distributions of the relevant feature on the source and target robots. This preserves the percentile of a behavior rather than transferring a physically inappropriate absolute threshold.

### Flow-based latent trajectories

Because the latent space is shared, a single conditional rectified-flow Transformer can generate latent trajectories for all robots. It operates on chunks

$$
z_{1:H}\in\mathcal Z^H,
\qquad H=64.
$$

Conditions may include a behavior-mode embedding, a null or unconditional token, the preceding 16 latent frames, frame-aligned retargeted motion features, and potentially free-form text through a frozen T5 encoder.

For Gaussian noise $\epsilon\sim\mathcal N(0,I)$, the flow interpolant is

$$
z_t=(1-t)\epsilon+t z_1,
$$

with target velocity

$$
v^\star=z_1-\epsilon.
$$

The conditional flow-matching objective is

$$
\mathcal L_{\mathrm{FM}}
=
\mathbb E_{t,\epsilon,(z_1,c)}
\left[
\left\|v_\theta(z_t,t,c)-v^\star\right\|_2^2
\right].
$$

Sampling integrates

$$
\frac{\mathrm dz}{\mathrm dt}=v_\theta(z,t,c)
$$

from $t=0$ to $t=1$, using 20 denoising steps and classifier-free guidance. Latents are reprojected onto $\mathcal Z$ after each step. Long sequences are generated autoregressively from 64-frame chunks conditioned on recent latent history:

$$
\text{prompt}\rightarrow\text{flow model}
\rightarrow z_{1:T}\rightarrow\pi_X
\rightarrow\text{joint actions}.
$$

The generator is evaluated through closed-loop behavior produced by the trackers, including walking, running, dancing, held-out clips, and real-robot rollouts.

## 5. Embodiments, experiments, and results

The experiments use the Unitree G1 as the source robot and three target robots:

| Robot | Role | DoF | Bodies |
|---|---|---:|---:|
| Unitree G1 | Source | 29 | 32 |
| Inhouse M3 | Target | 27 | 30 |
| Booster T1 | Target | 23 | 24 |
| Fourier N1 | Target | 23 | 29 |

Motion data come from the LAFAN corpus. The encoder is trained on 30 clips totaling 195,746 frames per robot and evaluated on 10 held-out clips covering eight behavior classes. The reward-prompting suite contains 41 source rewards spanning standing, locomotion, rotation, arm raising, locomotion with arm configurations, rotation with arms, crouching, and sitting.

### Motion tracking

Latent-conditioned tracking remains close to joint-conditioned tracking:

| Robot | Joint-conditioned MAE | Latent-conditioned MAE | Gap |
|---|---:|---:|---:|
| M3 | $0.1777$ rad | $0.2024$ rad | $0.0247$ rad |
| T1 | $0.1844$ rad | $0.1901$ rad | $0.0057$ rad |
| N1 | $0.1362$ rad | $0.1391$ rad | $0.0029$ rad |

The largest tracking gap is approximately $0.025$ radians. On unseen validation clips, inferred latents add only 1.4–6.7 mm of local key-body error relative to oracle latents while maintaining balance.

### Goal reaching

Goal-reaching mean absolute errors are $0.2345$ radians for M3, $0.1967$ radians for T1, and $0.1512$ radians for N1. The corresponding degradation relative to tracking is at most $0.032$ radians. The robots reach prompted poses smoothly without falling.

### Reward optimization

All 41 reward prompts succeed on the target robots. Mean normalized return is $0.431$ for the target-side method and $0.183$ for source-side reward latents transferred using the source state distribution, a $2.35\times$ improvement. The target-side method obtains:

- **M3**: $0.508$ versus $0.145$, a $3.50\times$ improvement.
- **T1**: $0.261$ versus $0.228$, a $1.14\times$ improvement.
- **N1**: $0.523$ versus $0.177$, a $2.95\times$ improvement.

These results support computing reward latents over states reachable by the executing robot.

### Data efficiency

Using one quarter of the motion corpus increases joint MAE from $0.2132$ radians to $0.2241$ radians, approximately a 5% performance cost. At 10% of the corpus, MAE reaches $0.2706$ radians.

### Unseen-robot generalization

When the encoder is trained on two robots and evaluated on the third, performance depends on morphological similarity:

| Unseen robot | Validation cosine | Closed-loop MAE | Signal gap closed |
|---|---:|---:|---:|
| M3 | $0.6251$ | $0.1829$ rad | $89.3\%$ |
| N1 | $0.5861$ | $0.2237$ rad | $55.5\%$ |
| T1 | $0.2533$ | $0.2886$ rad | $12.8\%$ |

M3 is close to the robots seen during training, whereas T1 is smaller and morphologically more distinct. The results indicate that CrossBFM performs best as interpolation across sufficiently diverse but structurally compatible embodiments. Cosine similarity alone is not a complete predictor of closed-loop control quality: a latent with moderate similarity may still lie within a useful behavior cluster.

### Real-robot validation

The method was deployed on M3 and T1. For M3:

| Prompt | Real performance | Simulation |
|---|---:|---:|
| Joint-reference tracking | $0.1985$ rad | $0.1777$ rad |
| Latent tracking | $0.2206$ rad | $0.2024$ rad |
| Goal reaching | $0.2675$ rad | $0.2345$ rad |
| Reward optimization | $0.562$ | $0.508$ |

The real-world results preserve the ordering and behavior observed in simulation, with moderate sim-to-real degradation.

## 6. Ablations, limitations, and significance

A unified encoder produces substantially higher cross-robot agreement than independently trained encoders. Per-robot alignment is approximately $0.831$–$0.903$, while unified cross-robot agreement is approximately $0.938$–$0.965$, roughly 0.07 higher than the per-robot alternatives.

The causal encoder slightly reduces offline alignment, from $0.8695$ to $0.8420$, but maintains or improves closed-loop tracking: bidirectional MAE is $0.2068$ radians, compared with $0.2043$ radians for the causal variant. This supports causal inference for streaming deployment.

Retargeting errors are tolerated when moderate. Moderate Gaussian noise and systematic geometric errors have limited effect, and timing offsets can produce a phase-shifted latent that may be recovered through one-dimensional temporal search. Large systematic geometric errors and random per-clip timing errors are more damaging.

The method depends on several assumptions. Retargeted motions must be sufficiently accurate and temporally aligned; canonical key-body and joint correspondences must be specified; the source behavior space must contain behaviors relevant to the target robots; and target robots must be capable of physically realizing approximate versions of those behaviors.

Its main limitations are that each new embodiment still requires a separate latent-conditioned tracker and PPO run; retargeted data and configuration maps remain necessary; strong morphological changes can cause poor generalization; and the transferred behavior space is limited by the source BFM’s latent manifold. Reward transfer may require feature redesign and percentile transport. The cosine-regression objective does not always predict closed-loop behavior, and the flow generator remains limited by the diversity and quality of its latent corpus.

CrossBFM’s principal methodological significance is a change in the scalable unit of transfer. A conventional approach learns a complete BFM independently for each robot:

$$
\text{robot }X
\rightarrow
\text{expensive FB training}
\rightarrow
\text{private latent space}.
$$

CrossBFM instead uses

$$
\text{one frozen source BFM}
+
\text{retargeted correspondence}
\rightarrow
\text{shared encoder}
\rightarrow
\text{robot-specific tracker}.
$$

The expensive value-functional behavior space is learned once. New robots learn how to interpret and execute that shared behavior language. The framework therefore separates semantic behavior representation from embodiment-specific control: the latent coordinate system is shared, while the low-level realization remains adapted to each robot’s morphology.

Source: https://www.emergentmind.com/topics/crossbfm