BFM-Zero: Zero-Shot RL & Gauge Theory Insights

• BFM-Zero is a dual framework that integrates zero-shot adaptation in behavioral foundation models for RL with a decoupling solution in non-Abelian gauge theory, offering novel insights in both control and QCD.
• It introduces belief-conditioned and rotation-based adaptations to overcome latent space interference, enabling effective policy generalization in multi-dynamics environments.
• In gauge theory, BFM-Zero elucidates the decoupling ghost dressing function, linking a finite infrared gluon mass with critical coupling behavior in Landau-gauge QCD.

BFM-Zero refers, in contemporary research literature, to two distinct but influential frameworks: (1) a class of zero-shot adaptation methods for Behavioral Foundation Models (BFMs) in control and reinforcement learning (RL), and (2) the “zero-momentum ghost dressing” (decoupling) solution of the ghost propagator Dyson–Schwinger equation (DSE) in the Pinch Technique–Background Field Method (PT–BFM) approach to non-Abelian gauge theory. The following entry synthesizes the mathematical formulation, algorithmic structure, and empirical significance of BFM-Zero in both modern RL/robotics (with emphasis on zero-shot control and skill embeddings) and gauge-theoretic studies (with a focus on infrared QCD Green’s functions).

1. Zero-Shot Behavioral Foundation Models: Problem Setting and FB Representation

Behavioral Foundation Models (BFMs) aim to produce control policies for arbitrary downstream tasks without task-specific fine-tuning or online learning. Zero-shot adaptation denotes the inference of such policies in new environments directly from offline data and generic model structure. The underlying mechanism in prominent BFM architectures is the Forward–Backward (FB) representation, which factorizes discounted future state occupancy for each latent policy $\pi_z$: $$M^{\pi_z}(s_0, a_0, X) \approx \int_{s^+ \in X} F(s_0, a_0, z)^\top B(s^+)\,\rho(ds^+)$$ with $F: S \times A \times Z \to \mathbb{R}^d$ the forward representation, $B : S \to \mathbb{R}^d$ the backward, $z \in Z \subset S^{d-1}$ the latent policy code, and $\rho$ the offline training state distribution.

Inference is conducted by selecting a policy vector $z$ and acting greedily: $\pi_z(s) \approx \arg\max_a\; F(s, a, z)^\top z.$ $F$ and $B$ are trained via an auxiliary Bellman-style loss that combines temporal-difference (TD) consistency and orthogonality constraints on $B$, encouraging the latent space’s directions to parametrize distinct behaviors (Bobrin et al., 19 May 2025).

However, the classic FB parameterization cannot effectively distinguish between environments or transition structures. Training on offline data from multiple dynamics $\{c_1, \dots, c_k\}$, standard FB models exhibit growing regret—the worst-case policy value gap increases monotonically with the number of dynamics $k$—and empirically display “interference” among modalities in the latent skill space.

2. BFM-Zero: Belief- and Rotation-Conditioned Extensions

To address inherent limitations of traditional FB representations, BFM-Zero introduces algorithmic innovations for dynamics-aware zero-shot generalization (Bobrin et al., 19 May 2025).

Belief-FB (BFB) modifies FB by conditioning on an explicit learned context representation. A permutation-invariant transformer encoder $f_\mathrm{dyn}$ is trained via self-supervised (variational) next-state prediction to produce a context embedding $h$ from a short, reward-free transition history $(s_t, a_t, s'_{t+1})_{t=1}^T$. Regularization ensures these embeddings are compact and informative. The forward map is then reformulated as $F(s, a, [h;z])$, concatenating $h$ and $z$ at every evaluation, allowing the decoding of context-specific successor features.

Rotation-FB (RFB) further structures the latent space using cluster-aware sampling. Instead of sampling policy codes $z$ isotropically, RFB draws $z$ in a cone around the estimated context embedding $h$, using a von Mises–Fisher distribution $z \sim \mathrm{vMF}(\mu=h, \kappa)$. This arrangement creates “latent cones” per context, avoids cross-context policy overlap, and yields a context-dependent regret bound independent of the total number of environments—thus restoring zero-shot adaptability.

The key inference pipeline is:

1. Encode recent transitions via $f_\mathrm{dyn}$ to obtain $h$;
2. Sample a batch of policy codes $z$ from $\mathrm{vMF}(h, \kappa)$;
3. For each $z$, compute $Q(s, a; h, z) = F(s, a, [h; z])^\top z$;
4. Act greedily: $\pi(s) = \arg\max_a Q(s, a; h, z)$.

No test-time adaptation or planning is needed—only prompt-style inference.

3. BFM-Zero in Humanoid Control: Architecture and Latent Skill Spaces

A second usage of BFM-Zero designates a practical framework for scalable, promptable whole-body humanoid control using unsupervised RL (Li et al., 6 Nov 2025).

In this setting, BFM-Zero learns a shared latent “skill” space $\mathcal{Z} \subset \mathbb{R}^d$ (with $d=256$) that encodes reference motions, goal states, and arbitrary reward functions into a common vector $z$. The architecture comprises:

• A backward encoder $B(s, o)$ extracting per-state features from simulator states $s$ and proprioceptive observations $o$;
• A forward predictor $F(o_{t,H}, s_t, a_t, z)$ learning successor features conditioned on both state-action histories and policy vector $z$;
• A policy actor $\pi(o_{t,H}, z)$ taking as input a short history ($H$ steps) of proprioception plus a task/skill vector $z$ and outputting control torques for the full degrees-of-freedom of the Unitree G1 humanoid.

Training employs a combination of Bellman-style losses, regularization (orthonormality of $B$), and critic-augmented objectives. The policy is trained off-policy and unsupervised, with critics accessing privileged simulation states and the actor observing only proprioceptive signals.

Latent codes are derived for:

• Goal encoding: $z_g = B(s_g, o_g)$ for a desired target pose/state,
• Reward encoding: $z_r = (1/N)\sum_i r(s_i) B(s_i, o_i)$ aggregating reward-weighted state features,
• Motion tracking: temporal sequences of $z_t = \sum_{t'} B(s_{t'}, o_{t'})$ to follow reference motion segments.

This yields a smooth, nearly Euclidean latent skill space supporting linear and spherical interpolation between semantically distinct behaviors.

4. Empirical Results and Benchmarks

In zero-shot dynamics adaptation scenarios, BFM-Zero exhibits pronounced gains over baseline approaches. For discrete and continuous (e.g., Randomized-FourRooms, Ant-Wind, Randomized-PointMass) environments, metrics such as average episode return demonstrate that BFM-Zero (both BFB and RFB) can yield up to $2\times$–$3\times$ returns over vanilla FB baselines, and consistently outperform alternative architectures such as HILP or Laplacian RL (Bobrin et al., 19 May 2025).

In real-world humanoid control, BFM-Zero on the Unitree G1 supports diverse task classes—including zero-shot reward optimization, robust whole-body motion tracking, and rapid disturbance recovery—without any post-training fine-tuning. The framework handles domain translation (sim-to-real) via domain randomization, reward shaping, and asymmetric (privileged-state) learning. Iterative few-shot latent optimization (e.g., Cross-Entropy Method) enables rapid adaptation to payload changes or terrain variation (Li et al., 6 Nov 2025).

Ablation studies confirm key hyperparameter regimes: BFM-Zero’s performance with respect to number of training contexts, transformer context length $T$, and concentration $\kappa$ for vMF sampling. Notably, performance plateaus beyond $\simeq30$ contexts, and moderate $\kappa$ balances context distinction and intra-context diversity (Bobrin et al., 19 May 2025).

5. BFM-Zero in Yang-Mills Theory: PT–BFM Decoupling Solution

Independently, “BFM-Zero” also designates the zero-momentum ghost dressing function (decoupling solution) in the PT–BFM analysis of Landau-gauge QCD Green’s functions (Rodríguez-Quintero, 2010, Rodríguez-Quintero, 2010). Defining the renormalized ghost dressing $F_R(q^2)$,

$D_{gh}(q^2) = \frac{F_R(q^2)}{q^2}$

and modeling the gluon propagator in the infrared as a simple massive pole $\Delta_R(q^2) = \frac{B(\mu^2)}{q^2 + M^2}$, one finds two types of asymptotic solutions to the ghost DSE:

• Decoupling (“BFM-Zero”): $F_R(0)$ is finite, with

$$F_R(q^2) = F_R(0)\left[1 + \frac{N_C H_1}{16\pi} \overline{\alpha}_T(0) \frac{q^2}{M^2} \left(\ln \frac{q^2}{M^2} - \frac{11}{6}\right) + \mathcal{O}(q^4)\right]$$

where $$\overline{\alpha}_T(0) = M^2 \frac{g^2_R(\mu^2)}{4\pi} F^2_R(0) \Delta_R(0)$$.

• Scaling (critical): $F_R(q^2) \sim (q^2)^{-1/2}$, corresponding to a diverging ghost dressing and an IR-exponent $\kappa=1/2$.

The critical coupling $\alpha_\mathrm{crit}$ where the decoupling branch terminates and scaling is approached is given by: $$\alpha_\mathrm{crit} \simeq \frac{5\pi}{2 N_C A^2(\mu^2) B(\mu^2)}$$ For $\alpha < \alpha_\mathrm{crit}$, decoupling solutions are observed numerically and match lattice data; as $\alpha \to \alpha_\mathrm{crit}^-$, $F_R(0)$ diverges as $\sim (\alpha_\mathrm{crit} - \alpha)^{-\kappa'}$ with secondary exponent $\kappa' \approx 0.085$ (Rodríguez-Quintero, 2010).

This framework clarifies the relationship between the presence of an infrared gluon mass, criticality in the gauge coupling, and the viability of decoupling versus scaling scenarios in nonperturbative gauge theory.

6. Significance and Outlook

BFM-Zero, in both RL/control and gauge-theoretical formulations, demonstrates how explicit structure in latent spaces or Green’s function asymptotics enables substantial gains in generalization, adaptability, and theoretical understanding. In RL/robotics, BFM-Zero sets a precedent for promptable, unsupervised, and generalist policies deployable on real hardware without online learning. In gauge theory, the “BFM-Zero” solution elucidates key nonperturbative phases and connects continuum functional approaches with lattice observations.

Methodological advances—belief conditioning, latent space clustering, structured sampling—are expected to generalize across RL domains requiring zero-shot or rapid adaptation, while the mathematical formalism of the ghost and gluon sector continues to inform infrared QCD phenomenology and emergent mass generation.

BFM-Zero Instantiation	Domain	Core Mechanism	Key Reference
RL/Control (FB Models)	Policy generalization	Belief conditioning, latent cones	(Bobrin et al., 19 May 2025, Li et al., 6 Nov 2025)
Gauge Theory (PT-BFM)	IR QCD Green’s functions	Ghost dressing at $q^2 \to 0$, massive gluon ansatz, criticality	(Rodríguez-Quintero, 2010, Rodríguez-Quintero, 2010)