Text-Aligned Variational Behavioral Bottleneck

• The paper introduces a VAE-style latent compression mechanism that converts long BFM policy latents into compact behavioral programs, preserving executable control signals.
• Methodology focuses on token- and frame-level contrastive text alignment combined with policy-aware reconstruction to maintain behavioral fidelity.
• Empirical evidence demonstrates improved semantic consistency and compositional planning in long text-to-motion generation tasks with optimal compression regimes.

Text-Aligned Variational Behavioral Bottleneck denotes a sequence-VAE-style latent compression model over Behavioral Foundation Model (BFM) policy latents, augmented with policy-aware reconstruction and token/frame-level contrastive text alignment. In Text2BFM, it is the representation layer that compresses long sequences of executable BFM latents into shorter behavioral programs that remain decodable into BFM commands while becoming more compatible with natural language. The mechanism was introduced in the context of long text-to-motion generation, where it enables motion synthesis to proceed as compact behavioral planning rather than direct pose prediction (Shvetsov et al., 28 May 2026).

1. Conceptual definition and representational role

The bottleneck sits between a frozen pretrained BFM and a lightweight language-conditioned generator. Its input is a tracked sequence of BFM policy latents $z_{1:T_z}$, and its output is a shorter latent program $m_{1:T_m}$ with $T_m < T_z$. The encoder defines a posterior over compact programs, and the decoder reconstructs an executable latent trajectory: $$z_{1:T_z} \;\rightarrow\; q_\phi(m_{1:T_m}\mid z_{1:T_z}) \;\rightarrow\; m_{1:T_m} \;\rightarrow\; D_\theta(m_{1:T_m})=\hat z_{1:T_z}.$$ In this formulation, the bottleneck is “behavioral” because it operates on control-side latents rather than on pose vectors or motion tokens, and because reconstruction is constrained not only numerically but also at the level of the BFM policy induced by those latents (Shvetsov et al., 28 May 2026).

The surrounding BFM defines a latent-conditioned control law

$$a_t \sim \pi_{\mathrm{BFM}}(a_t \mid s_t, z_t), \qquad s_{t+1} \sim p_{\mathrm{env}}(s_{t+1} \mid s_t, a_t),$$

where $z_t$ is a local behavioral latent. For each training trajectory $\tau=\{s_t\}_{t=1}^T$, Text2BFM first infers a policy-latent sequence $z_{1:T_z}$, with $T_z=T-1$, using frozen MetaMotivo/Forward-Backward machinery. The bottleneck then compresses these latents into a shorter program $m$, so that the text-conditioned model generates plans in a compact behavioral manifold rather than generating raw motion directly (Shvetsov et al., 28 May 2026).

A common misconception is that the bottleneck is a text-conditioned VAE over poses. It is not. It is a VAE-style latent model over BFM policy-latent sequences; text does not directly parameterize the posterior or prior of the bottleneck. Instead, text alignment shapes the geometry of the compact program space, and a separate conditional generator later maps text to that space (Shvetsov et al., 28 May 2026).

2. Formalization over BFM policy latents

The tracked policy latents are computed from future states by the frozen backward map $m_{1:T_m}$0 and then projected onto a normalized latent sphere: $m_{1:T_m}$1

$m_{1:T_m}$2

This makes the bottleneck operate on future-conditioned executable commands rather than on framewise appearance descriptors (Shvetsov et al., 28 May 2026).

The bottleneck encoder is a Gaussian posterior over latent sequences: $m_{1:T_m}$3 with prior $m_{1:T_m}$4. The decoder reconstructs executable policy latents through

$m_{1:T_m}$5

The appendix describes the implementation as a causal 1D convolutional encoder-decoder with residual temporal blocks and hierarchical downsampling, with temporal compression factor $m_{1:T_m}$6, latent dim $m_{1:T_m}$7, input width $m_{1:T_m}$8, stride per level $m_{1:T_m}$9, residual depth $T_m < T_z$0, dropout $T_m < T_z$1, $T_m < T_z$2, $T_m < T_z$3, and $T_m < T_z$4 (Shvetsov et al., 28 May 2026).

The reconstruction objective is explicitly behavior-preserving: $T_m < T_z$5 The first term penalizes latent mismatch, while the second penalizes mismatch between the action distributions induced by reconstructed and original policy latents. This is the defining distinction between a merely latent-preserving bottleneck and a behavior-preserving bottleneck (Shvetsov et al., 28 May 2026).

The KL regularizer is the standard VAE rate term,

$T_m < T_z$6

and the appendix interprets the model as a rate–distortion tradeoff: $T_m < T_z$7 The full Text2BFM objective augments this with semantic supervision: $T_m < T_z$8 The paper characterizes this composite as a text-aligned variational behavioral bottleneck rather than a plain sequence VAE (Shvetsov et al., 28 May 2026).

3. Text alignment as token-aware, phase-aware contrastive geometry

Text alignment is achieved through a shared embedding space with token-level matching. Text is represented as a token sequence $T_m < T_z$9. Motion-program tokens $$z_{1:T_z} \;\rightarrow\; q_\phi(m_{1:T_m}\mid z_{1:T_z}) \;\rightarrow\; m_{1:T_m} \;\rightarrow\; D_\theta(m_{1:T_m})=\hat z_{1:T_z}.$$0 and text tokens $$z_{1:T_z} \;\rightarrow\; q_\phi(m_{1:T_m}\mid z_{1:T_z}) \;\rightarrow\; m_{1:T_m} \;\rightarrow\; D_\theta(m_{1:T_m})=\hat z_{1:T_z}.$$1 are projected and normalized as

$$z_{1:T_z} \;\rightarrow\; q_\phi(m_{1:T_m}\mid z_{1:T_z}) \;\rightarrow\; m_{1:T_m} \;\rightarrow\; D_\theta(m_{1:T_m})=\hat z_{1:T_z}.$$2

Implementation details report text token dim $$z_{1:T_z} \;\rightarrow\; q_\phi(m_{1:T_m}\mid z_{1:T_z}) \;\rightarrow\; m_{1:T_m} \;\rightarrow\; D_\theta(m_{1:T_m})=\hat z_{1:T_z}.$$3, context dim $$z_{1:T_z} \;\rightarrow\; q_\phi(m_{1:T_m}\mid z_{1:T_z}) \;\rightarrow\; m_{1:T_m} \;\rightarrow\; D_\theta(m_{1:T_m})=\hat z_{1:T_z}.$$4, and a text adapter of width $$z_{1:T_z} \;\rightarrow\; q_\phi(m_{1:T_m}\mid z_{1:T_z}) \;\rightarrow\; m_{1:T_m} \;\rightarrow\; D_\theta(m_{1:T_m})=\hat z_{1:T_z}.$$5, depth $$z_{1:T_z} \;\rightarrow\; q_\phi(m_{1:T_m}\mid z_{1:T_z}) \;\rightarrow\; m_{1:T_m} \;\rightarrow\; D_\theta(m_{1:T_m})=\hat z_{1:T_z}.$$6, heads $$z_{1:T_z} \;\rightarrow\; q_\phi(m_{1:T_m}\mid z_{1:T_z}) \;\rightarrow\; m_{1:T_m} \;\rightarrow\; D_\theta(m_{1:T_m})=\hat z_{1:T_z}.$$7 (Shvetsov et al., 28 May 2026).

For motion token $$z_{1:T_z} \;\rightarrow\; q_\phi(m_{1:T_m}\mid z_{1:T_z}) \;\rightarrow\; m_{1:T_m} \;\rightarrow\; D_\theta(m_{1:T_m})=\hat z_{1:T_z}.$$8 from sample $$z_{1:T_z} \;\rightarrow\; q_\phi(m_{1:T_m}\mid z_{1:T_z}) \;\rightarrow\; m_{1:T_m} \;\rightarrow\; D_\theta(m_{1:T_m})=\hat z_{1:T_z}.$$9 against text $$a_t \sim \pi_{\mathrm{BFM}}(a_t \mid s_t, z_t), \qquad s_{t+1} \sim p_{\mathrm{env}}(s_{t+1} \mid s_t, a_t),$$0, the score is

$$a_t \sim \pi_{\mathrm{BFM}}(a_t \mid s_t, z_t), \qquad s_{t+1} \sim p_{\mathrm{env}}(s_{t+1} \mid s_t, a_t),$$1

This is a softmax-pooled similarity over text tokens with normalization by $$a_t \sim \pi_{\mathrm{BFM}}(a_t \mid s_t, z_t), \qquad s_{t+1} \sim p_{\mathrm{env}}(s_{t+1} \mid s_t, a_t),$$2, preventing long descriptions from receiving inflated scores (Shvetsov et al., 28 May 2026).

The model then computes frame-importance weights

$$a_t \sim \pi_{\mathrm{BFM}}(a_t \mid s_t, z_t), \qquad s_{t+1} \sim p_{\mathrm{env}}(s_{t+1} \mid s_t, a_t),$$3

and aggregates them into a motion-text similarity

$$a_t \sim \pi_{\mathrm{BFM}}(a_t \mid s_t, z_t), \qquad s_{t+1} \sim p_{\mathrm{env}}(s_{t+1} \mid s_t, a_t),$$4

The resulting semantic similarity is explicitly token-aware and phase-aware rather than a single pooled sentence-level regression (Shvetsov et al., 28 May 2026).

The semantic loss is bidirectional contrastive: $$a_t \sim \pi_{\mathrm{BFM}}(a_t \mid s_t, z_t), \qquad s_{t+1} \sim p_{\mathrm{env}}(s_{t+1} \mid s_t, a_t),$$5

$$a_t \sim \pi_{\mathrm{BFM}}(a_t \mid s_t, z_t), \qquad s_{t+1} \sim p_{\mathrm{env}}(s_{t+1} \mid s_t, a_t),$$6

with $$a_t \sim \pi_{\mathrm{BFM}}(a_t \mid s_t, z_t), \qquad s_{t+1} \sim p_{\mathrm{env}}(s_{t+1} \mid s_t, a_t),$$7 a learnable logit scale, and

$$a_t \sim \pi_{\mathrm{BFM}}(a_t \mid s_t, z_t), \qquad s_{t+1} \sim p_{\mathrm{env}}(s_{t+1} \mid s_t, a_t),$$8

This is contrastive shared latent-space alignment with local token/frame matching. The paper is explicit that the bottleneck is not a conditional VAE in the usual sense; text does not define the posterior or prior of $$a_t \sim \pi_{\mathrm{BFM}}(a_t \mid s_t, z_t), \qquad s_{t+1} \sim p_{\mathrm{env}}(s_{t+1} \mid s_t, a_t),$$9, but instead regularizes the geometry of $z_t$0-space so that a separate generator can later map text into that space (Shvetsov et al., 28 May 2026).

4. Temporal compression, behavioral planning, and compositionality

The bottleneck is applied to the whole policy-latent trajectory: $z_t$1 so it is a temporally compressed latent sequence rather than a global code or a set of framewise independent latents. The paper argues that BFM latents encode behavioral intent and are future-averaged, so they are smoother than pose trajectories and therefore more compressible. The appendix formalizes this through total variation,

$z_t$2

and states that a bounded-variation latent trajectory can be approximated by a piecewise-constant sequence with bounded rollout error under Lipschitz closed-loop dynamics. This supports the use of compact latent plans as surrogates for longer executable behaviors (Shvetsov et al., 28 May 2026).

The planning/execution split is the central representational claim. Planning occurs in the compact program space $z_t$3, while execution occurs after decoding back into $z_t$4 and rolling out the frozen BFM policy. This separation underwrites the paper’s “plan, don’t pose” formulation: the text-conditioned model generates a short behavioral program, and the frozen BFM handles low-level realization, contact-rich execution, and physically plausible control (Shvetsov et al., 28 May 2026).

The same representation also supports compositional prompting. In the Text2BFM-Compose variant, one compact program is generated per clause, decoded to policy latents, concatenated, and blended at segment boundaries: $z_t$5 with boundary blending

$z_t$6

This compositional procedure depends on the bottleneck representing motion as a sequence of local behavior phases rather than as a monolithic pose rollout (Shvetsov et al., 28 May 2026).

5. Training procedure, inference path, and empirical evidence

Training is stage-wise. First, a pretrained BFM policy is trained separately on HY-Motion via MetaMotivo and then kept frozen. Second, for each paired text-motion example $z_t$7, the system infers $z_t$8, trains the bottleneck encoder $z_t$9, decoder $\tau=\{s_t\}_{t=1}^T$0, and semantic projections $\tau=\{s_t\}_{t=1}^T$1, and optimizes

$\tau=\{s_t\}_{t=1}^T$2

Third, after bottleneck training, the decoder $\tau=\{s_t\}_{t=1}^T$3 is frozen and a text-conditioned flow model is trained in bottleneck space using

$\tau=\{s_t\}_{t=1}^T$4

with objective

$\tau=\{s_t\}_{t=1}^T$5

At inference, one samples $\tau=\{s_t\}_{t=1}^T$6, solves

$\tau=\{s_t\}_{t=1}^T$7

decodes $\tau=\{s_t\}_{t=1}^T$8 into $\tau=\{s_t\}_{t=1}^T$9, and rolls out the frozen BFM policy with those decoded latents (Shvetsov et al., 28 May 2026).

The bottleneck is directly tested in ablations. On HumanML3D, average pooling yielded FID $z_{1:T_z}$0, MM-Dist $z_{1:T_z}$1, and R-Prec $z_{1:T_z}$2; no compression yielded FID $z_{1:T_z}$3, MM-Dist $z_{1:T_z}$4, and R-Prec $z_{1:T_z}$5; VBB without semantic loss yielded FID $z_{1:T_z}$6, MM-Dist $z_{1:T_z}$7, and R-Prec $z_{1:T_z}$8; and the full text-aligned VBB yielded FID $z_{1:T_z}$9, MM-Dist $T_z=T-1$0, and R-Prec $T_z=T-1$1. The comparison between “VBB w/o semantic loss” and “Text-aligned VBB” isolates the effect of $T_z=T-1$2: R-Precision improves from $T_z=T-1$3 to $T_z=T-1$4, MM-Dist from $T_z=T-1$5 to $T_z=T-1$6, and FID from $T_z=T-1$7 to $T_z=T-1$8 (Shvetsov et al., 28 May 2026).

Compression-factor studies further delimit the useful regime. At $T_z=T-1$9 compression, reconstruction is $m$0 and Action KL is $m$1; at $m$2, reconstruction is $m$3 and Action KL is $m$4; at $m$5, reconstruction is $m$6 and Action KL is $m$7. The paper concludes that $m$8 and $m$9 preserve behavior well, while $m_{1:T_m}$00 overcompresses and degrades policy fidelity (Shvetsov et al., 28 May 2026).

On the main benchmarks, Text2BFM achieves R-Precision Top-3 $m_{1:T_m}$01 and MM-Dist $m_{1:T_m}$02 on HumanML3D, and R-Precision Top-3 $m_{1:T_m}$03 and MM-Dist $m_{1:T_m}$04 on KIT-ML. The paper notes that the method is weaker on FID than some pose-space baselines, attributing that to the frozen BFM prior and domain bias. This makes the empirical profile specific: the bottleneck most clearly improves semantic consistency and compositional ordering rather than pose-distribution matching alone (Shvetsov et al., 28 May 2026).

6. Intellectual context, adjacent formulations, and scope

The Text-Aligned Variational Behavioral Bottleneck combines ingredients that appear separately in earlier literatures. Variational Information Bottleneck work provides the basic template of maximizing relevance information while penalizing code-input information, exemplified by the objective $m_{1:T_m}$05 and its variational lower bound with tractable $m_{1:T_m}$06 and $m_{1:T_m}$07 (Chalk et al., 2016). The Variational Deficiency Bottleneck shifts from information sufficiency to channel approximation and gives a decision-theoretic excess-risk interpretation, $m_{1:T_m}$08, which makes the behavioral reading of bottlenecks explicit (Banerjee et al., 2018). In emergent communication, VQ-VIB imposes a three-way tradeoff between utility, informativeness, and complexity, and can be interpreted as a variational behavioral bottleneck over communicative acts (Tucker et al., 2022).

Text-specific bottlenecking has taken several forms. DB-VAE introduced a discrete latent bottleneck for text generation to mitigate posterior collapse under strong autoregressive decoders (Zhao et al., 2020). NVIB and NVAE reinterpreted Transformer embeddings as mixture distributions and regularized cross-attention memory through Bayesian nonparametrics, thereby bottlenecking both the number of accessible vectors and the information in each vector (Henderson et al., 2022). Later work used NVIB to induce increasing levels of textual abstraction across Transformer layers (Behjati et al., 2023) and to construct privacy-preserving stochastic transformer embeddings with a utility–privacy tradeoff (Zein et al., 5 Jan 2026). In multimodal extraction, MMIB combined variational bottlenecks with a mutual-information-based alignment regularizer for text-image consistency (Cui et al., 2023). Text2BFM differs from these by making the latent object a compact sequence of executable BFM commands rather than a text-only or text-image representation (Shvetsov et al., 28 May 2026).

The scope of the Text-Aligned Variational Behavioral Bottleneck is also delimited by its dependence on the frozen BFM. If the requested behavior is not in the BFM latent space, neither the bottleneck nor the generator can invent it reliably. The paper also notes difficulty with rare motions, acrobatic actions, object-dependent interactions, and motions underrepresented in the BFM or motion data. Its alignment mechanism is token/frame-level and contrastive, but it does not explicitly ground clauses to exact temporal spans with supervision; very long $m_{1:T_m}$09 composite prompts remain challenging even though the composition variant helps (Shvetsov et al., 28 May 2026).

A final distinction concerns the term “alignment bottleneck.” In a separate line of work, “The Alignment Bottleneck” models the human-feedback loop as a constrained channel $m_{1:T_m}$10 and is not explicitly variational in the classical Information Bottleneck optimization sense (Cao, 19 Sep 2025). By contrast, the Text-Aligned Variational Behavioral Bottleneck is an internal latent representation model with an explicit VAE-style KL term, a policy-aware distortion term, and a semantic contrastive term. The two notions are related by their information-constraining perspective, but they operate at different interfaces: one at the human-feedback channel, the other at the text-to-behavior latent interface.