General Policy Composition Techniques

• General Policy Composition (GPC) is a framework that constructs a unified, actionable policy by integrating multiple pre-existing policies via score-level, gating, or modular techniques.
• It employs principled methods such as convex combinations and product-of-experts to optimize performance while ensuring safety, controllability, and cross-domain generalization.
• Empirical studies in robotics and reinforcement learning demonstrate GPC’s potential to improve accuracy and adaptability, despite challenges in calibration and conflicting objectives.

Searching arXiv for papers on General Policy Composition and closely related policy composition work. General Policy Composition (GPC) denotes a family of methods for constructing a single actionable policy by composing multiple pre-existing policies or policy-like objects, rather than training one monolithic model over pooled heterogeneous data. Across the literature, GPC appears in several formal guises: distribution-level composition of diffusion or flow-based robot policies, product-of-experts and score-space composition of heterogeneous diffusion policies, prioritized composition of soft $Q$-functions under task ordering, compositional evaluation of switching policies through geometric horizon models, modular calling of reusable policy sketches, and free-energy-based gating of policy primitives. A recurring theme is that composition is performed in a representation aligned with the underlying decision model—scores, log-densities, $Q$-functions, visitation models, or module semantics—so that independently trained components can be reused while preserving structure needed for generalization, controllability, or safety (Cao et al., 16 Mar 2025, Cao et al., 1 Oct 2025, Wang et al., 2024).

1. Distribution-level composition in generative robot policies

In recent robotics work, GPC is instantiated as test-time distribution-level composition of pre-trained generative policies. The central formulation is given by “Compose Your Policies! Improving Diffusion-based or Flow-based Robot Policies via Test-time Distribution-level Composition” (Cao et al., 1 Oct 2025), which defines GPC as a training-free framework that “forms a stronger, more adaptable policy by convexly combining the distributional scores of multiple pre-trained diffusion-based or flow-based policies, and then sampling actions from the composed score.” A closely related precursor is “Modality-Composable Diffusion Policy (MCDP) via Inference-Time Distribution-level Composition” (Cao et al., 16 Mar 2025), which composes pre-trained single-modality diffusion policies during inference without retraining.

For diffusion-based policies, the basic sampler is written as

$$\tau_{t-1} = \alpha_t \tau_t + \beta_t s_\theta(\tau_t, t) + \gamma_t \eta,\qquad \eta \sim N(0,\sigma_t^2 I),$$

and GPC composes the parent policies at the score level: $$\hat{s}_{\mathrm{comp}}(\tau_t,t)=\sum_{i=1}^{n} w_i\, s_\theta(\tau_t,t,c_i),\qquad w_i\ge 0,\quad \sum_{i=1}^{n} w_i=1.$$ The resulting action trajectory is sampled from the composed score rather than from any parent individually (Cao et al., 1 Oct 2025). In MCDP, the corresponding rule is expressed in the common $\epsilon$-parameterization: $$\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)=\sum_{i=1}^{n} w_i\,\epsilon_\theta(\tau_t,t,\mathcal{M}_i),\qquad \sum_i w_i=1,$$ with the reverse update

$$\tau^{t-1}=\alpha^t\big(\tau^t-\gamma^t\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)+\xi\big),\qquad \xi\sim \mathcal{N}(0,\sigma_t^2 I).$$

The paper emphasizes that this is CFG-free and avoids additional training or architectural modification of the parent experts (Cao et al., 16 Mar 2025).

These methods treat policy composition as an operation on conditional trajectory distributions rather than as feature concatenation or action averaging. In MCDP, each expert models $p_\theta(\tau\mid o^{(m)})$ for a single modality such as RGB or point cloud, and GPC combines these experts “in a principled, distribution-level manner” to improve “cross-modality, cross-domain, and cross-embodiment” generalization (Cao et al., 16 Mar 2025). In the broader GPC formulation, heterogeneous parents may include vision-action and vision-language-action policies, as well as diffusion-based and flow-matching policies, provided their outputs can be mapped into a common score representation (Cao et al., 1 Oct 2025).

2. Mathematical principles and theoretical guarantees

A major line of GPC work grounds composition in product-of-experts or additive score algebra. MCDP states the product-of-experts view explicitly: $$p_{\mathrm{comp}}(\tau)\propto \prod_i p_i(\tau\mid o^{(m_i)})^{w_i},$$ so that

$$\log p_{\mathrm{comp}}=\sum_i w_i \log p_i + C,\qquad \nabla_\tau \log p_{\mathrm{comp}}=\sum_i w_i \nabla_\tau \log p_i.$$

Under score-based diffusion, adding scores corresponds, up to scale, to adding noise predictions, which yields the practical inference rule used by MCDP (Cao et al., 16 Mar 2025). PoCo adopts an analogous score-space formulation for heterogeneous diffusion policies and analytic costs: $Q$0 with the corresponding policy-level product-of-experts form

$Q$1

(Wang et al., 2024).

Theoretical guarantees are developed most explicitly in the 2025 GPC paper (Cao et al., 1 Oct 2025). Its first proposition establishes single-step improvement via convex combination: if two score estimators have different bias and noise components, the convex combination

$Q$2

has mean-squared error

$Q$3

with a minimizer $Q$4 satisfying

$Q$5

with strict inequality unless the errors are perfectly aligned. This formalizes the claim that convex averaging can cancel misaligned error components (Cao et al., 1 Oct 2025).

The same paper then proves a score-to-sample stability result using a Grönwall-type bound. For dynamics

$Q$6

if the estimated score $Q$7 has uniformly bounded error $Q$8 and the dynamics satisfy Lipschitz conditions, then terminal sampling error is bounded by

$Q$9

A corollary states that if convex score composition reduces integrated score MSE, then it reduces terminal trajectory error as well (Cao et al., 1 Oct 2025). This is the strongest formal result among the cited generative-policy GPC papers.

Other GPC formulations rely on different mathematical objects. “Generalised Policy Improvement with Geometric Policy Composition” composes geometric horizon models (GHMs) to evaluate non-Markov geometric switching policies without additional learning (Thakoor et al., 2022). Given a switching sequence $$\tau_{t-1} = \alpha_t \tau_t + \beta_t s_\theta(\tau_t, t) + \gamma_t \eta,\qquad \eta \sim N(0,\sigma_t^2 I),$$0, it shows that careful composition of base-policy GHMs yields an unbiased estimator of $$\tau_{t-1} = \alpha_t \tau_t + \beta_t s_\theta(\tau_t, t) + \gamma_t \eta,\qquad \eta \sim N(0,\sigma_t^2 I),$$1, after which generalized policy improvement (GGPI) produces a new Markov policy satisfying

$$\tau_{t-1} = \alpha_t \tau_t + \beta_t s_\theta(\tau_t, t) + \gamma_t \eta,\qquad \eta \sim N(0,\sigma_t^2 I),$$2

for suffix-closed collections $$\tau_{t-1} = \alpha_t \tau_t + \beta_t s_\theta(\tau_t, t) + \gamma_t \eta,\qquad \eta \sim N(0,\sigma_t^2 I),$$3 of switching policies (Thakoor et al., 2022). By contrast, “Towards Task-Prioritized Policy Composition” composes maximum-entropy soft $$\tau_{t-1} = \alpha_t \tau_t + \beta_t s_\theta(\tau_t, t) + \gamma_t \eta,\qquad \eta \sim N(0,\sigma_t^2 I),$$4-functions with state-action dependent indifference weights,

$$\tau_{t-1} = \alpha_t \tau_t + \beta_t s_\theta(\tau_t, t) + \gamma_t \eta,\qquad \eta \sim N(0,\sigma_t^2 I),$$5

where the weight is constructed from the Jacobian and Hessian of the higher-priority $$\tau_{t-1} = \alpha_t \tau_t + \beta_t s_\theta(\tau_t, t) + \gamma_t \eta,\qquad \eta \sim N(0,\sigma_t^2 I),$$6, with the goal of preserving high-priority constraints locally (Rietz et al., 2022).

3. Heterogeneity, modularity, and representation choices

A defining feature of GPC is that the constituent experts need not share identical inputs or training data. PoCo addresses “heterogeneous robot learning” by training separate diffusion policies on simulation, human video, and real-robot data, across RGB, depth/point-cloud, tactile, and proprioceptive modalities, provided all policies share the same action space: “6-DoF end-effector velocity: xyz + roll, pitch, yaw” (Wang et al., 2024). Composition is then performed in score-space over trajectories rather than over observations. This allows domain-level fusion of separate policies trained on different modalities and domains (Wang et al., 2024).

MCDP makes the same design choice in a more specific setting. It leverages standard image-based DP and point-cloud-based DP3 policies from RoboTwin and composes them at inference time without architectural changes (Cao et al., 16 Mar 2025). Practical composition requires scheduler alignment: in the RoboTwin experiments, DP3’s DDIM with 10 steps was replaced by DDPM with 100 steps so that the image and point-cloud experts shared compatible $$\tau_{t-1} = \alpha_t \tau_t + \beta_t s_\theta(\tau_t, t) + \gamma_t \eta,\qquad \eta \sim N(0,\sigma_t^2 I),$$7 and update magnitudes (Cao et al., 16 Mar 2025). The broader GPC paper generalizes this normalization principle by converting heterogeneous outputs—$$\tau_{t-1} = \alpha_t \tau_t + \beta_t s_\theta(\tau_t, t) + \gamma_t \eta,\qquad \eta \sim N(0,\sigma_t^2 I),$$8-prediction, $$\tau_{t-1} = \alpha_t \tau_t + \beta_t s_\theta(\tau_t, t) + \gamma_t \eta,\qquad \eta \sim N(0,\sigma_t^2 I),$$9-prediction, $$\hat{s}_{\mathrm{comp}}(\tau_t,t)=\sum_{i=1}^{n} w_i\, s_\theta(\tau_t,t,c_i),\qquad w_i\ge 0,\quad \sum_{i=1}^{n} w_i=1.$$0-prediction, or flow-style representations—into a common score representation before composition (Cao et al., 1 Oct 2025).

GPC also appears in explicitly symbolic or programmatic forms. “On Policy Reuse: An Expressive Language for Representing and Executing General Policies that Call Other Policies” extends a sketch language with internal finite memory states, indexical features over dynamically loaded object registers, and reusable modules that call other modules with parameter passing (Bonet et al., 2024). Here composition is not probabilistic but semantic: a module

$$\hat{s}_{\mathrm{comp}}(\tau_t,t)=\sum_{i=1}^{n} w_i\, s_\theta(\tau_t,t,c_i),\qquad w_i\ge 0,\quad \sum_{i=1}^{n} w_i=1.$$1

encapsulates arguments, memory, registers, features, and rules; call rules transfer control to a callee module, and do rules select ground actions directly. Composition is thus realized as nested calls with stack-based scoping, proper returns, and width-bounded search-based execution (Bonet et al., 2024).

A different representational axis is policy gating. “Neural Policy Composition from Free Energy Minimization” models the composed policy as a convex mixture of primitives,

$$\hat{s}_{\mathrm{comp}}(\tau_t,t)=\sum_{i=1}^{n} w_i\, s_\theta(\tau_t,t,c_i),\qquad w_i\ge 0,\quad \sum_{i=1}^{n} w_i=1.$$2

with the weights $$\hat{s}_{\mathrm{comp}}(\tau_t,t)=\sum_{i=1}^{n} w_i\, s_\theta(\tau_t,t,c_i),\qquad w_i\ge 0,\quad \sum_{i=1}^{n} w_i=1.$$3 optimized by minimizing a free-energy objective subject to the simplex constraint (Rossi et al., 4 Dec 2025). The equilibrium satisfies

$$\hat{s}_{\mathrm{comp}}(\tau_t,t)=\sum_{i=1}^{n} w_i\, s_\theta(\tau_t,t,c_i),\qquad w_i\ge 0,\quad \sum_{i=1}^{n} w_i=1.$$4

so task structure, encoded through the generative model and cost, determines the gate rather than the gate being fixed heuristically (Rossi et al., 4 Dec 2025). This suggests a general distinction within GPC between composing distributions directly and learning or computing a gating distribution over reusable primitives.

4. Empirical evidence across robotics and reinforcement learning

The empirical record for GPC is strongest in robot policy composition. On RoboTwin manipulation tasks, MCDP composes image-based DP and point-cloud-based DP3 and reports several cases where the composed policy exceeds both parents. For example, on Empty Cup Place, the unimodal experts achieve $$\hat{s}_{\mathrm{comp}}(\tau_t,t)=\sum_{i=1}^{n} w_i\, s_\theta(\tau_t,t,c_i),\qquad w_i\ge 0,\quad \sum_{i=1}^{n} w_i=1.$$5 and $$\hat{s}_{\mathrm{comp}}(\tau_t,t)=\sum_{i=1}^{n} w_i\, s_\theta(\tau_t,t,c_i),\qquad w_i\ge 0,\quad \sum_{i=1}^{n} w_i=1.$$6 success, while MCDP reaches $$\hat{s}_{\mathrm{comp}}(\tau_t,t)=\sum_{i=1}^{n} w_i\, s_\theta(\tau_t,t,c_i),\qquad w_i\ge 0,\quad \sum_{i=1}^{n} w_i=1.$$7 at $$\hat{s}_{\mathrm{comp}}(\tau_t,t)=\sum_{i=1}^{n} w_i\, s_\theta(\tau_t,t,c_i),\qquad w_i\ge 0,\quad \sum_{i=1}^{n} w_i=1.$$8; on Shoe Place, it improves from $$\hat{s}_{\mathrm{comp}}(\tau_t,t)=\sum_{i=1}^{n} w_i\, s_\theta(\tau_t,t,c_i),\qquad w_i\ge 0,\quad \sum_{i=1}^{n} w_i=1.$$9 and $\epsilon$0 to $\epsilon$1 at $\epsilon$2 (Cao et al., 16 Mar 2025). The same study also documents failure regimes: when one expert is much weaker, the composed policy may not exceed the stronger parent, as in Pick Apple Messy and Block Hammer Beat (Cao et al., 16 Mar 2025).

The later GPC paper broadens the evidence base. On Robomimic and PushT, convex score composition yields average improvements such as DP+MP from $\epsilon$3 and $\epsilon$4 to $\epsilon$5, Florence-D+DP to $\epsilon$6, Florence-F+FP to $\epsilon$7, and $\epsilon$8+FP to $\epsilon$9 (Cao et al., 1 Oct 2025). On RoboTwin 2.0, the base policies have average success rates $$\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)=\sum_{i=1}^{n} w_i\,\epsilon_\theta(\tau_t,t,\mathcal{M}_i),\qquad \sum_i w_i=1,$$0 for DPimg, $$\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)=\sum_{i=1}^{n} w_i\,\epsilon_\theta(\tau_t,t,\mathcal{M}_i),\qquad \sum_i w_i=1,$$1 for DPpcd, and $$\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)=\sum_{i=1}^{n} w_i\,\epsilon_\theta(\tau_t,t,\mathcal{M}_i),\qquad \sum_i w_i=1,$$2 for RDT; convex composition raises these to $$\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)=\sum_{i=1}^{n} w_i\,\epsilon_\theta(\tau_t,t,\mathcal{M}_i),\qquad \sum_i w_i=1,$$3, $$\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)=\sum_{i=1}^{n} w_i\,\epsilon_\theta(\tau_t,t,\mathcal{M}_i),\qquad \sum_i w_i=1,$$4, and $$\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)=\sum_{i=1}^{n} w_i\,\epsilon_\theta(\tau_t,t,\mathcal{M}_i),\qquad \sum_i w_i=1,$$5 for the evaluated pairs (Cao et al., 1 Oct 2025). Real-world robotic evaluations on 20 trials per task show GPC improving over both DPimg and DPpcd on Place Bottles, Hang Mug, Clean Table, and Punch Holes (Cao et al., 1 Oct 2025).

PoCo provides empirical support for heterogeneous-domain and behavior-level composition. In simulation, adding smoothness or workspace costs reduces their respective metrics while slightly reducing success: normal success $$\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)=\sum_{i=1}^{n} w_i\,\epsilon_\theta(\tau_t,t,\mathcal{M}_i),\qquad \sum_i w_i=1,$$6, +smoothness success $$\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)=\sum_{i=1}^{n} w_i\,\epsilon_\theta(\tau_t,t,\mathcal{M}_i),\qquad \sum_i w_i=1,$$7 with smoothness reduced from $$\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)=\sum_{i=1}^{n} w_i\,\epsilon_\theta(\tau_t,t,\mathcal{M}_i),\qquad \sum_i w_i=1,$$8 to $$\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)=\sum_{i=1}^{n} w_i\,\epsilon_\theta(\tau_t,t,\mathcal{M}_i),\qquad \sum_i w_i=1,$$9, and +workspace success $$\tau^{t-1}=\alpha^t\big(\tau^t-\gamma^t\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)+\xi\big),\qquad \xi\sim \mathcal{N}(0,\sigma_t^2 I).$$0 with workspace reduced from $$\tau^{t-1}=\alpha^t\big(\tau^t-\gamma^t\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)+\xi\big),\qquad \xi\sim \mathcal{N}(0,\sigma_t^2 I).$$1 to $$\tau^{t-1}=\alpha^t\big(\tau^t-\gamma^t\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)+\xi\big),\qquad \xi\sim \mathcal{N}(0,\sigma_t^2 I).$$2 (Wang et al., 2024). In real multi-task composition with classifier-free guidance $$\tau^{t-1}=\alpha^t\big(\tau^t-\gamma^t\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)+\xi\big),\qquad \xi\sim \mathcal{N}(0,\sigma_t^2 I).$$3, PoCo improves average success to $$\tau^{t-1}=\alpha^t\big(\tau^t-\gamma^t\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)+\xi\big),\qquad \xi\sim \mathcal{N}(0,\sigma_t^2 I).$$4 relative to $$\tau^{t-1}=\alpha^t\big(\tau^t-\gamma^t\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)+\xi\big),\qquad \xi\sim \mathcal{N}(0,\sigma_t^2 I).$$5 for multi-task unconditioned and $$\tau^{t-1}=\alpha^t\big(\tau^t-\gamma^t\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)+\xi\big),\qquad \xi\sim \mathcal{N}(0,\sigma_t^2 I).$$6 for multi-task conditioned, though single-task specialists remain strongest at $$\tau^{t-1}=\alpha^t\big(\tau^t-\gamma^t\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)+\xi\big),\qquad \xi\sim \mathcal{N}(0,\sigma_t^2 I).$$7 (Wang et al., 2024). In hammering, domain composition of simulation and real data achieves $$\tau^{t-1}=\alpha^t\big(\tau^t-\gamma^t\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)+\xi\big),\qquad \xi\sim \mathcal{N}(0,\sigma_t^2 I).$$8 across four generalization axes, exceeding the human-only, simulation-only, and real-only policies (Wang et al., 2024).

Outside generative robotics, the evidence is more varied but still supportive. In multi-objective RL, “Policy composition in reinforcement learning via multi-objective policy optimization” uses teacher-policy adherence objectives inside MO-MPO, enabling sequential and parallel composition in humanoid and point-mass domains (Mishra et al., 2023). The paper reports that teacher policies can markedly speed learning, especially in sparse-reward settings, but also notes that strong adherence can cap final performance if teachers conflict with the optimal task policy (Mishra et al., 2023). In geometric policy composition, depth-2 GGPI on sparse-reward MuJoCo Ant outperforms standard GPI, with approximately $$\tau^{t-1}=\alpha^t\big(\tau^t-\gamma^t\hat{\epsilon}_{\mathcal{M}^*}(\tau_t,t)+\xi\big),\qquad \xi\sim \mathcal{N}(0,\sigma_t^2 I).$$9 success at 100 samples and favorable pairwise comparisons across seeds and episodes (Thakoor et al., 2022). In object-centric policy refactorization for compositional generalizability, the object-centric GNN student with self-supervised proposals outperforms CNN and RelationNet baselines on Multi-MNIST, BigFish, and FallingDigit under changes in object count and background, though Pacman shows that object-centric GNN is not always the strongest student architecture (Mu et al., 2020).

5. Constraints, priorities, and security-oriented composition

Not all GPC work is aimed at performance improvement alone. Several formulations focus on safety, ordering, or hard constraints. In prioritized reinforcement learning, the objective is explicitly asymmetric: task $p_\theta(\tau\mid o^{(m)})$0 has higher priority than $p_\theta(\tau\mid o^{(m)})$1, and the compound reward is

$p_\theta(\tau\mid o^{(m)})$2

with $p_\theta(\tau\mid o^{(m)})$3 suppressing the lower-priority objective whenever it would considerably worsen the higher-priority return (Rietz et al., 2022). The paper interprets this as an RL analogue of null-space control and introduces the “indifference-space” of a policy to identify actions that do not materially degrade the high-priority task (Rietz et al., 2022). This is a distinctly non-symmetric conception of GPC: not consensus among peers, but filtered insertion of lower-priority objectives into the admissible slack of a dominant task.

PoCo adds another constrained composition mechanism by incorporating analytic energy terms such as smoothness and workspace safety directly into the composed score (Wang et al., 2024). This permits behavior-level composition in addition to domain- and task-level composition. Because $p_\theta(\tau\mid o^{(m)})$4 is added in score-space, the method can shape the policy at inference time without retraining. This suggests a general GPC pattern in which learned experts and analytic constraints coexist as additive terms in a shared representation.

A more recent non-robotic extension appears in “Securing Multi-Tool AI Agent Chains With Dynamic, Real-Time Compositional Policies” (Schneider et al., 3 Jul 2026). Here GPC is defined over per-tool security policies for tool-using agents rather than over action distributions. The Dynamic Security Control Compositor (DSCC) computes an effective chain-level policy by combining control bindings, classification levels, prohibitions, zones, and time-to-live via monotone operators such as max, intersection, and min. The paper proves a monotonicity invariant: if a tool chain $p_\theta(\tau\mid o^{(m)})$5 is extended to $p_\theta(\tau\mid o^{(m)})$6, then the composed policy can only become more restrictive, never weaker (Schneider et al., 3 Jul 2026). Runtime taint tracking then further tightens the effective policy as more sensitive data are touched. This is a different domain from robot control, but it retains the core GPC idea of composing local policies into a chain-level policy with explicit semantics and guarantees.

6. Limitations, failure modes, and open directions

Across the literature, GPC methods inherit limitations from the representations they compose. Distribution-level score composition requires calibration and alignment. MCDP identifies expert miscalibration—different $p_\theta(\tau\mid o^{(m)})$7 schedules or score scales—as a source of dominance or cancellation, and notes that conflicting signals and sensitivity to manually tuned weights can degrade performance (Cao et al., 16 Mar 2025). The broader GPC paper similarly notes that gains saturate when parent errors are nearly identical, while poor parents can harm performance if assigned large weights (Cao et al., 1 Oct 2025). Superposition operators such as logical AND and logical OR can produce larger gains in some settings, but they require per-step reweighting and are less directly applicable to flow-based models (Cao et al., 1 Oct 2025).

PoCo emphasizes that heterogeneous pooling of all data into one shared model can perform poorly despite larger model size; in its ablation, the data-pooling baseline is reported at “$p_\theta(\tau\mid o^{(m)})$8 success vs composed policies” (Wang et al., 2024). The same paper records sensitivity to domain-composition weights and failure modes under conflicting distributions, such as bad point-cloud segmentation against RGB priors (Wang et al., 2024). In multi-objective teacher composition, teachers can act as beneficial priors early in learning yet impose performance ceilings if their behavior is suboptimal for the final task, so annealing or state-dependent teacher weighting becomes necessary (Mishra et al., 2023).

Prioritized and geometric formulations have their own restrictions. Task-prioritized composition depends on differentiability of the higher-priority $p_\theta(\tau\mid o^{(m)})$9-function with respect to actions and can become expensive in high-dimensional action spaces because Jacobians and Hessians are required (Rietz et al., 2022). Geometric policy composition requires fixed switching probability $$p_{\mathrm{comp}}(\tau)\propto \prod_i p_i(\tau\mid o^{(m_i)})^{w_i},$$0, suffix-closed candidate sets for the main guarantee, and sufficiently accurate GHMs; deeper composition expands the candidate set combinatorially and increases computation (Thakoor et al., 2022). GateMod assumes convexity of the free-energy objective in the gating weights, entropic regularization $$p_{\mathrm{comp}}(\tau)\propto \prod_i p_i(\tau\mid o^{(m_i)})^{w_i},$$1, and time-scale separation in its neural implementation (Rossi et al., 4 Dec 2025).

Several future directions recur across papers. The generative robotics literature repeatedly points to automatic or online weight selection, uncertainty-aware weighting, calibration estimation, and stronger theory for stability under conflicting experts (Cao et al., 16 Mar 2025, Cao et al., 1 Oct 2025). PoCo suggests distilling composed policies into faster architectures to address diffusion-sampling cost, and extending composition to more temporally mismatched domains (Wang et al., 2024). GateMod proposes broader regularizers and gating rules beyond softmax and integration with learning of transition kernels or primitives (Rossi et al., 4 Dec 2025). DSCC identifies non-monotonic constraints and safe exception mechanisms as open problems for compositional security (Schneider et al., 3 Jul 2026). Taken together, these directions suggest that GPC remains an active research area whose central open problem is not merely how to combine policies, but how to do so with calibration, guarantees, and manageable compute across increasingly heterogeneous experts and deployment constraints.