GateFlow: Optimal Policy Gating

• GateFlow is a continuous-time, energy-based dynamical system that optimally gates policies by minimizing a free-energy functional in a mixture-of-experts framework.
• It leverages a proximal-gradient flow with a softmax-update mechanism to derive unique, globally convergent gating weights based on KL divergence minimization.
• Empirical evaluations show GateFlow’s effectiveness in multi-agent coordination and human decision-making tasks, providing robust, interpretable policy composition.

GateFlow is a continuous-time, energy-based dynamical system for optimal policy gating derived from the minimization of a free-energy functional on mixture-of-experts decision architectures. Developed as part of the GateMod computational model, GateFlow connects the task-dependent composition of multiple behavioral or control primitives to a mathematically principled, convergent dynamics. This approach provides a unified, interpretable framework for hierarchical policy composition in artificial and biological agents, with empirically demonstrated applications ranging from multi-agent collective behaviors to human exploratory decision making (Rossi et al., 4 Dec 2025).

1. Mathematical Setting and GateFrame Foundation

GateFlow originates in the GateFrame framework, which formalizes gating as the selection of mixture weights $w$ for $n$ fixed primitives $\pi^1, \ldots, \pi^n$, each representing a policy over next-state/action pairs $(X_k, U_k)$ conditional on current state $X_{k-1}$. The agent's global policy is a convex combination: $$U_k \sim \sum_{\alpha=1}^n w_\alpha\, \pi^\alpha(\cdot|X_{k-1}),\quad w \in \Delta^n \equiv \{w \mid w_\alpha \geq 0, \sum_\alpha w_\alpha = 1\}.$$

The optimal gating weights $w$ are selected by minimizing an entropy-regularized Kullback–Leibler divergence ("free-energy") between the mixture policy and a generative model $q$, subject to $w \in \Delta^n$: $$\min_{w\in\Delta^n} F(w) \equiv D_{\rm KL}\bigl(p(\cdot|x_{k-1};w) \| q(\cdot|x_{k-1})\bigr) + \varepsilon \sum_\alpha w_\alpha \ln w_\alpha,$$ where $$p(x_k, u_k|x_{k-1}; w) = \sum_\alpha w_\alpha\, \pi^\alpha(x_k, u_k|x_{k-1})$$. The entropy regularizer $\varepsilon > 0$ controls the softness of the gating.

The free-energy landscape is strongly convex in $w$ due to the convexity of $F_0(w) = D_{\rm KL}$ in $w$ and the strict concavity of the entropy; as a result, GateFrame guarantees a unique global optimum.

2. Derivation and Structure of GateFlow Dynamics

GateFlow is derived via a continuous-time proximal-gradient (forward-backward) flow for the GateFrame minimization problem. Decomposing the objective into a smooth component and a simplex-indicator-constrained nonsmooth component, one arrives at the closed-form gate dynamics: $$\tau\,\dot w = -w + \mathrm{softmax}\left(-\frac{1}{\varepsilon}\nabla_w F_0(w)\right),$$ where $\tau$ is a time constant. Expanding coordinate-wise: $$\tau\,\dot w_i = -w_i + \frac{\exp\left(-\varepsilon^{-1}\,\frac{\partial F_0}{\partial w_i}(w)\right)}{\sum_{j=1}^n \exp\left(-\varepsilon^{-1}\,\frac{\partial F_0}{\partial w_j}(w)\right)}.$$ The partial derivatives have explicit form: $$\frac{\partial F_0}{\partial w_i} = \sum_{x,u}\, \pi^i(x, u|x_{k-1}) \left[\ln\left(\sum_{\beta} w_\beta\, \pi^\beta(x, u|x_{k-1})\right) - \ln q(x, u|x_{k-1}) \right].$$ If $q(x,u|x_{k-1}) \propto \exp(-c(x,u))$, the terms become expected "cost-plus-log-density."

GateFlow, therefore, implements a softmax-weighted update of the gating distribution, where each primitive's weight is adjusted according to its contribution to reducing the KL divergence (and cost) with respect to the generative model.

3. Theoretical Properties: Contractivity and Robustness

GateFlow is globally exponentially convergent, as the underling flow is a contracting dynamical system in the Euclidean norm per Lohmiller–Slotine contraction theory. The Jacobian's symmetric part is negative-definite, leading to the following properties:

• Forward invariance: $\Delta^n$ is preserved under the flow for any initial $w(0) \in \Delta^n$.
• Exponential contraction: For any two solutions $w(t), w'(t)$,

$$\|w(t) - w'(t)\| \leq e^{-t/\tau} \|w(0) - w'(0)\|.$$

• Unique equilibrium: There exists a unique optimal $w^* \in \Delta^n$, and all trajectories converge at rate $1/\tau$.
• Robustness: Input-to-state stability entails small transient errors produce only bounded deviations.

In the $\varepsilon \rightarrow 0$ limit, GateFlow recovers hard argmax gating (sparse mixture-of-experts), while for $\varepsilon > 0$ it yields dense soft assignments.

4. Neural Circuit Realization: GateNet

GateFlow admits a mechanistically interpretable implementation as a two-layer recurrent neural circuit, "GateNet," with fast and slow dynamical components:

• Fast "gradient" unit: Computes $y = -\varepsilon^{-1} \nabla F_0(w)$ via iterative updates over the mixture densities and their log-values, using linear summation and pointwise nonlinearity.
• Slow "softmax" unit: Integrates $y$ to produce normalizing factors and the final gating weights $w$ via exponentiation and normalization.

The fast dynamics obey

$$\tau_g \dot a = -a + \Pi(x_{k-1}) w,\ \tau_g \dot b = -b + \ln(a),\ \widetilde{\tau}_g \dot y = -\varepsilon y - \Pi(x_{k-1})^{\!\top}(b + c),$$

while the slow dynamics evolve as

$$\tau_s \dot m = -m + \sum_{\alpha=1}^n e^{y_\alpha},\ \tau_s \dot r = -r + y - \mathbf{1} \ln(m),\ \tau \dot w = -w + e^r.$$

With an appropriate time scale separation ($\widetilde{\tau}_g \ll \tau_s \ll \tau$), these equations ensure that $w$ tracks the GateFlow ODE. All operations are local; the matrix-vector multiplication $\Pi(x)w$ corresponds to Sigma–Pi dendritic computations, and all state variables are nonnegative, corresponding to plausible firing rate interpretations.

5. Empirical Evaluation

GateFlow was evaluated within the broader GateMod model across two domains: multi-agent collective behavior (boid flocking) and human multi-armed bandit decision making.

Multi-Agent Coordination (Boids)

• Primitives: Social-force kernels for separation, alignment, and cohesion.
• Generative model: Matches local neighbor velocity/position statistics.
• Metrics: Polarization $P(t)$ and final distance to goal.
• Results: For $N=40$ boids and $10$ leaders ($T=100$),

$$P_{{\rm GateMod}} \approx 0.98,\quad {\rm DistToGoal}_{{\rm GateMod}} \approx 0.04,$$

compared to static equal gating $P\approx 0.85$, distance $\approx 0.25$. This demonstrates GateFlow's superior coordination and goal attainment.

Human Multi-Armed Bandits

• Primitives: Exploitation (max mean), uncertainty-seeking (max variance), risk-averse (min variance).
• Metrics: Protected Exceedance Probability (PXP) via BIC-based model selection.

Model	Experiment 1 PXP	Experiment 2 PXP
Hybrid [18]	0.32	0.38
UCB	0.25	0.27
Thompson	0.18	0.15
Value	0.10	0.08
GateMod	0.76	0.82

GateMod (with GateFlow gating) yields higher PXP in both experiments. It produces interpretable, trial-by-trial mixture weights, demonstrating dominance of exploitation when appropriate and rhythmic alternation with uncertainty-seeking under task demands.

6. Interpretation and Significance

GateFlow provides a normative account of how gating in policy composition emerges from first principles of free-energy minimization. Its global exponential stability and contractive properties ensure robust, non-pathological adaptation, making it suitable for dynamically evolving tasks and for noise-robust neural circuit implementation. The approach unifies classical mixture-of-experts, control-as-inference, and neural computation perspectives. In empirical settings, it delivers interpretable insight into internal policy arbitration and matches or exceeds established benchmarks in both collective and individual agent domains. The mechanism's connection between task structure, optimality conditions, and local recurrent computation positions it as a fundamental solution concept for neural policy composition (Rossi et al., 4 Dec 2025).