Hierarchical Active Inference Models

• Hierarchical active inference models are generative and variational architectures that minimize free energy over multi-level latent states to model adaptive, goal-directed behavior.
• They integrate continuous inference with discrete policy selection by combining top–down priors and bottom–up prediction errors, enabling dynamic planning and online adaptation.
• Applications range from biological modeling and cognitive robotics to multi-agent coordination, demonstrating improved flexibility and sample efficiency over flat models.

Hierarchical Active Inference Models define a class of generative, variational, and control architectures dedicated to modeling adaptive, goal-directed, and context-sensitive behavior in complex systems—biological or artificial—through the recursive minimization of variational free energy over a multi-level structure of latent states. These models formalize cognition, perception, and action as recursive probabilistic inference under a factorized, layered generative model, leveraging top–down priors and bottom–up prediction errors to guide both the learning of structure and the planning of behavior across time and scale. Hierarchical active inference has been mathematically elaborated in continuous, discrete, and hybrid forms, and has found application in biological modeling, cognitive robotics, collaborative agents, navigation, and manipulation in dynamically changing environments (Priorelli et al., 2024, Ofner et al., 2018, Pöppel et al., 2021, Priorelli et al., 2024, Collis et al., 2024, Fujii et al., 1 Dec 2025).

1. Mathematical Foundations and Generative Structure

Hierarchical active inference extends the Free-Energy Principle—minimizing variational bounds on surprisal—into domains where actions, goals, and beliefs interact across interlocked temporal and spatial scales. The fundamental architecture is a stack of generative model units indexed by hierarchical level ($i$) and possibly sub-units ($j$):

$$p\left(\{x^{(i,j)}, v^{(i,j)}, o^{(i,j)}\}\right) = \prod_{i=1}^I \prod_{j=1}^J p(o^{(i,j)}|x^{(i,j)}) \, p(x^{(i,j)}|x^{(i-1,\cdot)}, v^{(i,j)}) \, p(v^{(i,j)})$$

• Each unit possesses its own sensory likelihood $g^{(i,j)}$, latent state $x^{(i,j)}$ (continuous or discrete), and dynamics $f^{(i,j)}$.
• Higher levels provide priors for lower-level units, instantiating a vertical top–down modulatory structure; lower levels report prediction errors upward.
• Hybrid units generalize this motif by gating $f^{(i,j)}$ with discrete “causes” (policies, intentions) instead of pure continuous gain fields (Priorelli et al., 2024).

The free energy at each level is

$$F = \mathbb{E}_{q(\tilde{x})} \left[ \ln q(\tilde{x}) - \ln p(\tilde{x}, \tilde{o}) \right]$$

with $\tilde{x}$ and $\tilde{o}$ denoting trajectories over latent and observed variables, respectively, and inference reducing (under the Laplace or variational mean-field approximations) to a set of coupled gradient flows over beliefs.

2. Hierarchical Architectures and Dynamic Planning

Central to these models is the recursive alternation between continuous and discrete inference:

• Intrinsic–Extrinsic modules (IE): Model articulated kinematic chains by decomposing into joint (intrinsic) and position/orientation (extrinsic) subunits. These chains can represent limbs, tools, or even other agents (Priorelli et al., 2024, Priorelli et al., 2024).
• Hybrid units: Replace continuous intention parameters with discrete hidden causes, enabling dynamic “selection” among attractor landscapes for behavior (Priorelli et al., 2024, Priorelli et al., 2024, Collis et al., 2024). This is implemented using Bayesian model reduction (BMR) over possible dynamic submodels.

The typical planning and inference cycle integrates:

Step	Operation
Continuous	Inference over $j$0 and $j$1 (or $j$2) under current prior for a duration $j$3
Accumulate	Compute log-evidence for candidate attractors or intentions ($j$4)
Compare	Perform BMR to update intentional weights
Discrete	Select policies via expected free energy $j$5
Transition	Advance to next discrete state according to plan

This structure enables online combination of rapid sensorimotor adaptation with slower deliberative planning, supporting complex strategies such as tool use, multi-agent interaction, and real-time replanning in dynamically changing contexts (Priorelli et al., 2024, Pöppel et al., 2021).

3. Hybrid Representations and Affordance Landscapes

Hierarchical active inference synthesizes both continuous (sensorimotor/trajectory) and discrete (policy/intention) inferential modes:

• Affordances are encoded as continuous attractor functions $j$6—each an intention-specific dynamical prior—and inference involves dynamically scoring the correspondence between current state evolution and these attractors.
• Selection and weighting of attractors are realized by matching prediction errors; BMR formalizes intention selection as a competition based on time-accumulated log-evidence and prior preference.
• Hybridization enables agents to “switch” between skills, goals, or plans based on discrete environmental triggers or internal schedules (Priorelli et al., 2024, Priorelli et al., 2024, Collis et al., 2024).

This affordance-centric approach allows for transparent decomposition of subgoals, modular policy composition, and context-adaptive skill chaining without reliance on explicit reward signals or reinforcement learning value backups.

4. Learning, Online Adaptation, and Multi-Agent Settings

Parameter learning and online adaptation proceed via gradient descent on the variational free energy with respect to parameters $j$7 (likelihood/dynamics weights, precisions):

$j$8

$j$9

• Online optimization of beliefs and the efficient selection of intentions/policies is the focus of current hierarchical active inference agents; deep parameter learning remains an open area (Priorelli et al., 2024, Priorelli et al., 2024, Collis et al., 2024).
• Notable examples include adaptive tool use, where body schema is extended at runtime to encompass external objects, and multi-agent inference, where the modeled agent constructs and uses a second hierarchy to infer—and coordinate with—another agent’s internal states (Priorelli et al., 2024, Pöppel et al., 2021).
• In collaborative contexts, explicit message passing and “belief resonance” enable rapid, resource-efficient joint action, outperforming traditional RL approaches in coordination and computational cost in multi-agent domains (Pöppel et al., 2021).

5. Comparison with Flat Models and Value-Based Control

Hierarchical active inference offers key advantages over both flat inference and hierarchical reinforcement learning (RL):

• Flat models cannot exploit modularity, suffering from scaling issues as degrees of freedom increase, and lack clear representational boundaries for subgoals or kinematic structure.
• Hierarchical RL incorporates options and temporally abstracted actions but separates epistemic and pragmatic drives and depends on explicit reward signals and value backups; active inference integrates these as terms in expected free energy, with epistemic (uncertainty-minimizing) and pragmatic (goal-seeking) components unified under a single principle (Priorelli et al., 2024, Priorelli et al., 2024, Collis et al., 2024).
• Predictive coding networks (PCNs) share message-passing motifs but typically lack the dynamical (plan-enabling) structure of hierarchical active inference.
• Main limitations include the need for hand-specified likelihoods and dynamics for each unit, with deep parameter identification and meta-learning remaining major engineering challenges (Priorelli et al., 2024).

6. Guidelines for Model Construction and Implementation

Canonical blueprints for hierarchical active inference agents recommend:

• Modular stacking: Build IE modules for each intrinsic coordinate and extrinsic mapping, stack to form kinematic trees, and append virtual branches for tools or external objects.
• Affordance modularity: Define reduced attractors $$p\left(\{x^{(i,j)}, v^{(i,j)}, o^{(i,j)}\}\right) = \prod_{i=1}^I \prod_{j=1}^J p(o^{(i,j)}|x^{(i,j)}) \, p(x^{(i,j)}|x^{(i-1,\cdot)}, v^{(i,j)}) \, p(v^{(i,j)})$$0 for each subgoal, allowing for flexible composition and gating via intentional selection variables.
• Hybridization: Transition from continuous gain-based selection to discrete intention gating (hybrid units) for online policy re-selection in response to environmental change.
• Coupling of discrete and continuous loops: Synchronize inference cycles, e.g., 1 kHz for continuous dynamics and ~10 Hz for discrete planning/policy selection.
• Attention and precision tuning: Adapt sensory and model precisions online to implement uncertainty-weighted attention and to balance reliance on model predictions versus incoming data.
• Explicit POMDP formalism: Top-level planning can be phrased in terms of discrete state-action-observation models, providing a clear interface for integrating prior preferences and policy evaluation (Priorelli et al., 2024, Priorelli et al., 2024).

By following these principles—recursive state factorization, modular stacking, hybrid intention gating, and integrated hierarchical planning—one constructs agents able to perform dynamic planning, exploit affordances flexibly, and adapt to novel, nonstationary environments in a unified variational framework (Priorelli et al., 2024, Priorelli et al., 2024, Collis et al., 2024).

7. Applications, Impact, and Future Directions

Hierarchical active inference models have demonstrated:

• Robustness in tool use, multi-agent settings, and dynamic bodily extension, where rapid continuous adaptation is married to slow deliberative planning (Priorelli et al., 2024, Priorelli et al., 2024).
• Superior sample efficiency and replanning ability in multi-room navigation, exploration, and collaborative problem solving compared to flat architectures and standard RL (Pöppel et al., 2021, Collis et al., 2024, Fujii et al., 1 Dec 2025, Pezzato et al., 23 Jul 2025).
• Generative model transparency and interpretability, affording explicit inspection and modification of subgoals, affordances, and hierarchical message passing.

Open challenges include scalable parameter learning, automated affordance discovery, meta-learning of model structure, and the integration of deep perceptual front-ends with modular dynamics. As these methods continue to evolve, it is expected they will inform both theoretical neuroscience (as models of brain architecture) and applied machine intelligence in embodied, context-dependent real-world control scenarios.