---
title: Schema-based Hierarchical Active Inference
url: https://www.emergentmind.com/topics/schema-based-hierarchical-active-inference-s-hai-826296a9-58df-4fe8-a891-542cae3fcdd2
type: topic
---

# Schema-based Hierarchical Active Inference

Schema-based Hierarchical Active Inference (S-HAI) is a computational framework integrating schema theory, hierarchical predictive coding, and active inference to support rapid generalization, flexible behavior, and neural coding of abstract task structures. S-HAI provides a normative, mechanistic account for how abstract relational structures—schemas—can be formed, reused, and remapped to concrete experiences via hierarchical generative modeling and free-energy–minimizing inference. The architecture has demonstrated behavioral and neural plausibility across domains such as spatial navigation, control, robotics, and dynamic multi-entity interaction [2601.18946][2409.01066][2507.17338][2402.10088][2309.09864].

## 1. Conceptual Foundations and Hierarchical Generative Model

S-HAI models cognition and control as inference under hierarchical generative models, with schemas at higher levels encoding abstract relational or structural knowledge, and subordinate levels grounding these abstractions in sensorimotor or spatial specifics. A minimal S-HAI comprises at least two levels:

- **Level 1** (concrete navigation or control): Discrete hidden states $s^1_t$ (e.g., spatial locations), observations $o^1_t$ (e.g., tile color), and atomic actions $a^1_t$ (e.g., up, down, left, right), governed by POMDP dynamics.
- **Level 2** (schema): Abstract hidden states $s^2_t$ representing schema phases or relational goals (e.g., A, B, C, D), observations $o^2_t$ identifying the current schema phase or goal, and abstract actions $a^2_t$ (e.g., "advance phase").

Crucially, S-HAI introduces a **grounding likelihood**, $A^{(2 \to 1)}$, mapping abstract schema observations to specific states at the lower level, parameterized as a Dirichlet-matrix and learned online [2601.18946]. This mapping supports "schema remapping," allowing abstract task structure to be rapidly rebound to new physical contexts.

For domains with both continuous and discrete states, recent S-HAI variants use rSLDS or discrete-continuous hybrid modules, with schemas emerging as dynamical switching modes [2409.01066][2402.10088].

## 2. Variational Inference, Learning Rules, and Policy Selection

Inference in S-HAI minimizes variational free energy separately at each hierarchical level. The variational posterior $Q(\tilde{s}^i)$ approximates $P(\tilde{s}^i | \tilde{o}^i, \tilde{a}^i)$, with free energy
\[
F = E_Q[\ln Q(\tilde{s}) - \ln P(\tilde{s},\tilde{o},\tilde{a})] = \mathrm{KL}[Q(\tilde{s})\|P(\tilde{s})] - E_Q[\ln P(\tilde{o}|\tilde{s})] + \text{const}.
\]

Beliefs about hidden states are updated via message passing, and the grounding likelihood $A^{(2 \to 1)}$ parameters are updated with conjugate Dirichlet increments:
\[
\alpha^{\text{new}} = \alpha^{\text{old}} + \eta \,\hat{s}^1_t \otimes o^2_t,
\]
where $\hat{s}^1_t$ is the inferred spatial reward location, and $\eta$ is a learning rate [2601.18946].

Policies $\pi$ (action sequences) are sampled according to their expected free energy:
\[
Q(\pi) \propto \exp\{-\gamma [G(\pi) + H(\pi)]\},
\]
with $G(\pi)$ reflecting epistemic (information gain) and extrinsic (reward) terms, and $H(\pi)$ imposing inductive costs such as target-seeking regularization.

In hybrid continuous-discrete S-HAI, schema transitions are chosen to minimize a free-energy bound integrating discrete information gain, utility, and cached subgoal costs, with rSLDS-based message passing for mode inference and Laplace-EM for continuous state estimation [2409.01066][2402.10088].

## 3. Schema Formation, Remapping, and Generalization

S-HAI operationalizes schema formation and reuse through hierarchical POMDPs and nonparametric mixture models over grounding likelihoods:

- **Fast schema remapping:** Abstract schema phases can be rebound to entirely novel lower-level arrangements with minimal data, by updating $A^{(2\to1)}$ after encountering new reward-location pairings [2601.18946].
- **Clone-structured schema graphs:** In tasks with ambiguous goal identities (e.g., repeated goals in one location), S-HAI-2C extensions use a clone-structured causal graph (CSCG) to maintain phase-specific state distinctions for disambiguation [2601.18946].
- **Assimilation vs. accommodation:** A Mixture of Grounding Likelihoods (MoGL) extension allows the agent to maintain multiple $A^{(2\to1)}$ components, with nonparametric spawning of new mappings when likelihood drops below a threshold. This enables reuse of old schemata on repeated blocks and creation of new mappings for novel contexts [2601.18946].

In hybrid domains, rSLDS architectures discover schema-like dynamical modes, supporting temporally-abstracted subgoals, information-theoretic exploration, and rapid "lifting" from low-level control to abstract planning [2409.01066][2402.10088].

## 4. Behavioral and Cognitive Validation

S-HAI has been evaluated in tasks requiring generalization, remapping, and hierarchy:

- **Spatial navigation ("ABCD" task):** S-HAI achieves near-optimal rapid generalization, outperforming hierarchical active inference without schema mechanisms (HAI), especially on blocks with novel reward-well mappings [2601.18946].
- **Alias-disambiguation ("ABCB" tasks):** S-HAI-2C agents uniquely disambiguate phase-aliased goals via cloned schema states, rapidly converging to optimal performance [2601.18946].
- **Sparse-reward control:** S-HAI with rSLDS mode discovery solves sparse-reward continuous control benchmarks (e.g., Continuous Mountain Car) more efficiently than standard RL, exploiting schema subdivisions for exploration and goal-reaching [2409.01066].
- **Hierarchical navigation in vision-based domains:** Multi-layer S-HAI achieves >90% coverage and high success rates in multi-room grid environments, outperforming count-based and curiosity-driven RL [2309.09864].
- **Long-horizon mobile manipulation:** S-HAI architectures integrating skill-level schema planning, retry modules, and continuous active inference achieve top performance in realistic rearrangement tasks, demonstrating flexibility, adaptability, and on-the-fly composition [2507.17338].

## 5. Neural Correspondence and Theoretical Implications

S-HAI explicitly predicts and replicates neural codes observed in mammalian frontal cortex and hippocampus:

- **Goal-progress cells:** The expectation of the inductive cost $H_t(s) = -\ln P(s_t)$ ramps as the agent approaches goals, reproducing ramping activity in medial prefrontal cortex populations [2601.18946].
- **Goal-identity cells:** Schema-level posteriors remain stable during traversal to each abstract goal, generating task-invariant, goal-selective activity.
- **Conjunctive cells:** Joint encoding via $A^{(2\to1)}$ parameters yields neurons selective for both abstract goal identity and spatial location, paralleling mPFC–hippocampal conjunction cells.
- **Place-like cells:** One-hot Level-1 state encodings replicate classic hippocampal place cell responses.

The framework aligns with data showing both structure-invariant and context-specific neural populations, and supports the hypothesis that predictive processing and active inference, implemented hierarchically, underlie schema-based learning, abstraction, and generalization [2601.18946].

## 6. Extensions and Applications

S-HAI generalizes to a spectrum of schema-rich domains:

- **Hybrid body–tool–object systems:** Deep S-HAI layers support composition of body schemas, tool affordances, and object-action relationships, integrating discrete policy inference with fast, continuous predictive-coding in multiple kinematic chains [2402.10088].
- **Robotic control:** S-HAI enables hierarchical skill selection, robust failure handling, and whole-body continuous inference, facilitating complex manipulation and rearrangement in realistic, online robotic settings [2507.17338].
- **Spatial and temporal map learning:** Cognitive map–based S-HAI architectures autonomously grow a graph of allocentric schemas, associating allocentric and egocentric representations, supporting efficient exploration, generalization, and scalable memory usage [2309.09864].

A summary of hierarchical organizations in representative S-HAI implementations:

| Paper          | Level 1            | Level 2 / 3      | Schema Mechanism                |
|----------------|--------------------|------------------|---------------------------------|
| [2601.18946]   | Spatial navigation | Schema phase     | Grounded POMDP + A^{(2→1)}      |
| [2409.01066]   | rSLDS continuous   | Discrete schema  | Learned mode switching          |
| [2507.17338]   | Whole-body control | Skill/retry/nav  | Discrete skill POMDP, retry     |
| [2309.09864]   | Egocentric state   | Alloc/Map node   | Growing graph of schemas        |
| [2402.10088]   | Hybrid dynamics    | Discrete policy  | Schema function composition     |

## 7. Broader Significance and Future Directions

S-HAI bridges behavioral, computational, and neural levels of analysis by making schemas explicit, hierarchically embedded computational objects subject to online Bayesian inference and learning. The framework provides:

- **Rapid, flexible generalization** by decoupling abstract structure from context-specific realization,
- **Efficient structure learning and memory use** by modularizing, growing, and merging schematic representations,
- **Interpretability** through mapping between computational states and observed neural codes,
- **Scalability** to hybrid, multi-entity, and continuous action-perception domains.

Current research extends S-HAI to deeper hierarchies, nonparametric schema discovery, and complex multi-agent cognition [2601.18946][2409.01066][2309.09864]. Empirically validated by behavioral and neural data, S-HAI establishes a unifying computational paradigm for schema-driven, hierarchical inference across cognitive science, neuroscience, AI, and robotics.

Source: https://www.emergentmind.com/topics/schema-based-hierarchical-active-inference-s-hai-826296a9-58df-4fe8-a891-542cae3fcdd2