---
title: Cortical Messaging Protocol (CMP)
url: https://www.emergentmind.com/topics/cortical-messaging-protocol-cmp
type: topic
---

# Cortical Messaging Protocol (CMP)

The Cortical Messaging Protocol (CMP) is the modality-independent interface by which all components of a Thousand Brains system—namely, sensor modules (SMs), learning modules (LMs), and motor modules—exchange spatial and object-centric information, specifically “features at poses.” CMP affords plug-and-play modularity in distributed sensorimotor architectures by permitting architectural elements to operate and communicate in standardized, serializable message formats. CMP is biologically inspired, drawing from the long-range cortico-cortical and thalamo-cortical information exchanges found in the neocortex, and implements a high-level “wire protocol” to support hierarchical, lateral (voting), and top-down exchanges of sensorimotor hypotheses, object identities, and goal states [2412.18354].

## 1. Purpose and Biological Motivations

CMP provides a unified, modality-agnostic protocol to manage the exchange of spatially anchored features and object hypotheses throughout Thousand Brains systems. The protocol standardizes the communication required for:

- Bottom-up inference: relaying sensed or inferred features and their poses (positions and orientations) from lower to higher levels.
- Lateral integration (voting): rapidly reaching consensus among parallel or redundant modules via hypothesis sharing.
- Top-down context: transmitting goal-states or priors, for example, desired poses and features, to bias inference and action selection.
- Sensor/actuator cohesion: coordinating multi-modal sensor integration and translating goals into actuator commands.

CMP is specifically motivated by biological findings that cortical columns in the neocortex exchange high-level hypotheses (e.g., object identities and pose information) without sharing their full internal models. This design reflects evidence that location-tagged hypotheses broadcast between cortical and thalamic regions facilitate efficient sensorimotor inference [2412.18354].

## 2. Structure of CMP Messages

A CMP message encapsulates information about features and their spatial context, enabling seamless integration across disparate modules. Each message comprises the following:

| Field               | Type/Structure                        | Purpose/Example                                   |
|---------------------|--------------------------------------|---------------------------------------------------|
| x                   | Vector $\in \mathbb{R}^3$            | Position in body-centric space (e.g., $(x, y, z)$)|
| R                   | Matrix $\in SO(3)$                    | Orientation: 3 orthonormal axes (e.g., $n$, $c_1$, $c_2$)|
| f                   | Discrete ID or Vector                 | Feature/object label (e.g., “mug”) or embedding   |
| $\{h_1, \ldots, h_F\}$| Array of floats (optional)         | Non-morphological features (e.g., color histogram)|
| confidence          | Float $[0,1]$                         | Belief measure for hypothesis                     |
| use_flag            | Boolean                              | $u=1$ for processing downstream                   |
| sender_id           | String                               | Module identifier                                 |
| sender_type         | Enum {SM, LM, Motor}                 | Module category                                   |

Not all fields are mandatory; optional fields can be omitted or set to null. This flexible structure allows diverse sensors and modules to participate without superfluous overhead.

```python
struct CMPMessage {
    Vector3 x;
    Matrix3 R;
    FeatureID f;
    float h[F];
    float confidence;
    bool use_flag;
    string sender_id;
    enum {SM, LM, Motor} sender_type;
};
```

## 3. Communication Topology and Connectivity

CMP supports three principal connection modalities:

- **Bottom-up (Hierarchical):** SM $\rightarrow$ LM $\rightarrow$ higher-level LM $\rightarrow$ …, with each LM treating the object ID output of its input module as a feature for its own inference.
- **Lateral (Voting, Non-hierarchical):** LM$_i \leftrightarrow$ LM$_j$, where modules exchange alternative object–pose hypotheses to speed consensus formation and error correction.
- **Top-down (Goal/Context):** LM$_n \rightarrow$ LM$_k$, transmitting expectations, priors, or goal-states to guide lower-level computation.

There is no fixed hierarchy or mandatory layering: modules connect according to simple adjacency lists, enabling arbitrary heterarchies. A single LM may interface with multiple SMs, several LMs, and the motor system simultaneously.

No routing tables or packet headers are implemented beyond `sender_id`, favoring architectural transparency and minimal communication overhead.

## 4. Operational Semantics and Protocol Dynamics

CMP communication is event-driven and synchronized to discrete time steps, usually coinciding with the agent’s movement/observation cycle. Protocol rules include:

- **Messaging Schedule:** At each time step $t$, any SM or LM with fresh information emits zero or more CMPMessages to downstream recipients. There is no global clock aside from synchronization at time steps.
- **Conflict and Consensus:** Each module integrates “evidence” for competing hypotheses, summing weighted votes. Inconsistent or low-confidence votes reduce net evidence and are pruned if below acceptance thresholds.
- **Temporal Synchronization:** Modules strictly process input messages from time $t$ before constructing outputs for time $t+1$. No asynchrony or message “lookahead” is supported.
- **Learning Isolation:** CMP carries hypotheses only—no transfer of internal parameters occurs over the protocol. Modules perform learning (e.g., Hebbian or STDP updates) internally, typically episodically after evidence integration.

## 5. Illustrative Pseudocode

A minimal forward-feedback CMP messaging loop involving two LMs can be outlined as:

```python
# Module A: update from sensor or lower LM
def step_module_A(sensor_obs, prev_obs_A):
    disp_A = sensor_obs.pose - prev_obs_A.pose
    A.update_local_hypotheses(sensor_obs.features, disp_A)
    # send current most likely hypothesis to downstream B
    msg_A = CMPMessage(
        x=A.best_pose,
        R=A.best_orientation,
        f=A.best_object_id,
        confidence=A.best_confidence,
        sender_id=A.id,
        sender_type='LM'
    )
    send(to=B, msg=msg_A)

# Module B: receive vote from A and update
def step_module_B(msgs_from_A, sensor_obs_B, prev_obs_B):
    disp_B = sensor_obs_B.pose - prev_obs_B.pose
    B.update_local_hypotheses(sensor_obs_B.features, disp_B)
    for m in msgs_from_A:
        pA_in_B = transform_pose(m.x, m.R, A_to_B_displacement)
        B.update_hypotheses_with_vote(m.f, pA_in_B, m.confidence)
    msg_B = B.compose_vote()
    send_to(peers_of_B, msg_B)
```
This pseudocode demonstrates context-driven hypothesis updating, evidence integration, and propagation of current best hypotheses in a fully modular and decentralized manner [2412.18354].

## 6. Multimodal and Hierarchical Applications

CMP enables several canonical neuro-inspired computation scenarios:

- **Multimodal Sensor Fusion:** E.g., a camera patch (SM$_1$) emits “feature=pointNormalType3” at a 3D pose; a touch patch (SM$_2$) emits “feature=curvatureHigh” at a nearly coincident pose, both with high confidence. Associated LMs exchange “mug” object hypotheses via CMP, rapidly reaching consensus after lateral voting.
- **Hierarchical Composition:** A low-level LM recognizes a “tire” at pose $p_t$, passing this via CMP to a higher-level LM, which concurrently ingests raw features (e.g., from a wide-span SM) and learns that a “car” is composed of “tire”, “window”, etc., at specific relative poses.

The protocol’s architectural neutrality allows modules representing distinct modalities, spatial fields, or abstraction levels to interoperate seamlessly.

## 7. Empirical Performance and Constraints

Empirical assessment in simulated object-recognition (Habitat environment) indicates:

- Single-LM systems required 4–6 steps to converge to an object hypothesis.
- Voting among five LMs reduced recognition to 2–3 steps on average.
- Tuning the evidence threshold parameter enables controllable tradeoff between speed (fewer steps) and classification accuracy.

No specific measurements of bandwidth or bit-rate are reported; CMP message payloads are typically 20–50 floats plus metadata, presenting negligible overhead for Python-based implementations. CMP does not specify or constrain network-level packetization, latency, nor scheduling at a bitwise or lowest-level cognitive cycle granularity.

---

All structural, procedural, and empirical descriptions are sourced directly from “The Thousand Brains Project: A New Paradigm for Sensorimotor Intelligence” [2412.18354].

Source: https://www.emergentmind.com/topics/cortical-messaging-protocol-cmp