LeabraTI Architecture: Temporal Prediction Model

• LeabraTI is a biologically oriented computational model that uses alternating prediction and sensation phases to learn invariances from spatiotemporal sensory input.
• It integrates a detailed six-layer neocortical microcircuit with neural dynamics and oscillatory gating mechanisms to implement robust sensory prediction.
• The architecture employs both error-driven and predictive learning rules along hierarchical simulations to achieve invariant object recognition and temporal integration.

LeabraTI (Temporal Integration) is a biologically grounded computational framework designed to implement sensory prediction in laminated neocortical circuits. It leverages the intrinsic spatiotemporal structure of the environment, using alternating cycles of prediction and sensation at the frequency of the brain's posterior alpha rhythm (~10 Hz), to learn and represent invariances in dynamic sensory input such as object features across changes in space and time. The architecture integrates detailed microcircuit organization, predictive learning rules, oscillatory gating mechanisms, and hierarchical simulation setups, enabling robust prediction and learning from spatiotemporal sequences (Wyatte, 2014).

1. Microcircuit Architecture and Connectivity

LeabraTI is instantiated within the canonical six-layer neocortex. The model organizes each region into distinct laminae, each with specific computational roles:

• Layers 2/3 (superficial): Primary conduit for feedforward and feedback projections across areas.
• Layer 4: Receives thalamic (sensory) afferents, functioning as the input gate, especially during sensory bursts ("plus-phase").
• Layer 5: Subdivided into regular-spiking (5a) cells serving local integration and intrinsic bursting (5b) neurons driving periodic gating at ~10 Hz.
• Layer 6: Facilitates corticothalamic feedback; maintains sustained depolarization to support predictions in superficial layers via meta-botabotropic glutamate receptor (mGluR)-mediated excitation.

The intra-areal circuit follows the loop: 4 → 2/3 → 5 → 6 → thalamus (and back to 4), with Layers 5→6 encoding temporal context, and 6→4 closing the loop either directly or via transthalamic relay, especially during prediction (minus-phase). Inter-areal connectivity features feedforward projections from superficial 2/3 in area N to layer 4 in area N+1, and feedback projections from layer 6 of area N+1 to superficial 2/3 of area N.

Gating between prediction and sensation is regulated by thalamic burst cells and layer 5b neurons: periodic bursts (every ~100 ms) restore full sensory drive (plus-phase), while in the interceding interval, sensory drive is suppressed, and deep layer 6 predictions modulate superficial activation (minus-phase) (Wyatte, 2014).

2. Predictive Dynamics and Learning Equations

LeabraTI employs both traditional Leabra error-driven learning and a temporally reversed (predictive) delta rule for temporal context integration:

• Neural activation: For each unit $j$,

$g_j^E = \sum_i w_{ij} x_i$

$y_j = \frac{1}{1 + \exp(-\gamma(g_j^E - \theta))}$

Point-unit formulations incorporate separate excitatory/inhibitory conductances and a kWTA competition mechanism.

• Error-driven learning (standard Leabra delta rule):

$\Delta w_{ij} = x_i^- (y_j^+ - y_j^-)$

Here, "−" denotes minus-phase and "+" plus-phase activation.

• Inverted LeabraTI learning rule (predictive context learning):

$$\Delta w_{ij}^{TI} = \text{super}_i^{t-1,+} \left( \text{super}_j^{t,+} - \text{super}_j^{t,-} \right)$$

Synaptic changes associate features at $t-1$ with prediction errors at $t$, supporting sequential learning.

• Synaptic weight scaling (modeling prolonged learning):

$$w_{ij}^{\text{new}} = \alpha w_{ij}^{\text{old}},\quad \alpha > 1$$

Used as a proxy for strengthening connections after extended self-organization (Wyatte, 2014).

3. Alpha Rhythm Gating and Cycle Implementation

Time in LeabraTI is discretized into alternating minus (prediction) and plus (sensation) phases, each lasting ~50 ms. This matches the biological alpha rhythm (10 Hz):

• Minus phase: Sensory input is suppressed; layer 6 context drives prediction in superficial layers 2/3.
• Plus phase: Thalamic and layer 5b burst neurons temporarily lift the sensory gate; new input resets predictions in layers 4→2/3.
• At each cycle, bursts in layer 5b update the context signal in layer 6 for the next round of prediction.

Legend of phases:

• Minus-phase: L6 context → L2/3, sensory input inhibited (thalamic quiescence).
• Plus-phase: Thalamic burst → L4 → L2/3; L5b bursts update L6 context (Wyatte, 2014).

4. Network Simulation and Training Procedures

LeabraTI’s implementation includes a hierarchical vision-like architecture and explicit training/testing protocols:

Layer	Organization & Inputs	Function
Preproc (V1)	24×24 images, 4 orientations × 2 polarities Gabor	Oriented edge code
Primary (V1)	24×24 columns, 4×2 filters per location	Retinotopic mapping
Secondary	6×6 topographic, pooling over 8×8 V1 patches	Spatial pooling
Output	10×10 localist units	Object identity

Training follows these steps:

1. Feature pre-training: 30-view object rotation in 12° increments.
2. Minus phase: 50 cycles, context only, no new sensory input.
3. Plus phase: Clamp current view and output identity, run 20 cycles.
4. Compute weight updates via LeabraTI or standard delta rule.
5. Copy superficial rates to L6 context.
6. Repeat for 20 epochs, halving learning rate every 8 epochs.

Testing involves four conditions: fully coherent in space/time (S+T+), spatial only (S+T–), temporal only (S–T+), and fully random (S–T–). Unpredictability is simulated by random view order (spatial) or skipping L5→6 updates (temporal). Prediction accuracy is monitored via minus/plus cosine similarity in early visual areas (Wyatte, 2014).

5. Algorithmic LeabraTI Cycle

A LeabraTI cycle interleaves prediction and sensation as follows:

$g_j^E = \sum_i w_{ij} x_i$0

This loop alternates prediction (minus) and sensation (plus), implements learning, and updates context for temporal prediction (Wyatte, 2014).

6. Spatiotemporal Prediction and Invariant Object Recognition

LeabraTI builds viewpoint invariance by strengthening associations between sequential features under spatially and temporally coherent conditions:

• Spatial coherence (S+): Enables Hebbian and error-driven updates relating features at $t$ and $t-1$.
• Temporal coherence (T+): Aligns learning updates with intrinsic alpha clock intervals (~100 ms).
• With repeated exposure to S+T+ sequences, descending projections in higher visual areas develop viewpoint-invariant codes.
• Such codes feed back to earlier visual areas, influencing even low-level retinotopic representations toward the 3D object structure.

Behavioral implications: Partial invariance enhances recognition of novel, static object views. However, prolonged S+T+ exposure and associated synaptic weight scaling can result in over-generalization, decreasing discriminability for ambiguous or degenerate views due to increased confusion among similar features (Wyatte, 2014).

7. Microcolumn Schematic

A microcolumn schematic illustrates the LeabraTI architecture:

$g_j^E = \sum_i w_{ij} x_i$1

Legend: Minus-phase drives prediction via L6 context to L2/3; plus-phase involves sensory input via L4 to L2/3, with L5b→L6 updating context. The alpha oscillatory gating (~10 Hz) is critical for cyclical error-driven learning of sequential features (Wyatte, 2014).