---
title: Group Event Transformer (GET)
url: https://www.emergentmind.com/topics/group-event-transformer-get
type: topic
---

# Group Event Transformer (GET)

The Group Event Transformer (GET) is an event-based vision Transformer backbone designed for neuromorphic sensors, notably event cameras. Unlike image-based backbones, GET explicitly decouples spatial information from temporal and polarity properties inherent to event streams. Its core innovations include the Group Token representation, the Event Dual Self-Attention (EDSA) block, and the Group Token Aggregation (GTA) module. These components collectively enable effective hierarchical feature extraction across both spatial and group (time-polarity) domains, delivering state-of-the-art performance on event-driven classification and detection benchmarks [2310.02642].

## 1. Event Representation and Group Token Construction

GET operates on the raw asynchronous streams produced by event cameras, in which each event is recorded as a 4-tuple \( E_i = (x_i, y_i, t_i, p_i) \), where \( (x_i, y_i) \) denotes pixel location, \( t_i \) is the timestamp, and \( p_i \in \{0,1\} \) encodes polarity. To realize explicit decoupling of spatial and temporal–polarity information, GET discretizes time into \( K \) uniform bins \( \{T_1, \dots, T_K\} \) and partitions events further by polarity, forming groups per bin and polarity (\( G = 2K \)).

Grouping is formalized as
\[
\mathcal{E}_j = \{ E_i \mid t_i \in T_j,\, p_i = P_j \}
\]
where each group collects events in one time bin and one polarity.

Spatial discretization relies on patching: the image (\( H \times W \)) is divided into \( P \times P \) non-overlapping patches. Each event is mapped into a unique 1-D bin index using its spatial, temporal, and polarity components:
\[
\begin{aligned}
    d_{t,i} &= \Big\lfloor K \cdot \frac{t_i-t_0}{t_{\rm end}-t_0+1} \Big\rfloor, \\
    pr_i    &= (x_i \bmod P) + (y_i \bmod P)\,P, \\
    pos_i   &= \Big\lfloor \frac{x_i}{P} \Big\rfloor + \Big\lfloor \frac{y_i}{P} \Big\rfloor\, \frac{W}{P}, \\
    \ell_i  &= (KHW) p_i + (HW) d_{t,i} + \frac{HW}{P^2} pr_i + pos_i.
\end{aligned}
\]
Events are accumulated into two dense vectors for each bin: an unweighted count and a time-normalized sum. The resulting representation, after reshaping and concatenation, is a tensor of shape
\[
\left(\tfrac{H}{P} \times \tfrac{W}{P}\right) \times (2K \cdot 2P^2).
\]

A \(3 \times 3\) group convolution and MLP further embed these patch-prior group features into the Group Token Embedding: \( \tfrac{H}{P} \times \tfrac{W}{P} \) tokens, each of dimension \( G \cdot C \) (with \( C \) channels per group).

## 2. Event Dual Self-Attention Block

The EDSA block is central to GET’s hierarchical feature extraction. Each block implements two parallel multi-head self-attention operations—SSA (spatial) and GSA (group)—with dual-path residual architecture.

- **Spatial Self-Attention (SSA):** Tokens are partitioned into non-overlapping spatial windows of size \( S \). Within each window \( X \in \mathbb{R}^{S \times (G C)} \),
  \[
  Q = XW^Q,\, K = XW^K,\, V = XW^V,
  \]
  and SSA operates as
  \[
  \mathrm{SSA}(X) = \mathrm{Softmax}\left( \tfrac{QK^\top}{\sqrt{d}} + B_p \right) V,
  \]
  where \( B_p \) is a learned relative position bias.
- **Group Self-Attention (GSA):** Follows SSA, with transposed input, learning self-attention in the group (temporal–polarity) dimension using learned group biases \( B_g \):
  \[
  \mathrm{GSA}(X) = \mathrm{Softmax}\left( \tfrac{Q_g K_g^\top}{\sqrt{d_g}} + B_g \right) V_g.
  \]
- **Dual Residuals and Fusion:** The inputs and outputs of SSA and GSA form two residual streams:
  \[
  Z_s = X + Y_s, \quad Z_{sp} = Y_s + Y_g,
  \]
  which are concatenated and passed through a point-wise MLP and LayerNorm.
- **Block Design:** EDSA blocks are implemented with multiple parallel heads; each residual is followed by LayerNorm and a two-layer MLP (with GELU nonlinearity) and additional residual links.

## 3. Group Token Aggregation Module

GTA introduces hierarchical aggregation, reducing both spatial and group dimensions between stages while preserving computational efficiency by doubling the per-group channel dimension.

- **Overlapping Group Convolution:** The input \( N \times (G C) \) (interpreted as \( N \) “pixels” with \( G \) group vectors) undergoes group convolution. Each of the \( \lfloor G/2 \rfloor \) new groups is constructed by sliding a kernel across adjacent groups, with overlapping determined by kernel and stride parameters.
- **Spatial Downsampling:** After group convolution, a \(3 \times 3\) max pooling reduces the spatial grid size by half. The resulting output is \( (N/2) \times (\lfloor G/2 \rfloor \cdot 2C) \), followed by LayerNorm.

## 4. Backbone Architecture and Parameterization

The GET backbone is modular, structured into three (classification) or four (detection) stages:

1. **Group Token Embedding (GTE):** Converts raw event stream to tokens (\( \tfrac{H}{P} \times \tfrac{W}{P} \), each of size \( G \cdot 48 \)).
2. **Stage 1:** Two EDSA–MLP–Norm blocks; token resolution unchanged.
3. **GTA:** Halves token and group resolution; doubles per-group channel width.
4. **Stage 2:** Two EDSA blocks.
5. **GTA:** Further halving and channel doubling.
6. **Stage 3:** Eight EDSA blocks; final token processing.
7. **Classification Head:** Global average pooling, followed by a linear classifier.

Typical parameterization with \( H\!=\!128, W\!=\!128, P\!=\!4, K\!=\!6 \) yields \( G\!=\!12 \). The model contains approximately 4.5M parameters. Computational cost is dominated by local SSA (\( S=7^2 \)) and GSA (\( G \times S \)) attention operations.

## 5. Downstream Integration and Optimization

**Classification:** A single linear head processes globally pooled features, trained with standard cross-entropy loss.

**Detection:** GET is employed as backbone within YOLOX, optionally augmented with ConvLSTM layers for temporal memory. The detection objective is the YOLOX compound loss: objectness, classification, and bounding-box regression.

**Data Augmentation:** Procedures include random horizontal flipping (with polarity swap for gesture tasks), spatio-temporal cropping, and event-based MixUp and CutMix.

**Training:** The model is trained from scratch for 1,000 epochs (classification) or 400,000 steps (detection) using AdamW, cosine learning-rate scheduling, and weight decay of 0.05.

## 6. Performance and Empirical Evaluation

GET achieves state-of-the-art results on four event-based classification and two detection datasets. Summary statistics are provided below:

| Dataset            | GET Top-1 Accuracy / mAP | Previous SOTA          | Metric            |
|--------------------|:-----------------------:|------------------------|-------------------|
| CIFAR10-DVS        |         84.8%           | 78.1% (Nested-T)       | Top-1 accuracy    |
| N-MNIST            |         99.7%           | 99.3%                  | Top-1 accuracy    |
| N-CARS             |         96.7%           | 95.3%                  | Top-1 accuracy    |
| DVS128Gesture      |         97.9%           | 96.2% (EvT)            | Top-1 accuracy    |
| Gen1 (detection)   |    38.7% / 47.9% (*)    | 38.2% / 47.2% (RVT-B)  | COCO mAP@50–95    |
| 1Mpx (detection)   |    40.6% / 48.4% (*)    | 40.3% / 47.4%          | COCO mAP@50–95    |

(*) With/without ConvLSTM memory.

Ablation studies indicate the Group Token Embedding provides a 2.7–3.1% performance gain; replacing EDSA with SSA reduces accuracy by 3.3%, and omitting GTA reduces detection mAP by 0.9% but increases runtime speed. The GET pipeline, including event conversion and inference, achieves ≈16 ms runtime for 50 ms of events on a 1080Ti GPU, outperforming all prior end-to-end approaches.

## 7. Innovations and Impact

GET introduces three key contributions: (1) Group Token representation for separating time/polarity from spatial domains, (2) Event Dual Self-Attention blocks enabling nuanced, hierarchical context modeling, and (3) Group Token Aggregation to maintain computational tractability while preserving representational power. Its architecture consistently outperforms other event-based backbones in both accuracy and runtime efficiency, establishing a new standard for event-driven vision models [2310.02642].

Source: https://www.emergentmind.com/topics/group-event-transformer-get