Group Event Transformer (GET)

• The paper introduces GET as an event-native transformer that decouples temporal-polarity and spatial information for improved feature extraction in event-based vision.
• GET employs a novel grouping token representation and dual self-attention to separately process temporal-polarity and spatial cues, optimizing downstream tasks.
• Empirical results demonstrate GET's superior performance and efficiency over state-of-the-art methods on multiple event-based classification and detection benchmarks.

Searching arXiv for the exact paper and closely related event-based transformer work to ground the article. Group Event Transformer (GET) is a group-based vision Transformer backbone for event-based vision that decouples temporal-polarity information from spatial information throughout the feature extraction process. It was introduced to address a recurrent limitation in event-based backbones: many methods transform asynchronous events into images and then apply image-based designs, thereby coupling time, polarity, and space in a way that overlooks event-native structure. GET instead organizes asynchronous events into “Group Token” representations, processes them with an Event Dual Self-Attention block and a Group Token Aggregation module, and connects the resulting backbone to downstream heads for classification and object detection. In the reported evaluations, it outperforms other state-of-the-art methods on four event-based classification datasets and two event-based object detection datasets (Peng et al., 2023).

1. Problem formulation and design rationale

Event cameras output asynchronous event streams in which each event encodes spatial location $(x, y)$, timestamp, and polarity. Relative to RGB images, event streams retain rich temporal information, reported as greater than 10K fps, together with polarity information, which is especially relevant in high-speed or low-power settings. The central claim motivating GET is that existing event-based backbones mostly transform event data into images, such as voxel grids or histograms, and then apply image-based transformer or CNN designs. This couples temporal, spatial, and polarity information and leads to suboptimal feature extraction. The paper also notes that some SNNs or GNNs can use temporal and polarity data, but are hardware-intensive or underperform (Peng et al., 2023).

GET’s response is to explicitly decouple spatial information from temporal-polarity information throughout the network. This design choice is not a minor implementation detail but the organizing principle of the backbone. The representation stage groups events by timestamps and polarities, while later attention and aggregation stages continue to preserve and communicate information separately in the spatial domain and the temporal-polarity domain. This suggests that GET should be understood less as a conventional transformer adapted to event inputs and more as an event-native backbone in which the data decomposition determines the attention structure.

The model is described as hierarchical and modular. It can be integrated with different downstream tasks by connecting it with various heads. For classification, the hierarchy is three-stage; for detection, it is four-stage. The code is reported as available at the project repository linked in the paper (Peng et al., 2023).

2. Group Token representation

The input event is defined as $(\vec{t}, \vec{p}, \vec{x}, \vec{y})$, corresponding to time, polarity, and spatial coordinates. GET begins by discretizing time into $K$ intervals,

$$\vec{d_t} = \left\lfloor K \cdot \frac{\vec{t}-t_0}{t_{end}-t_0+1} \right\rfloor .$$

The spatial domain of size $H \times W$ is divided into $P \times P$ patches. Two indices are then defined: a patch rank,

$$\vec{pr} = \left\lfloor \vec{x} \mod P \right\rfloor + \left\lfloor \vec{y} \mod P \right\rfloor \times P ,$$

and a patch position,

$$\vec{pos} = \left\lfloor \vec{x} / P \right\rfloor + \left\lfloor \vec{y} / P \right\rfloor \times (W/P) .$$

These are mapped to a one-dimensional group index,

$$\vec{l} = (KHW)\cdot \vec{p} + HW \cdot \vec{d_t} + \frac{HW}{P^2} \cdot \vec{pr} + \vec{pos}.$$

This index corresponds to a distinct combination of time interval, polarity, patch rank, and patch position. Counts and relative times are accumulated for each group, producing two one-dimensional arrays of length $HW*2K$ for both polarities and all time intervals. After concatenation and reshaping, the representation has shape

$(\vec{t}, \vec{p}, \vec{x}, \vec{y})$0

A $(\vec{t}, \vec{p}, \vec{x}, \vec{y})$1 group convolution and an MLP are then used to embed each group’s features, yielding a final Group Token shape

$(\vec{t}, \vec{p}, \vec{x}, \vec{y})$2

The significance of this representation is twofold. First, events are grouped by timestamp intervals and polarity, so temporal-polarity structure is preserved independently from spatial structure. Second, the grouped tokens, rather than images, become the transformer's basic input units. The paper’s Figure 1 is described as illustrating this transformation from asynchronous events to grouped tokens (Peng et al., 2023).

3. Event Dual Self-Attention

The Event Dual Self-Attention (EDSA) block is the core feature interaction mechanism in GET. Its purpose is to extract correlations efficiently in both spatial and temporal-polarity domains while preserving the separation initiated by the Group Token representation. Each block contains an EDSA layer, an MLP, and normalization layers, and uses dual residual connections to maintain two parallel feature streams, one spatial and one temporal-polarity (Peng et al., 2023).

The attention computation proceeds in two axes. After partitioning feature maps into $(\vec{t}, \vec{p}, \vec{x}, \vec{y})$3 windows with $(\vec{t}, \vec{p}, \vec{x}, \vec{y})$4 tokens per window, GET applies Spatial Self-Attention (SSA) to each window. For input $(\vec{t}, \vec{p}, \vec{x}, \vec{y})$5,

$(\vec{t}, \vec{p}, \vec{x}, \vec{y})$6

and the spatial attention is

$(\vec{t}, \vec{p}, \vec{x}, \vec{y})$7

Here $(\vec{t}, \vec{p}, \vec{x}, \vec{y})$8 is a spatial relative position bias. The output is then transposed to shape $(\vec{t}, \vec{p}, \vec{x}, \vec{y})$9, and Group Self-Attention (GSA) is computed over the group axis:

$K$0

$K$1

In this expression, $K$2 is a relative group bias that guides attention along the time-plus-polarity axis.

The distinction from image-based self-attention is explicit. Standard image-based attention emphasizes patch-to-patch spatial relations. EDSA splits attention into a spatial axis and a group axis, uses different biases for each, and retains both via dual residuals. The paper describes one residual path as preserving the SSA output only, while the other applies additional GSA for group-context information. This dual mechanism is intended to prevent cross-domain interference while enabling dynamic fusion of spatial and temporal-polarity features. A common simplification is therefore misleading: GET is not merely a Swin-like local transformer applied to an event image, because its attention structure is defined over the event grouping itself rather than only over spatial windows.

4. Group Token Aggregation and hierarchical fusion

Between stages, GET uses the Group Token Aggregation (GTA) module to hierarchically integrate information in both spatial and temporal-polarity domains. Its input has shape

$K$3

and it operates by dividing feature maps into overlapping groups of neighboring group indices, followed by group-wise convolution. The overlap is designed to aggregate across adjacent temporal-polarity groups as well as spatial patches (Peng et al., 2023).

Two hyperparameters govern the module: the group-wise kernel $K$4 and the group-wise stride $K$5. The paper specifies

$K$6

After aggregation, the group number halves, the channel count doubles, and the feature field expands in both domains; max pooling downsamples the spatial dimension. The paper interprets this as a receptive-field growth mechanism analogous to deeper CNNs, but extended jointly over spatial and event domains.

Taken together, Group Token, EDSA, and GTA define a hierarchical backbone. The summary in the paper characterizes the architecture as scalable and modular, with three stages for classification and four stages for detection. This staged design is important because the temporal-polarity grouping is not only an input encoding; it is preserved and progressively aggregated throughout the hierarchy.

5. Empirical results and ablation evidence

GET is evaluated on four event-based classification datasets—Cifar10-DVS, N-MNIST, N-CARS, and DVS128Gesture—and two object detection datasets, 1Mpx and Gen1. On classification, the reported results are 84.8 on Cifar10-DVS, 99.7 on N-MNIST, 96.7 on N-CARS, and 97.9 on DVS128Gesture, with 4.5M parameters. On detection, GET reports 38.7 mAP without memory and 47.9 mAP with memory on Gen1, and 40.6 mAP without memory and 48.4 mAP with memory on 1Mpx, with runtime 15.9/16.8 ms. The paper states that GET achieves the best mAP in all cases and is the fastest in runtime, and that its parameter count for detection, 18–22M, is lower than other top performers (Peng et al., 2023).

Classification benchmarks

Dataset	GET	Comparison noted in the paper
Cifar10-DVS	84.8	best listed competitor 78.1 (Nested-T)
N-MNIST	99.7	best listed competitor 99.6 (PLIF)
N-CARS	96.7	best listed competitor 93.3 (Nested-T)
DVS128Gesture	97.9	best listed competitor 97.6 (PLIF)

Detection benchmarks

Dataset	GET	Comparison noted in the paper
Gen1 mAP w/o memory	38.7	38.2 (ASTMNet)
Gen1 mAP w/ memory	47.9	47.2 (RVT-B)
1Mpx mAP w/o memory	40.6	40.3 (ASTMNet)
1Mpx mAP w/ memory	48.4	48.3 (ASTMNet)

The reported ablations clarify which components are responsible for the gains. Replacing a Nested-T backbone progressively with GET modules—GTE, EDSA, and GTA—improves performance on CIFAR10-DVS and 1Mpx, and combining all GET modules yields +7.6% on Cifar10-DVS and +3.1 mAP on 1Mpx over baseline. In the representation ablation, GTE reaches 84.8% versus 80.4% for the best listed competitor among event histogram, voxel grid, time surface, or TORE, and has the shortest preprocessing time, 0.052s per $K$7 events, reported as 6–300x faster. In the EDSA ablation, the full block with dual residuals reaches Top-1 84.8%, whereas alternative SSA-only or mixed designs range from 75% to 81.9%. The group-number study reports that $K$8 is optimal on the short-range Cifar10-DVS dataset, while $K$9 is best on the long-range DVS128Gesture dataset; too low a group number merges diverse information, whereas too high a group number causes sparsity. The figures described in the paper further indicate that GET captures event motion directions and object locations more accurately and robustly than SSA-only transformers.

These findings support a specific reading of the architecture. The performance gains are not attributed to a single generic transformer scaling effect; they are tied to the event-specific representation, the dual-axis attention, and the hierarchical group aggregation.

6. Position within event-based transformer research

GET belongs to a broader line of event-based transformer research, but its design emphasis is distinctive. Event Transformer+ processes event camera data with a refined patch-based event representation, a robust and modular backbone, and distinct output heads for event-stream classification and dense per-pixel prediction. It retains only activated patches, so computation scales with token count rather than frame size, and it can accommodate different data modalities such as event streams and grayscale images (Sabater et al., 2022). Event Voxel Set Transformer (EVSTr), by contrast, converts event streams into voxel sets and uses the Multi-Scale Neighbor Embedding Layer for local aggregation, the Voxel Self-Attention Layer for global interaction, and a segment modeling strategy for long-range temporal structure (Xie et al., 2023).

Against this background, GET is characterized by explicit decoupling of temporal-polarity information from spatial information across the full backbone. Event Transformer+ remains patch-based and sparsity-aware; EVSTr remains voxel-set-based and attention-aware; GET organizes the backbone around grouped event tokens and dual-axis attention. This suggests that the main conceptual contribution of GET is not merely efficiency or sparsity exploitation, both of which also appear elsewhere, but the claim that event-specific structure should remain factorized during representation learning and hierarchical fusion.

A common misconception is that event-based transformers differ only in how they serialize sparse events before applying standard attention. The comparison above indicates a more consequential divergence. In GET, token formation, attention factorization, and aggregation hierarchy are all aligned with the temporal-polarity/spatial decomposition. That design choice explains why the paper presents Group Token, EDSA, and GTA as a tightly coupled system rather than independent modules.