Event-Guided Multimodal Sparsification (EGMS)

• EGMS is a family of event-guided token reduction strategies that exploits the spatiotemporal sparsity of event streams to remove redundant tokens before multimodal fusion.
• Implementations like EventFlash, FocusMamba, and EventSTU use adaptive temporal and spatial pruning to balance efficiency with semantic accuracy in various tasks.
• EGMS improves throughput and reduces computational cost significantly while addressing challenges in low-density or noisy event conditions.

Searching arXiv for the specified papers and related EGMS work to ground the article. Event-Guided Multimodal Sparsification (EGMS) denotes a family of event-guided token reduction strategies that exploit the native statistics of event streams—specifically their spatial sparsity and change-triggered temporal structure—to remove redundant visual representations before or during multimodal processing. In the event-based MLLM setting, EGMS is defined as using “the native statistics of event streams—how events are distributed in time and space—to reduce token redundancy before multimodal fusion in an MLLM,” with EventFlash presented as a concrete realization of that principle (Liu et al., 3 Feb 2026). Closely related uses of the same term appear in RGB-event detection and video LLM inference, where event-derived motion or saliency signals regulate token retention, masking, and cross-modal fusion in a sample-adaptive manner (Yang et al., 4 Sep 2025, Xu et al., 24 Nov 2025).

1. Event structure, redundancy, and the rationale for sparsification

An event camera outputs asynchronous events of the form $(x, y, t, p)$, where $(x, y)$ is the pixel location, $t$ is the timestamp, and $p$ is the polarity of the log-intensity change. In the EventFlash formulation, events are described as sparse in both space and time because they occur only at pixels where brightness changes exceed a threshold, leaving large portions of the space-time volume empty (Liu et al., 3 Feb 2026). This property makes dense image-like processing inefficient: voxelization into dense frames, uniform tokenization of fixed bins, or indiscriminate patch extraction generates many empty or low-information tokens.

Across the EGMS literature, the underlying computational argument is consistent. Attention cost grows with token count; in transformer-style modules it is frequently the dominant bottleneck. EGMS therefore treats event activity as a proxy for informativeness and uses it to modulate token budgets. The specific implementation differs by task. EventFlash performs temporal and spatial sparsification before fusion with a language backbone. FocusMamba uses event-derived activity to adaptively discard low-information regions in both RGB and event streams before cross-modality fusion. EventSTU applies event guidance in a training-free manner to keyframe sampling and patch pruning for long-video question answering (Liu et al., 3 Feb 2026, Yang et al., 4 Sep 2025, Xu et al., 24 Nov 2025).

A recurrent misconception is that EGMS is simply fixed-rate token pruning. The detection formulation in FocusMamba explicitly rejects fixed top-$k$ or fixed-threshold selection and instead derives a per-sample threshold from an event-guided scene activity proxy $r$ (Yang et al., 4 Sep 2025). Similarly, EventSTU allocates pruning budgets per frame according to normalized question relevance rather than a uniform retention rule (Xu et al., 24 Nov 2025).

2. EventFlash: EGMS for event-based multimodal LLMs

EventFlash instantiates EGMS as a two-part sparsification pipeline—Adaptive Temporal Window Aggregation (ATWA) and Sparse Density-Guided Attention (SDGA)—followed by event-language projection and decoding with Qwen2.5 (Liu et al., 3 Feb 2026). The pipeline begins by segmenting the event stream into fine-grained bins with default $10$ ms duration, constructing polarity-aware spatiotemporal intensity maps, encoding bins or merged windows with CLIP ViT-L/14, aligning compact event tokens with a two-layer MLP projector, and concatenating them with instruction tokens for the LLM.

Temporal sparsification is performed by ATWA in two stages. The first stage merges adjacent bins according to a spatiotemporal spike similarity computed from a polarity-aware Gaussian intensity function:

$$\mathcal{A}_{B}(x, y, t, p) = \sum_{n \in B} f(p_n)\,\exp\!\left(-\frac{(x-x_n)^2}{\sigma_x^2} - \frac{(y-y_n)^2}{\sigma_y^2} - \frac{(t-t_n)^2}{\sigma_t^2}\right).$$

Adjacent bins are compared through the distance $D(B_i, B_{i+1})$, and bins with $D < \tau_1$ are iteratively merged into meta event windows. The second stage encodes each window to obtain a semantic representation $(x, y)$0, computes cosine similarity between adjacent windows, and forms an adaptive merging score

$(x, y)$1

where $(x, y)$2 controls decay sensitivity and $(x, y)$3 is a normalized density factor (Liu et al., 3 Feb 2026). The paper synthesis notes that the exact textual definition of $(x, y)$4 is not explicit in the paper and then provides proposed consistent formulations; these proposed derivations should be distinguished from the explicit formulas attributed to the method.

Spatial sparsification is performed by SDGA. Given patch-level features $(x, y)$5, EventFlash first computes standard multi-head attention and then augments the attention logits with a density-derived term obtained from patch-level event density:

$(x, y)$6

The resulting density-guided attention biases computation toward dense, informative regions and suppresses empty or sparse areas. A TokenSelector then ranks aggregated responses and prunes low-importance spatial tokens (Liu et al., 3 Feb 2026). The EventFlash synthesis further provides proposed, clearly marked interpretations of hard selection, differentiable soft gating, and expected token reduction, but these are presented as consistent derivations rather than paper-defined operators.

The training substrate for this EGMS realization is EventMind, a dataset with 500k instruction samples spanning seven tasks: 200k simple captioning, 90k scene captioning, 30k motion captioning, 90k EventQA, 60k FGQA, 10k MCQA, and 20k human action QA. EventMind combines real event streams from DSEC, N-ImageNet, HARDVS, and E2VID with simulated events from Kinetics-700, UCF-101, Wevid-10M, PLM-Data, and MotionBench via V2E; GPT-4o filters videos for simulation quality, Qwen-VL-Max generates annotations where needed, and human curation is used for quality control. EventFlash is trained with a short-to-long curriculum: Stage 1 uses 200k short sequences and trains only the event-language projector at learning rate $(x, y)$7 and batch size $(x, y)$8; Stage 2 uses 110k medium sequences with all parameters unfrozen at learning rate $(x, y)$9, batch size $t$0, and gradient accumulation $t$1; Stage 3 uses 190k long sequences with the same hyperparameters and a cosine learning rate schedule (Liu et al., 3 Feb 2026).

3. FocusMamba: adaptive EGMS in RGB-event object detection

In FocusMamba, EGMS is formulated as an adaptive collaborative sparsification strategy for RGB-event object detection, where event-derived scene content changes regulate token selection in both modalities before fusion (Yang et al., 4 Sep 2025). The event stream is represented as asynchronous events $t$2 sampled within RGB inter-frame intervals—$t$3 ms for DSEC-Det and $t$4 ms for PKU-DAVIS-SOD—and converted to an event voxel tensor for backbone processing, while raw events are used directly to derive sparsification signals.

The central control variable is the event spatial ratio

$t$5

where $t$6 if at least one event occurs at pixel $t$7 during the current RGB frame interval and $t$8 otherwise. This scalar acts as a scene activity proxy and controls both score contrast and thresholding. RGB tokens $t$9 and event tokens $p$0 are scored separately. For RGB, token importance is the channelwise $p$1 norm,

$p$2

For events, scoring begins with timestamp accumulation within each patch footprint, followed by max pooling at the current stage resolution and Gaussian neighborhood aggregation to favor spatiotemporally continuous activity over isolated noise (Yang et al., 4 Sep 2025).

Event-Guided Control Mechanism (EGCM) then transforms these modality-specific scores using two event-dependent quantities:

$p$3

The scores are normalized as $p$4, and binary masks are defined by thresholding at $p$5. For sparse scenes, small $p$6 reduces $p$7 and sharpens the softmax, while increasing $p$8 through $p$9; for dense scenes, the mechanism relaxes pruning to avoid discarding informative tokens. At stage 1 only, the image mask is replaced by the event mask, $k$0, because early RGB activations are described as edge-dominant and less reliable (Yang et al., 4 Sep 2025).

These masks feed directly into Cross-Modality Focus Fusion (CMFF), which has two components. Complementarity-Aware Enhancement (CAE) identifies regions retained in one modality but not the other and boosts features in the degraded modality using a coefficient $k$1. Focused Interlaced Mamba (FI-Mamba) then gathers tokens from the union mask $k$2, interleaves RGB and event token sequences, processes them with VMamba Bidi-Scan, and scatters the enhanced representations back before a Sparse MLP refinement (Yang et al., 4 Sep 2025). The notable architectural point is that EGMS in this setting is not merely a pruning front-end; it structurally conditions the subsequent fusion operator.

The paper also specifies that EGMS introduces no Gumbel-Softmax, straight-through estimator, or sparsity regularizer. The masks are deterministic and parameter-free, gradients flow only through retained tokens, and no additional sparsity-balancing loss is introduced (Yang et al., 4 Sep 2025). This differentiates the method from learned importance prediction schemes and reflects a deliberate preference for low-cost event-driven control.

4. EventSTU: training-free EGMS for long-video LLMs

EventSTU extends EGMS to video LLMs through a training-free pipeline that sparsifies both keyframes and within-frame tokens, then allocates budgets according to question relevance (Xu et al., 24 Nov 2025). The method supports real event cameras as well as simulated events computed from consecutive RGB frames. Given frames $k$3 and $k$4, EventSTU applies reverse gamma correction and computes per-pixel log-intensity change,

$k$5

then derives positive and negative event counts using thresholds $k$6. The event frame $k$7 stores the total count $k$8 per pixel (Xu et al., 24 Nov 2025).

Temporal sparsification is organized as coarse-to-fine keyframe sampling. The coarse stage uses per-frame event density

$k$9

and cumulative sampling with threshold

$r$0

to select a candidate set of frames. This stage is designed to balance sampling across high- and low-density segments rather than over-selecting only the most active intervals. The fine stage computes text-image similarities between the query embedding and candidate frame embeddings, normalizes them to relevance scores $r$1, divides candidates into temporal bins, and retains the highest-similarity frame per bin to preserve both temporal diversity and semantic alignment (Xu et al., 24 Nov 2025).

Spatial sparsification then proceeds in two passes. Physics-aware pruning defines a per-patch event saliency score

$r$2

and removes the lowest $r$3 of tokens in frame $r$4. Semantic-aware pruning reuses the last-layer attention matrix of the vision encoder to compute token importance

$r$5

then prunes the lowest $r$6 among the surviving tokens. The paper refers to this token-pruning mechanism as ZAP and emphasizes that the physics-aware component is effectively zero-cost because it exploits already available event information (Xu et al., 24 Nov 2025).

The distinctive feature of EventSTU is holistic budget allocation. Let $r$7 be the number of fine-sampled frames, $r$8 the number of tokens per frame, $r$9 the overall target pruning ratio, and $10$0 a per-frame base retention ratio. The method defines

$10$1

allocates retained tokens to frame $10$2 as

$10$3

and converts this to a frame-specific pruning ratio

$10$4

This ratio is then split into physics-aware and semantic-aware components subject to caps (Xu et al., 24 Nov 2025). A plausible implication is that EventSTU treats query relevance not as an auxiliary cue but as the global controller of where sparsification should be relaxed.

5. Reported performance, scaling behavior, and ablation evidence

The three EGMS realizations report improvements in efficiency while preserving or improving task metrics, but the quantitative form of those gains depends strongly on task and architecture.

System	Setting	Reported outcomes
EventFlash	Event-based MLLM	$10$5 tokens/sec, $10$6 over EventFlash-Zero; up to $10$7 bins vs EventGPT’s $10$8
FocusMamba	RGB-event detection	EGMS alone: $10$9 FLOPs and $$\mathcal{A}_{B}(x, y, t, p) = \sum_{n \in B} f(p_n)\,\exp\!\left(-\frac{(x-x_n)^2}{\sigma_x^2} - \frac{(y-y_n)^2}{\sigma_y^2} - \frac{(t-t_n)^2}{\sigma_t^2}\right).$$0 mAP in ablation
EventSTU	Video LLM	Up to $$\mathcal{A}_{B}(x, y, t, p) = \sum_{n \in B} f(p_n)\,\exp\!\left(-\frac{(x-x_n)^2}{\sigma_x^2} - \frac{(y-y_n)^2}{\sigma_y^2} - \frac{(t-t_n)^2}{\sigma_t^2}\right).$$1 FLOPs reduction and $$\mathcal{A}_{B}(x, y, t, p) = \sum_{n \in B} f(p_n)\,\exp\!\left(-\frac{(x-x_n)^2}{\sigma_x^2} - \frac{(y-y_n)^2}{\sigma_y^2} - \frac{(t-t_n)^2}{\sigma_t^2}\right).$$2 prefilling speedup

For EventFlash, the main efficiency claim is a throughput increase from $$\mathcal{A}_{B}(x, y, t, p) = \sum_{n \in B} f(p_n)\,\exp\!\left(-\frac{(x-x_n)^2}{\sigma_x^2} - \frac{(y-y_n)^2}{\sigma_y^2} - \frac{(t-t_n)^2}{\sigma_t^2}\right).$$3 tokens/sec for EventFlash-Zero to $$\mathcal{A}_{B}(x, y, t, p) = \sum_{n \in B} f(p_n)\,\exp\!\left(-\frac{(x-x_n)^2}{\sigma_x^2} - \frac{(y-y_n)^2}{\sigma_y^2} - \frac{(t-t_n)^2}{\sigma_t^2}\right).$$4 tokens/sec for EventFlash-3B, while maintaining comparable performance across tasks (Liu et al., 3 Feb 2026). The same paper reports support for up to $$\mathcal{A}_{B}(x, y, t, p) = \sum_{n \in B} f(p_n)\,\exp\!\left(-\frac{(x-x_n)^2}{\sigma_x^2} - \frac{(y-y_n)^2}{\sigma_y^2} - \frac{(t-t_n)^2}{\sigma_t^2}\right).$$5 bins per input, contrasted with EventGPT’s $$\mathcal{A}_{B}(x, y, t, p) = \sum_{n \in B} f(p_n)\,\exp\!\left(-\frac{(x-x_n)^2}{\sigma_x^2} - \frac{(y-y_n)^2}{\sigma_y^2} - \frac{(t-t_n)^2}{\sigma_t^2}\right).$$6-bin limit, and gives representative task results for EventFlash-3B—GDC $$\mathcal{A}_{B}(x, y, t, p) = \sum_{n \in B} f(p_n)\,\exp\!\left(-\frac{(x-x_n)^2}{\sigma_x^2} - \frac{(y-y_n)^2}{\sigma_y^2} - \frac{(t-t_n)^2}{\sigma_t^2}\right).$$7, FGQA $$\mathcal{A}_{B}(x, y, t, p) = \sum_{n \in B} f(p_n)\,\exp\!\left(-\frac{(x-x_n)^2}{\sigma_x^2} - \frac{(y-y_n)^2}{\sigma_y^2} - \frac{(t-t_n)^2}{\sigma_t^2}\right).$$8, HAQA $$\mathcal{A}_{B}(x, y, t, p) = \sum_{n \in B} f(p_n)\,\exp\!\left(-\frac{(x-x_n)^2}{\sigma_x^2} - \frac{(y-y_n)^2}{\sigma_y^2} - \frac{(t-t_n)^2}{\sigma_t^2}\right).$$9, MCQA $D(B_i, B_{i+1})$0—with the 7B variant improving these to GDC $D(B_i, B_{i+1})$1, FGQA $D(B_i, B_{i+1})$2, HAQA $D(B_i, B_{i+1})$3, and MCQA $D(B_i, B_{i+1})$4 (Liu et al., 3 Feb 2026). The ablations isolate temporal sparsification alone at $D(B_i, B_{i+1})$5 tokens/sec, spatial sparsification alone at $D(B_i, B_{i+1})$6 tokens/sec, and full EGMS at $D(B_i, B_{i+1})$7 tokens/sec, with $D(B_i, B_{i+1})$8 ms identified as the default aggregation interval balancing efficiency and accuracy.

For FocusMamba, the ablation on DSEC-Det in the FocusMamba-B setting gives a compact view of EGMS’s standalone effect. Base VSS reports $D(B_i, B_{i+1})$9 mAP50/mAP at $D < \tau_1$0G FLOPs, while adding EGMS yields $D < \tau_1$1 at $D < \tau_1$2G FLOPs (Yang et al., 4 Sep 2025). The same section reports that the full method reduces FLOPs by $D < \tau_1$3 on DSEC-Det and $D < \tau_1$4 on PKU-DAVIS-SOD, and runtime by $D < \tau_1$5 and $D < \tau_1$6, respectively, relative to a baseline without EGMS. The EGCM ablation further shows that using both scale and control factors outperforms fixed kept rate, scale-only, or control-only variants.

For EventSTU, the reported gains concern long-video prefilling and FLOPs. The framework achieves up to $D < \tau_1$7 FLOPs reduction and $D < \tau_1$8 prefilling speedup over strong keyframe-sampling-plus-pruning baselines (Xu et al., 24 Nov 2025). At a $D < \tau_1$9 token budget, EventSTU reaches $(x, y)$00 average on EventBench versus $(x, y)$01 for AKS + DyCoKe while using only $(x, y)$02 of its FLOPs. Physics-only pruning, denoted ZAP_phy, already improves average performance from $(x, y)$03 to $(x, y)$04 at $(x, y)$05 of the original model’s FLOPs, and full ZAP reaches $(x, y)$06 with the lowest FLOPs among pruning baselines. EventSTU also reports a TTFT reduction of $(x, y)$07 relative to the original model.

A shared pattern emerges from these results. Event guidance is used either as a direct importance prior or as a global budget controller, and the measured gains are strongest when temporal and spatial sparsification are combined rather than applied in isolation. This suggests that EGMS is best understood as a coordinated reduction policy over the full spatiotemporal token graph rather than a single pruning heuristic.

6. Limitations, interpretive boundaries, and proposed extensions

The literature records several limitations that recur across implementations. EventFlash notes failure modes in extremely low-density scenes, such as near-static or very dark settings, where over-aggregation in time and over-pruning in space may discard context; it also notes sensitivity to thresholds $(x, y)$08 and the risk of merging across semantic transitions (Liu et al., 3 Feb 2026). FocusMamba similarly acknowledges that when both scene and objects are static, the event spatial ratio $(x, y)$09 may not accurately reflect object information, reducing the effectiveness of the control mechanism (Yang et al., 4 Sep 2025). EventSTU reports related issues: event noise or sensor artifacts can distort saliency, static-object queries are not naturally aligned with event cues, and highly uniform background changes such as exposure flicker may produce high event density without corresponding semantic importance (Xu et al., 24 Nov 2025).

These limitations correct another common misunderstanding: EGMS does not imply that events are always sufficient statistics for visual importance. In all three systems, event cues are either complemented by semantics or explicitly modulated to avoid over-commitment. EventFlash uses semantic similarity in the second ATWA stage; FocusMamba preserves modality-specific scoring rather than collapsing RGB and event importance into a single signal; EventSTU augments event saliency with question relevance and encoder attention (Liu et al., 3 Feb 2026, Yang et al., 4 Sep 2025, Xu et al., 24 Nov 2025).

The EventFlash synthesis also lists several future directions, but it explicitly labels some of them as proposed rather than established components of the method. These include learned sparsity controllers that replace hand-tuned thresholds, reinforcement learning for window selection, differentiable top-$(x, y)$10 relaxations such as Gumbel-softmax or sparsemax, global-local hybrid attention, and hardware-aware sparse implementations (Liu et al., 3 Feb 2026). Because these items are presented as improvements rather than reported components, they should be treated as design proposals.

Taken together, the current EGMS literature defines a technical pattern rather than a single architecture: extract event-derived measures of change, density, or saliency; convert them into adaptive temporal or spatial retention policies; preserve complementary semantic cues where event guidance is insufficient; and reduce multimodal token load before the dominant sequence-processing stage. In EventFlash this pattern is realized through ATWA and SDGA for event-based MLLMs, in FocusMamba through EGCM-driven masking and CMFF for RGB-event detection, and in EventSTU through training-free coarse-to-fine sampling, ZAP pruning, and question-aware budget allocation for long-video LLMs (Liu et al., 3 Feb 2026, Yang et al., 4 Sep 2025, Xu et al., 24 Nov 2025).