EvEnhancer: Event-Driven Video SR

• EvEnhancer is an event-driven continuous space-time video super-resolution method that reconstructs arbitrary high-resolution frames by combining low-res RGB images and aligned event streams.
• It utilizes an Event-Adapted Synthesis Module and a Local Implicit Video Transformer to fuse spatial and temporal cues through continuous querying and local cross-scale attention.
• The extension EvEnhancerPlus introduces a controllable switching mechanism and staged training strategy to boost efficiency and robustness, particularly under out-of-distribution conditions.

Searching arXiv for the named systems to ground the article in the cited papers. First, retrieving the core "EvEnhancer" continuous space-time video super-resolution paper. Then retrieving the follow-up/extended formulation with EvEnhancerPlus. Also checking the similarly named VEnhancer paper because the source material explicitly notes possible name confusion. EvEnhancer is an event-driven method for continuous space-time video super-resolution (C-STVSR) that combines low-resolution RGB frames with aligned event streams to reconstruct high-resolution, high-frame-rate video at arbitrary spatial and temporal scales (Wei et al., 7 May 2025). In its core formulation, the model addresses the joint problem of spatial upsampling and temporal interpolation through a unified continuous representation rather than separate fixed-scale modules. The method is built around two principal components: an Event-Adapted Synthesis Module (EASM), which uses event information to model long-term motion trajectories and synthesize informative spatiotemporal features, and a Local Implicit Video Transformer (LIVT), which learns a continuous video implicit neural representation (INR) with local cross-scale spatiotemporal attention (Wei et al., 7 May 2025). A later extension, EvEnhancerPlus, adds a controllable switching mechanism and a staged training strategy to improve efficiency while maintaining robustness at out-of-distribution (OOD) scales (Wei et al., 4 Oct 2025).

1. Definition and problem setting

EvEnhancer is defined in the literature as an approach for continuous space-time video super-resolution with events. The task is to reconstruct an arbitrary number of intermediate high-resolution frames at arbitrary timestamps and arbitrary spatial magnifications from a pair of low-resolution, low-frame-rate frames and an aligned event stream (Wei et al., 7 May 2025).

The core input notation consists of low-resolution frames $V_{LR} = \{I_t^{LR}\}$, typically focusing on endpoints $I_0^{LR}$ and $I_1^{LR}$, together with an event stream $E = \{(x_i, y_i, t_i, p_i)\}$ where $p_i \in \{+1,-1\}$ denotes polarity (Wei et al., 7 May 2025). The target is to predict high-resolution frames at arbitrary continuous spatiotemporal coordinates, expressed as $I^{HR}(x,y,\mathcal{T}; s)$ for any $\mathcal{T} \in [0,1]$ and real spatial scale $s \ge 1$ (Wei et al., 7 May 2025).

A central design choice is to learn a continuous mapping

$$f_\theta : (x, y, \mathcal{T}, s; V_{LR}, E) \rightarrow I(x,y,\mathcal{T}; s) \in \mathbb{R}^3,$$

with HR pixel coordinates normalized to LR coordinates as $(x_{lr}, y_{lr}) = (x_{hr}/s, y_{hr}/s)$ (Wei et al., 7 May 2025). The decoder uses LIIF-style cell decoding, with $I_0^{LR}$0, and reconstructs pixels through

$I_0^{LR}$1

This formulation is intended to support arbitrary spatial scales and arbitrary frame rates by repeated continuous querying (Wei et al., 7 May 2025).

The motivation is that prevailing C-STVSR methods often generalize poorly at OOD spatial and temporal scales, while event streams provide high temporal resolution and high dynamic range that are beneficial for motion modeling and robustness under challenging conditions (Wei et al., 7 May 2025). This suggests that EvEnhancer occupies a distinct position relative to purely frame-based C-STVSR models: it is designed not merely for interpolation or fixed-factor upsampling, but for continuous, event-guided reconstruction across space and time.

2. Event representation and continuous formulation

EvEnhancer relies on event cameras’ asynchronous brightness-change measurements. In the 2025 formulation, an event is represented as $I_0^{LR}$2, with polarity triggered when log-intensity changes exceed a threshold (Wei et al., 7 May 2025). The later EvEnhancerPlus paper restates the event formalism as $I_0^{LR}$3 and gives the firing rule

$I_0^{LR}$4

together with the accumulated log-intensity change

$I_0^{LR}$5

These expressions formalize the connection between event density and local brightness change (Wei et al., 4 Oct 2025).

For processing, EvEnhancer converts the raw event stream into a 3D spatiotemporal voxel grid $I_0^{LR}$6 by discretizing time into short segments between the endpoint frames (Wei et al., 7 May 2025). A typical aggregation is

$I_0^{LR}$7

with $I_0^{LR}$8 for positive and $I_0^{LR}$9 for negative events (Wei et al., 7 May 2025). The 2025 extension describes a linearly weighted voxelization,

$I_1^{LR}$0

followed by percentile clipping normalization,

$I_1^{LR}$1

with $I_1^{LR}$2 equal to the 98-th percentile among non-zero values (Wei et al., 4 Oct 2025).

The event voxels are aligned to the two LR frames and bicubic-downsampled to match LR resolution (Wei et al., 7 May 2025). Their function is to provide fine-grained temporal cues between sparse RGB observations. In the source papers, this is repeatedly linked to improved handling of rapid motion, high dynamic range variation, and OOD scales (Wei et al., 7 May 2025, Wei et al., 4 Oct 2025).

3. Event-Adapted Synthesis Module

The Event-Adapted Synthesis Module is the first major architectural component of EvEnhancer. It is designed to fuse frames and events to learn long-term motion trajectories and informative spatiotemporal features through two submodules: Event-Modulated Alignment (EMA) and Bidirectional Recurrent Compensation (BRC) (Wei et al., 7 May 2025).

Event-Modulated Alignment

EMA operates on a 3-level pyramid at scales $I_1^{LR}$3 (Wei et al., 7 May 2025). At each level $I_1^{LR}$4, frame features $I_1^{LR}$5 and $I_1^{LR}$6 and event features $I_1^{LR}$7 for segment $I_1^{LR}$8 are used to estimate an initial motion vector:

$I_1^{LR}$9

where $E = \{(x_i, y_i, t_i, p_i)\}$0 is a Conv+LReLU block and $E = \{(x_i, y_i, t_i, p_i)\}$1 is the coarse-to-fine offset propagated from the previous level (Wei et al., 7 May 2025). Events then modulate motion through an Event Modulation Block:

$E = \{(x_i, y_i, t_i, p_i)\}$2

and the final offsets are obtained as

$E = \{(x_i, y_i, t_i, p_i)\}$3

Deformable convolutions align features in both forward and backward directions:

$E = \{(x_i, y_i, t_i, p_i)\}$4

The DCN sampling rule is explicitly given as

$E = \{(x_i, y_i, t_i, p_i)\}$5

These aligned features are aggregated coarse-to-fine to produce a discrete aligned temporal feature sequence $E = \{(x_i, y_i, t_i, p_i)\}$6 across timestamps $E = \{(x_i, y_i, t_i, p_i)\}$7 (Wei et al., 7 May 2025).

Bidirectional Recurrent Compensation

Because each event segment corresponds to a narrow temporal window, BRC propagates information forward and backward to enlarge temporal coverage (Wei et al., 7 May 2025). At timestamp $E = \{(x_i, y_i, t_i, p_i)\}$8, it combines event feature $E = \{(x_i, y_i, t_i, p_i)\}$9, aligned frame feature $p_i \in \{+1,-1\}$0, and recurrent hidden states:

$p_i \in \{+1,-1\}$1

$p_i \in \{+1,-1\}$2

The final fusion is residual:

$p_i \in \{+1,-1\}$3

The output of EASM is thus a high-temporal-resolution feature sequence $p_i \in \{+1,-1\}$4 (Wei et al., 7 May 2025).

The stated architectural details are specific: RGB features are extracted by a $p_i \in \{+1,-1\}$5 convolution with LReLU followed by 5 residual blocks; event features use a $p_i \in \{+1,-1\}$6 convolution and 5 residual blocks; BRC uses a $p_i \in \{+1,-1\}$7 convolution for event extraction, $p_i \in \{+1,-1\}$8 convolutions for channel attention and FFN, and $p_i \in \{+1,-1\}$9 convolutions elsewhere (Wei et al., 7 May 2025).

A plausible implication is that EASM is the locus where event information most directly reshapes motion estimation, while subsequent modules primarily exploit the improved latent trajectory representation.

4. Local Implicit Video Transformer and continuous decoding

The Local Implicit Video Transformer is the second principal module. It constructs a unified continuous spatiotemporal INR by selecting local temporal neighborhoods and applying cross-scale spatiotemporal attention around each query coordinate (Wei et al., 7 May 2025).

For a query timestamp $I^{HR}(x,y,\mathcal{T}; s)$0, LIVT selects a local temporal neighborhood of size $I^{HR}(x,y,\mathcal{T}; s)$1 by

$I^{HR}(x,y,\mathcal{T}; s)$2

Spatially, it defines a local grid of size $I^{HR}(x,y,\mathcal{T}; s)$3 around $I^{HR}(x,y,\mathcal{T}; s)$4 (Wei et al., 7 May 2025).

From $I^{HR}(x,y,\mathcal{T}; s)$5, 3D convolutions generate embeddings $I^{HR}(x,y,\mathcal{T}; s)$6, $I^{HR}(x,y,\mathcal{T}; s)$7, and $I^{HR}(x,y,\mathcal{T}; s)$8 (Wei et al., 7 May 2025). The target query embedding is obtained by 3D trilinear sampling and upsampling:

$I^{HR}(x,y,\mathcal{T}; s)$9

For each selected $\mathcal{T} \in [0,1]$0, keys and values are sampled on the local LR grid:

$\mathcal{T} \in [0,1]$1

The relative spatiotemporal coordinates $\mathcal{T} \in [0,1]$2 are encoded by a cosine positional encoding

$\mathcal{T} \in [0,1]$3

with $\mathcal{T} \in [0,1]$4, followed by a linear projection $\mathcal{T} \in [0,1]$5 to produce positional bias $\mathcal{T} \in [0,1]$6 (Wei et al., 7 May 2025). Local cross-scale attention is then computed as

$\mathcal{T} \in [0,1]$7

The unified continuous video representation is formed by concatenating the attended features:

$\mathcal{T} \in [0,1]$8

Finally, a 5-layer MLP with widths $\mathcal{T} \in [0,1]$9 and GELU reconstructs RGB values via

$s \ge 1$0

These definitions are given explicitly in the original paper (Wei et al., 7 May 2025).

The architecture uses one $s \ge 1$1 convolution to encode $s \ge 1$2 and three $s \ge 1$3 convolutions to produce $s \ge 1$4, $s \ge 1$5, and $s \ge 1$6 (Wei et al., 7 May 2025). Recommended settings are $s \ge 1$7 event segments, channel count $s \ge 1$8, and local grid $s \ge 1$9 for the main model; the light variant reduces LIVT channels to 16 and uses $$f_\theta : (x, y, \mathcal{T}, s; V_{LR}, E) \rightarrow I(x,y,\mathcal{T}; s) \in \mathbb{R}^3,$$0 (Wei et al., 7 May 2025).

The papers explicitly contrast this unified space-time INR with decoupled spatial and temporal INRs used in models such as VideoINR and MoTIF, arguing that direct 3D local attention preserves cross-domain dependencies and improves OOD generalization (Wei et al., 7 May 2025, Wei et al., 4 Oct 2025).

5. Training, datasets, and empirical performance

EvEnhancer is trained end-to-end with a Charbonnier reconstruction loss on the Y channel:

$$f_\theta : (x, y, \mathcal{T}, s; V_{LR}, E) \rightarrow I(x,y,\mathcal{T}; s) \in \mathbb{R}^3,$$1

with $$f_\theta : (x, y, \mathcal{T}, s; V_{LR}, E) \rightarrow I(x,y,\mathcal{T}; s) \in \mathbb{R}^3,$$2 (Wei et al., 7 May 2025). The paper states that no perceptual, adversarial, or explicit temporal losses are used (Wei et al., 7 May 2025).

Training follows a two-stage schedule on Adobe240 (Wei et al., 7 May 2025). Stage 1 uses fixed $$f_\theta : (x, y, \mathcal{T}, s; V_{LR}, E) \rightarrow I(x,y,\mathcal{T}; s) \in \mathbb{R}^3,$$3 and $$f_\theta : (x, y, \mathcal{T}, s; V_{LR}, E) \rightarrow I(x,y,\mathcal{T}; s) \in \mathbb{R}^3,$$4 for 450K iterations. Stage 2 fine-tunes with $$f_\theta : (x, y, \mathcal{T}, s; V_{LR}, E) \rightarrow I(x,y,\mathcal{T}; s) \in \mathbb{R}^3,$$5 uniformly sampled in $$f_\theta : (x, y, \mathcal{T}, s; V_{LR}, E) \rightarrow I(x,y,\mathcal{T}; s) \in \mathbb{R}^3,$$6 for an additional 150K iterations (Wei et al., 7 May 2025). Optimization uses Adam with $$f_\theta : (x, y, \mathcal{T}, s; V_{LR}, E) \rightarrow I(x,y,\mathcal{T}; s) \in \mathbb{R}^3,$$7, $$f_\theta : (x, y, \mathcal{T}, s; V_{LR}, E) \rightarrow I(x,y,\mathcal{T}; s) \in \mathbb{R}^3,$$8, and cosine annealing from $$f_\theta : (x, y, \mathcal{T}, s; V_{LR}, E) \rightarrow I(x,y,\mathcal{T}; s) \in \mathbb{R}^3,$$9 to $(x_{lr}, y_{lr}) = (x_{hr}/s, y_{hr}/s)$0 over 150K iterations in fine-tuning (Wei et al., 7 May 2025). Data augmentation consists of random cropping of $(x_{lr}, y_{lr}) = (x_{hr}/s, y_{hr}/s)$1 LR patches, random rotations, and horizontal flips (Wei et al., 7 May 2025).

The principal datasets and settings reported in the paper are summarized below.

Category	Dataset	Details
Training/validation/test	Adobe240	133 sequences; 100 train / 16 val / 17 test
Synthetic test	GoPro, Adobe240	events simulated by vid2e
Real-world test	BS-ERGB, ALPIX-VSR	event+RGB evaluation

Performance is reported using PSNR and SSIM on the Y channel (Wei et al., 7 May 2025).

On the in-distribution setting $(x_{lr}, y_{lr}) = (x_{hr}/s, y_{hr}/s)$2, EvEnhancer achieves the following reported results (Wei et al., 7 May 2025):

Dataset split	EvEnhancer	EvEnhancer-light
GoPro-Center	33.52 dB / 0.9295	33.11 dB / 0.9242
GoPro-Average	33.30 dB / 0.9279	32.73 dB / 0.9203
Adobe240-Center	32.43 dB / 0.9129	31.90 dB / 0.9033
Adobe240-Average	32.18 dB / 0.9116	31.47 dB / 0.8988

The paper states that these results outperform HR-INR, MoTIF, and VideoINR while using fewer parameters, with parameter counts of 6.55M for EvEnhancer and 5.81M for EvEnhancer-light (Wei et al., 7 May 2025).

The model also reports strong OOD results on GoPro. Representative examples include $(x_{lr}, y_{lr}) = (x_{hr}/s, y_{hr}/s)$3 with 32.07 / 0.9116 for EvEnhancer; $(x_{lr}, y_{lr}) = (x_{hr}/s, y_{hr}/s)$4 with 28.86 / 0.8434; $(x_{lr}, y_{lr}) = (x_{hr}/s, y_{hr}/s)$5 with 25.50 / 0.7323; and temporal-only upsampling at $(x_{lr}, y_{lr}) = (x_{hr}/s, y_{hr}/s)$6 with 38.80 / 0.9714 (Wei et al., 7 May 2025). On BS-ERGB with fixed $(x_{lr}, y_{lr}) = (x_{hr}/s, y_{hr}/s)$7, EvEnhancer reports 25.44 / 0.7338 at $(x_{lr}, y_{lr}) = (x_{hr}/s, y_{hr}/s)$8, 24.70 / 0.7215 at $(x_{lr}, y_{lr}) = (x_{hr}/s, y_{hr}/s)$9, and 24.29 / 0.7138 at $I_0^{LR}$00 (Wei et al., 7 May 2025). On ALPIX-VSR for 2× VSR, it reports 40.84 dB / 0.9786 without additional fine-tuning (Wei et al., 7 May 2025).

Ablation studies identify the importance of both core modules. The paper attributes robustness at arbitrary scales to the unified space-time INR, cross-scale attention, and event-guided synthesis (Wei et al., 7 May 2025). Qualitative analyses further report clearer textures, sharper details, and smoother temporal evolution with fewer flickering artifacts than VideoINR and MoTIF, especially under fast motion, non-linear trajectories, low light, and HDR scenes (Wei et al., 7 May 2025).

6. EvEnhancerPlus: efficiency-oriented extension

EvEnhancerPlus extends EvEnhancer with two additional elements: a controllable switching mechanism (CSM) and a cross-derivative training strategy, termed staged cross-optimization in the detailed description (Wei et al., 4 Oct 2025). The goal is to reduce computational cost while preserving or improving reconstruction quality.

Controllable switching mechanism

The CSM is based on the observation that spatiotemporal pixels differ in reconstruction difficulty (Wei et al., 4 Oct 2025). Difficulty is estimated from event statistics using the relation between events and brightness changes:

$I_0^{LR}$01

After max-min normalization,

$I_0^{LR}$02

the per-pixel difficulty map at query time $I_0^{LR}$03 is

$I_0^{LR}$04

Pixels are then routed to one of $I_0^{LR}$05 upsamplers using a parameter-free distributor:

$I_0^{LR}$06

where

$I_0^{LR}$07

With $I_0^{LR}$08, the threshold $I_0^{LR}$09 controls the trade-off between simple and complex pathways (Wei et al., 4 Oct 2025).

The paper emphasizes that this mechanism is non-parametric, controllable, and flexible at inference, and reports substantial compute savings: at $I_0^{LR}$10, $I_0^{LR}$11, EvEnhancerPlus uses approximately 6.13 TFLOPs compared with 7.13 for EvEnhancer, described as a 12.8% reduction; the abstract summarizes this as about 85% of EvEnhancer’s cost (Wei et al., 4 Oct 2025).

Cross-derivative training strategy

The multi-pathway system is reported to be unstable under naive joint training, with the simpler path dominating gradients (Wei et al., 4 Oct 2025). To address this, EvEnhancerPlus introduces a staged training procedure:

1. Stage 1: independent pathway training at fixed $I_0^{LR}$12 and fixed $I_0^{LR}$13.
2. Stage 2: fine-tuning with variable $I_0^{LR}$14.
3. Stage 3: freezing the shared feature extractor from the most complex pathway and updating only the remaining upsamplers.

The paper states that this staged cross-initialization preserves the capacity of the complex pathway and stabilizes the multi-pathway system (Wei et al., 4 Oct 2025). The reported schedule is 450K iterations for Stage 1, 150K for Stage 2, and 150K for Stage 3, with Adam, batch size 4, cosine annealing from $I_0^{LR}$15 to $I_0^{LR}$16, and the same Y-channel Charbonnier loss (Wei et al., 4 Oct 2025).

Empirically, EvEnhancerPlus slightly improves or matches EvEnhancer on several benchmarks. On GoPro at $I_0^{LR}$17, $I_0^{LR}$18, it reports 33.57 dB / 0.9303 SSIM (Center) and 33.39 / 0.9291 (Average), compared with 33.52 / 0.9295 and 33.30 / 0.9279 for EvEnhancer (Wei et al., 4 Oct 2025). On ALPIX-VSR, EvEnhancerPlus reports 42.01 dB / 0.9802, compared with 40.84 / 0.9786 for EvEnhancer (Wei et al., 4 Oct 2025). The extension also reports improved OOD performance at settings such as $I_0^{LR}$19 and $I_0^{LR}$20 (Wei et al., 4 Oct 2025).

7. Positioning, ambiguities, and limitations

Within the event-guided C-STVSR literature, EvEnhancer is positioned against both frame-based continuous models and event-based alternatives. The papers explicitly compare against VideoINR and MoTIF, which decouple spatial and temporal INRs, and against HR-INR, which uses events but still relies on decoupled INR formulations (Wei et al., 7 May 2025, Wei et al., 4 Oct 2025). EvEnhancer’s distinct claim is that unified 3D INR with local cross-scale attention and event-guided synthesis offers better OOD generalization and stronger temporal consistency (Wei et al., 7 May 2025).

The method is also contrasted with two-stage event-based pipelines that combine video frame interpolation and video super-resolution, such as TimeLens, REFID, CBMNet-L, EGVSR, and EvTexture (Wei et al., 7 May 2025). The single-stage continuous formulation is presented as avoiding error accumulation and enabling arbitrary querying in both space and time (Wei et al., 7 May 2025).

There is, however, a nomenclatural ambiguity around the string “EvEnhancer.” The source material explicitly notes that “EvEnhancer” may refer to the system named VEnhancer in the video-generation paper “VEnhancer: Generative Space-Time Enhancement for Video Generation” (He et al., 2024). That system is a diffusion-based enhancement framework for AI-generated videos, not an event-guided C-STVSR model. It upscales spatial and temporal resolution jointly while removing artifacts and flicker through a pretrained video diffusion backbone with a video ControlNet (He et al., 2024). The same data block also includes an unrelated “Enhancer” module for speaker diarization in AED-EEND (Chen et al., 2023). These usages are technically distinct.

The limitations reported for EvEnhancer itself concern event quality, scene content, and extreme scale extrapolation. Strong event noise or extremely sparse events can degrade modulation and recurrent propagation, making robustness dependent on voxelization quality (Wei et al., 7 May 2025). In very static scenes, where event activity is scarce, gains over frame-only methods may shrink (Wei et al., 7 May 2025). Performance decreases at very large spatial or temporal scales, although the model remains competitive (Wei et al., 7 May 2025). The later extension similarly notes that sparse or noisy events and extreme OOD scales such as $I_0^{LR}$21 or $I_0^{LR}$22 remain challenging despite unified INR and event-guided synthesis (Wei et al., 4 Oct 2025).

Future directions named in the papers include explicit temporal consistency losses, adaptive binning, noise-robust event encodings, multi-head attention, stronger robustness to real event noise, and broader exploitation of dynamic routing for efficiency (Wei et al., 7 May 2025, Wei et al., 4 Oct 2025). This suggests that EvEnhancer should be understood not as a closed design, but as a family of event-guided continuous reconstruction methods whose main research significance lies in unifying event-aware synthesis with continuous implicit video representations.