Event Quality Score (EQS)

• EQS is a unified, differentiable metric that evaluates event stream realism via deep-feature extraction and fuzzy temporal matching.
• For simulated event cameras, EQS leverages a pretrained Recurrent Vision Transformer and patch-wise cosine distances to compare simulated and real streams.
• For time series detection, EQS uses a soft alignment mechanism with temporal tolerance, yielding a continuous F1 score that rewards near-miss predictions.

The Event Quality Score (EQS) serves as a quantitative metric for assessing either the realism of simulated event camera streams relative to real sensor outputs or the quality of event detection in time series. Originating independently in two domains—deep-feature–based evaluation of event streams for vision systems (Chanda et al., 16 Apr 2025) and soft evaluation of temporal event detection (Salles et al., 2023)—EQS provides unified, task-agnostic, and differentiable frameworks for comparing events in either high-dimensional latent spaces or with soft temporal tolerance. EQS enables practitioners to characterize the closeness of simulated or detected events to ground truth based on domain-appropriate definitions, leveraging either transformer-derived latent distances or temporally fuzzy overlaps.

1. Formal Definition of EQS

Deep Feature-Based EQS for Event Camera Streams

Let $S_{\text{sim}}^{(i)}$ and $S_{\text{real}}^{(i)}$ denote the $i$-th paired simulated and real event streams in a set of size $N$. Given a feature mapping $\phi(\cdot)$ extracting a $D$-dimensional latent vector (via Recurrent Vision Transformer, RVT), the EQS is

$$\text{EQS} = \frac{1}{N} \sum_{i=1}^{N} \lVert \phi(S_{\text{sim}}^{(i)}) - \phi(S_{\text{real}}^{(i)}) \rVert_p \,,$$

where $p$ denotes the norm (typically $p = 2$ or cosine distance) (Chanda et al., 16 Apr 2025).

SoftED EQS for Time Series Event Detection

Given time series $X$ of length $S_{\text{real}}^{(i)}$0, true event point set $S_{\text{real}}^{(i)}$1 at times $S_{\text{real}}^{(i)}$2, predicted detections $S_{\text{real}}^{(i)}$3 at times $S_{\text{real}}^{(i)}$4, and a tolerance $S_{\text{real}}^{(i)}$5 (window half-width), define a soft membership function:

$S_{\text{real}}^{(i)}$6

Event- and detection-level scores are assigned via greedy one-to-one matching, yielding soft-precision and soft-recall, and the EQS (Soft F$S_{\text{real}}^{(i)}$7):

$S_{\text{real}}^{(i)}$8

(Salles et al., 2023).

2. EQS for Event Camera Simulation: Workflow and Backbone

EQS for simulated event stream realism proceeds through a strict protocol:

• Convert raw event streams (tuple format) to $S_{\text{real}}^{(i)}$9 tensors, binning events by time and polarity.
• Input tensors to a pretrained RVT backbone: a hybrid network that incorporates spatial Vision Transformer blocks and ConvLSTM temporal recurrence.
• Extract convolutional feature maps at three spatial scales, downsampling by $i$0, $i$1, and $i$2.
• From each scale, patch-average activations to produce feature vectors.
• Compute the per-patch cosine distance between simulated and real feature vectors, average across all patches and scales, and finally across all stream pairs to yield the EQS.

Explicit pseudocode formalizes this pipeline (see Table 1).

Step	Operation	Output
Conversion	Events $i$3 2D tensor	$i$4
Backbone processing	Tensor $i$5 RVT (scales 1-3)	Feature maps
Patch-wise feature aggregation	Feature maps $i$6 patch vectors	Vectors per patch
Distance computation and averaging	Cosine similarity on patches	Scalar EQS

In this context, $i$7 comprises three patch-pooled intermediate convolutional activation tensors $i$8 from the RVT (Chanda et al., 16 Apr 2025).

3. Time Series Event Detection: Soft Matching Framework

EQS, as formalized by the SoftED framework, evaluates detection quality via a fuzzy-set–inspired mechanism. For every detection, the closest eligible event within the temporal window is determined; membership decreases linearly with temporal displacement (within $i$9). Each event claims its most aligned detection (breaking ties by maximal membership). True positives, false positives, and false negatives are computed with soft weighting rather than binary inclusion, culminating in a continuous-valued F$N$0 (the EQS).

This approach is robust to early or late detections, rewarding partially aligned predictions and mitigating binary misclassification artifacts (Salles et al., 2023).

4. Empirical Properties and Experimental Findings

In the sim-to-real validation on the DSEC automotive driving dataset, three simulators (V2E, ESIM, PIX2NVS) were calibrated and assessed:

• Table of average EQS:

Simulator	Full DSEC	Five seq. subset	Sim→Real mAP gap
V2E	0.773	0.758	22.8 pts
ESIM	0.866	0.828	15.8 pts
PIX2NVS	0.501	0.599	11.7 pts

• A higher EQS consistently predicted a smaller loss in mean average precision (mAP) when evaluating models trained on simulated streams and tested on real data, confirming a strong correlation between EQS and sim-to-real generalization (Chanda et al., 16 Apr 2025).

In SoftED experiments, the incorporation of a temporal tolerance ($N$1) led to more informative distinctions among detection methods in over 36% of experimental cases compared to standard “hard” evaluation metrics. Domain expertise is required for optimal $N$2 selection, ensuring the measure aligns with real-world utility (Salles et al., 2023).

5. Applications and Interpretable Insights

EQS has two principal application domains:

• Simulator benchmarking and optimization: EQS guides tuning of event camera simulators. End-to-end differentiability allows direct insertion as a loss term for optimizing video-to-event models or simulation parameters (e.g., noise models, contrast thresholds) (Chanda et al., 16 Apr 2025).
• Temporal event detection evaluation: As a soft-alignment–based measure, EQS reflects the practical impact of temporally near-miss detections, avoiding rigid penalties for slight temporal offsets (Salles et al., 2023).

A plausible implication is that maximizing EQS can drive development of simulators and detection algorithms that produce outputs more congruent with real-sensor data or operational needs.

6. Limitations and Prospective Extensions

Limitations of EQS as described in (Chanda et al., 16 Apr 2025) include reliance on the first three convolutional layers of RVT; incorporating additional temporal features such as ConvLSTM hidden states could enhance sensitivity to dynamic event characteristics. The scope of validation has so far been limited to automotive sequences; cross-domain benchmarking is needed. For SoftED-EQS, the linear consent model and window size selection are currently domain-driven rather than data-adaptive.

Future research directions highlighted are:

• Exploration of alternative distance metrics on latent features, such as other $N$3-norms or learned Mahalanobis distances.
• Integration of EQS directly into the training loops of differentiable event simulators for automated hyperparameter and noise model tuning.
• Extending evaluation to additional tasks (e.g., SLAM, optical flow) and scene types (indoor, outdoor) (Chanda et al., 16 Apr 2025).

7. Comparative Context and Unique Contributions

EQS, in both vision and event detection contexts, introduces a rigorously defined, task-agnostic, and directly computable metric for event stream comparison. In the vision domain, it constitutes the first differentiable measure capable of guiding simulation development by quantifying closeness in deep feature space. In time series, EQS improves evaluation transparency and utility by explicitly modeling temporal tolerance and one-to-one correspondence, addressing inadequacies in rigid classification-based metrics (Chanda et al., 16 Apr 2025, Salles et al., 2023).

By providing theoretically principled, implementation-friendly, and empirically validated metrics, EQS advances the quantitative study of both event simulation and detection quality.