Event Probability Mask (EPM)

• Event Probability Mask (EPM) is a probabilistic framework for labeling pixel-wise, spatio-temporal data in neuromorphic camera streams by leveraging APS intensity frames and IMU measurements.
• It computes soft event labels by correlating intensity changes with sensor data, thereby providing objective benchmarks for event denoising and performance evaluation.
• EPM enables the training of event denoising networks and sensor parameter calibration, as validated on the DVSNOISE20 benchmark with improved noise robustness.

The Event Probability Mask (EPM) is a probabilistic framework for labeling pixel-wise spatio-temporal data in neuromorphic, or event-based, camera streams. It computes the likelihood that a real event—registered by a dynamic vision sensor (DVS)—should have occurred at each pixel within a specified temporal window, based on the underlying intensity dynamics captured by synchronous active pixel sensor (APS) frames, camera intrinsics, and inertial measurement unit (IMU) data. EPM enables objective benchmarking of event denoising, provides soft-label ground truth for training event denoising networks, and permits principled calibration of neuromorphic sensor parameters (Baldwin et al., 2020).

1. Formal Definition of Event Probability Mask

Consider an APS intensity video $I:\mathbb{Z}^2 \times \mathbb{R} \to \mathbb{R}$ representing pixel radiance at spatial coordinate $X \in \mathbb{Z}^2$ and time $t \in \mathbb{R}$. The DVS log-intensity is

$J(X,t) = \log(a I(X,t) + b)$

where $a$ and $b$ are gain and offset parameters, respectively. Ideal (noise-free) DVS pixels trigger events when the log-intensity $J$ undergoes a change of at least $\epsilon$:

• $$t_i(X) := \min\left\{ t > t_{i-1}(X) \mid |J(X,t) - J(X, t_{i-1}(X))| \geq \epsilon \right\}$$
• $$p_i(X) := \operatorname{sign}(J(X, t_i) - J(X, t_{i-1}(X)))$$

For any real (possibly noisy) event at location and time $X \in \mathbb{Z}^2$0, the goal is to assign a soft label $X \in \mathbb{Z}^2$1 indicating the probability that an event "should" have occurred in the preceding APS exposure window $X \in \mathbb{Z}^2$2. The binary event indicator function is defined by:

$X \in \mathbb{Z}^2$3

Under pure rotational motion and piecewise-constant illumination, the EPM is given by:

$X \in \mathbb{Z}^2$4

where $X \in \mathbb{Z}^2$5. Here,

• $X \in \mathbb{Z}^2$6 is the per-pixel image velocity, computed from IMU angular velocity $X \in \mathbb{Z}^2$7 and camera intrinsics $X \in \mathbb{Z}^2$8 via $X \in \mathbb{Z}^2$9, with $t \in \mathbb{R}$0 the skew-symmetric matrix of $t \in \mathbb{R}$1.
• $t \in \mathbb{R}$2, where $t \in \mathbb{R}$3 is the APS measurement and $t \in \mathbb{R}$4 the combined offset.

2. EPM Computation Algorithm

The calculation of the EPM proceeds as follows:

1. Input data:
   • Continuous DVS event stream $t \in \mathbb{R}$5
   • APS frame sequence $t \in \mathbb{R}$6 for $t \in \mathbb{R}$7 and exposure time $t \in \mathbb{R}$8
   • Synchronized IMU gyroscope $t \in \mathbb{R}$9
   • Camera intrinsics $J(X,t) = \log(a I(X,t) + b)$0
   • Parameters: contrast threshold $J(X,t) = \log(a I(X,t) + b)$1, APS-to-DVS offset $J(X,t) = \log(a I(X,t) + b)$2
2. Procedure:
   • Preprocess APS: compute spatial gradients $J(X,t) = \log(a I(X,t) + b)$3 for all frames $J(X,t) = \log(a I(X,t) + b)$4.
   • For each APS frame $J(X,t) = \log(a I(X,t) + b)$5 at time $J(X,t) = \log(a I(X,t) + b)$6:
     • Interpolate $J(X,t) = \log(a I(X,t) + b)$7 over $J(X,t) = \log(a I(X,t) + b)$8.
     • Compute $J(X,t) = \log(a I(X,t) + b)$9 using camera intrinsics and IMU.
     • Compute $a$0.
     • Compute $a$1.
     • Compute $a$2 using the EPM closed-form expression.
   • Store $a$3 for all pixels and frames as the event probability masks.

3. Benchmarking and Denoising Applications

EPM enables several key applications in neuromorphic vision processing:

• Objective Denoising Benchmark (RPMD): For any denoised event indicator $a$4, its log-probability under the EPM is:

$a$5

The oracle classifier $a$6 if $a$7, else $a$8. The Relative Plausibility Measure of Denoising (RPMD) quantifies denoising performance as:

$a$9

Lower RPMD indicates better denoising, with $b$0 being optimal.

• Training Event Denoising CNN (EDnCNN): EPM supplies soft ground truth masks for supervised learning of pixel-wise binary classifiers $b$1. Training can minimize one of three equivalent loss functions—maximum likelihood, $b$2 loss, or classification error to $b$3—for large datasets.
• Internal Parameter Calibration: The optimal contrast threshold $b$4 and offset $b$5 are estimated by maximizing the likelihood of observed raw DVS events under the EPM, i.e.,

$b$6

A two-stage one-dimensional search (first over $b$7, then $b$8) is sufficient for practical purposes.

4. Experimental Evaluation and DVSNOISE20

The DVSNOISE20 benchmark dataset underpins empirical analyses of EPM and EDnCNN:

• Platform: DAVIS346 (346×260 DVS pixels, 40 fps APS, 6-axis IMU)
• Scenes: 16 static indoor/outdoor scenes per 2-axis gimbal rotation, each recorded $b$9 s
• APS settings: 41–56 fps, $J$0
• Labels: EPM-generated “soft” masks across all pixels/frames serve as denoising ground truth
• Volume: 48 sequences, approximately $J$1 events in total

Experimental Results

• Real-world Denoising: EDnCNN trained on EPM achieved an average RPMD gain of 148 points over raw data in leave-one-scene-out tests, outperforming eight other denoising filters in 12 out of 16 scenes ($J$2, Wilcoxon signed-rank test).
• Simulated Noise Robustness: In ESIM-simulated data with injected background-activity noise, raw-data RPMD scaled linearly with noise rate, whereas EDnCNN maintained near-optimal performance; other methods degraded more quickly.
• Generalization: EDnCNN demonstrated qualitative denoising efficacy on DVSFLOW16 and IROS18 datasets, preserving edges amid translational motion and multiple moving objects.

5. Limitations and Future Directions

Several constraints underlie the current EPM formulation:

• Assumes pure rotational motion and static scenes during APS exposures to avoid occlusions.
• Neglects rapid illumination flicker and requires constant scene brightness.
• EPM values diminish at low velocities (low event rates), reducing sensitivity for noise discrimination.

Potential extensions include accommodating translation-induced depth variation, explicitly segmenting dynamic foreground/background components, and addressing flickering illumination. Integrating EPM-based principles into unsupervised loss formulations could further reduce reliance on synchronized APS and IMU signals.

6. Summary Table: EPM Key Components and Applications

Component/Application	Description	Formula/Method
Event Probability Mask	Soft label: $J$3 under null hypothesis of event occurrence	$J$4 closed-form as above
Denoising Benchmark (RPMD)	Relative plausibility measure for denoising	$J$5
Training EDnCNN	EPM as ground truth for supervised learning	Minimize maximum likelihood/$J$6/classification loss
Parameter Calibration	Estimate $J$7 maximizing event likelihood under EPM	2-stage 1D search via $J$8

EPM constitutes a mathematically principled framework for probabilistic event labeling in neuromorphic vision, enabling standardized objective benchmarks and effective supervised learning paradigms for event denoising (Baldwin et al., 2020).