Ev-Stereo3D: Event-Only Continuous 3D Detection

• Ev-Stereo3D is a stereo event-based system that uses asynchronous event cameras to achieve continuous-time 3D object detection during fast-motion driving.
• It leverages dual semantic-geometric filtering to jointly refine depth estimation and fuse stereo disparity with high temporal resolution cues.
• The framework employs object-centric ROI alignment and 3D voxel construction to overcome blind sensor intervals and handle rapid scene changes.

Searching arXiv for Ev-Stereo3D and closely related event-stereo papers to ground the article. Ev-Stereo3D is a stereo event-camera framework for continuous-time 3D object detection in dynamic driving scenes, particularly during blind time, defined as periods between synchronized sensor frames when conventional RGB/LiDAR data are unavailable. It is presented in “Unleashing the Temporal Potential of Stereo Event Cameras for Continuous-Time 3D Object Detection” (Kang et al., 4 Aug 2025). The method is described as the first fully asynchronous stereo event-camera system for 3D object detection, using only stereo event cameras and detecting 3D boxes at arbitrary timestamps. Its central premise is that stereo events can provide both temporal continuity and geometric cues through disparity, but that event data are sparse and weak in direct semantic and geometric content; accordingly, the architecture is organized around joint extraction and coupling of semantic and geometric information (Kang et al., 4 Aug 2025).

1. Problem formulation and research context

Ev-Stereo3D addresses continuous-time 3D object detection under conditions in which frame-based sensors create perception gaps. The motivating setting is fast-motion driving, where LiDAR and RGB cameras typically operate at 10–20 Hz and objects can move substantially before the next synchronized frame. The paper contrasts this with prior event-based continuous-time methods such as Ev-3DOD, which still rely heavily on synchronized LiDAR/RGB for geometry and use events primarily to bridge blind intervals; the reported consequence is degradation when scene changes during blind time are large (Kang et al., 4 Aug 2025).

The framework is explicitly stereo rather than monocular. In the paper’s formulation, stereo event cameras provide asynchronous sensing, very high temporal resolution, and low latency, while also supplying geometric cues through disparity. This removes the dependence on conventional 3D sensors and is intended to maintain robustness when motion is large, scene geometry changes quickly, or new objects appear during blind intervals (Kang et al., 4 Aug 2025).

This positioning places Ev-Stereo3D within a broader line of event-stereo research that treats events as a basis for 3D inference under extreme dynamics. A related but different line of work is correspondence-free stereo depth estimation from ego-motion, where per-camera ray-density DSIs are fused using generalized means such as the harmonic mean, geometric mean, or minimum rather than explicit event matching (Ghosh et al., 2022). Another related direction is event-based stereo disparity estimation with deep networks, exemplified by EV-MGDispNet, which emphasizes clearer event representations, reduced pixel shifts before cost-volume construction, and left-right consistency supervision (Jiang et al., 2024). Ev-Stereo3D differs from both by targeting full 3D object detection rather than only depth or disparity estimation.

2. Event representation and continuous-time formulation

The temporal input to Ev-Stereo3D is a continuous-time window of recent events,

$$\mathcal{E}_{(\tau - \Delta \tau) \rightarrow \tau} = \{(u, v, t, p) \mid \tau - \Delta \tau \leq t < \tau\},$$

so that detection at time $\tau$ depends only on recent events over interval $\Delta \tau$ (Kang et al., 4 Aug 2025). This formulation is described as time-agnostic and enables inference at arbitrary timestamps.

The underlying event stream is

$$\mathcal{E} = \{e_i\}_{i=1}^N, \quad e_i = (u_i, v_i, t_i, p_i),$$

with the triggering condition

$$L(u,v,t) - L(u,v,t-\Delta t) \geq \mathbf{p}\mathcal{C}, \quad \mathbf{p} \in \{-1,+1\}.$$

Events are discretized into a spatio-temporal voxel grid,

$E(u, v, b) = \sum _{i=1}^N p_i k_{\mathcal{B}(u - u_i) k_{\mathcal{B}(v - v_i) k_{\mathcal{B} (b - b^*_i)}$

with

$b^*_i = (B - 1)(t_i - t_1)/(t_N - t_1).$

The role of this voxelization is to preserve temporal ordering while enabling CNN-style feature extraction, and it is performed separately for the left and right event streams, $E_L, E_R$. The implementation uses 5 temporal bins in the voxel grid (Kang et al., 4 Aug 2025).

A plausible implication is that the method adopts the standard compromise in event-camera learning systems: it does not remain fully event-asynchronous at the level of raw signal processing, but instead converts recent events into a compact spatio-temporal tensor suitable for dense stereo and 3D detection backbones. That interpretation is consistent with other event-stereo pipelines that likewise convert events into structured intermediate representations before matching or volume construction (Jiang et al., 2024).

3. Network architecture and stereo geometric reasoning

The end-to-end framework comprises four stages: event voxelization, geometric plane-sweep volume construction, dual semantic-geometric filter, and global 3D detection with object-centric ROI alignment (Kang et al., 4 Aug 2025).

The front-end stereo feature extractor is inspired by PSMNet and is modified to produce two feature types for each camera:

• semantic features $F^{sem}_{L,R}$,
• geometric features $F^{geo}_{L,R}$,

both of shape $\tau$0 (Kang et al., 4 Aug 2025). The semantic branch is described as activating on object regions, while the geometric branch focuses on matching or correspondence over the full scene.

For each candidate depth level $\tau$1, left and right geometric features are concatenated into a plane-sweep volume: $\tau$2 where $\tau$3 is the focal length, $\tau$4 is the stereo baseline, and $\tau$5 is the depth corresponding to index $\tau$6 (Kang et al., 4 Aug 2025). This volume is processed by 3D convolutions to produce a depth probability volume

$\tau$7

from which the initial depth $\tau$8 is computed as a weighted sum over depth bins.

The stereo volume is then lifted to 3D voxels via

$\tau$9

forming the initial geometric 3D representation (Kang et al., 4 Aug 2025).

This plane-sweep construction makes Ev-Stereo3D structurally closer to stereo disparity networks than to event-only object detectors. In that respect it is aligned with event-stereo work such as EV-MGDispNet, which also constructs stereo feature pyramids and cost volumes for disparity inference (Jiang et al., 2024). The difference is that Ev-Stereo3D carries the stereo geometry forward into a 3D voxelized detection representation rather than stopping at a 2D disparity map.

4. Dual semantic-geometric filter

The dual semantic-geometric filter is identified as the key novelty of the method. Its motivation is that semantic features are useful for object localization and geometric features are useful for depth or disparity estimation, but neither is sufficient alone (Kang et al., 4 Aug 2025). The filter operates in two directions.

Semantic-guided depth refinement

First, semantic features refine the geometric depth distribution. Using the predicted depth, the method computes left-right semantic similarity: $\Delta \tau$0 and a confidence term

$\Delta \tau$1

Here $\Delta \tau$2 measures stereo semantic consistency and $\Delta \tau$3 measures uncertainty or spread in the depth distribution (Kang et al., 4 Aug 2025).

The refined volume is obtained by neighboring aggregation: $\Delta \tau$4

$\Delta \tau$5

and the refined depth $\Delta \tau$6 is then derived from $\Delta \tau$7 (Kang et al., 4 Aug 2025).

The stated interpretation is that if semantic agreement is high and uncertainty is well behaved, those depth hypotheses should be trusted more. This suggests an explicit coupling between recognition-oriented object evidence and correspondence-oriented stereo evidence, rather than treating semantics and geometry as independent streams.

Geometric-filtered semantic volume

Second, refined depth improves the semantic representation. The right semantic feature is warped into the left view,

$\Delta \tau$8

and then fused using transformer-style channel attention: $\Delta \tau$9

$$\mathcal{E} = \{e_i\}_{i=1}^N, \quad e_i = (u_i, v_i, t_i, p_i),$$0

$$\mathcal{E} = \{e_i\}_{i=1}^N, \quad e_i = (u_i, v_i, t_i, p_i),$$1

The stated role of this stage is to use geometry-guided warping to bring right-view semantics into the left view, while attention suppresses occlusion and misalignment artifacts, thereby producing richer semantics for object localization (Kang et al., 4 Aug 2025).

Semantic features are then lifted to 3D and masked by the geometric probabilities: $$\mathcal{E} = \{e_i\}_{i=1}^N, \quad e_i = (u_i, v_i, t_i, p_i),$$2 This maintains geometry awareness in the semantic volume (Kang et al., 4 Aug 2025).

The geometric and semantic 3D voxels, $$\mathcal{E} = \{e_i\}_{i=1}^N, \quad e_i = (u_i, v_i, t_i, p_i),$$3 and $$\mathcal{E} = \{e_i\}_{i=1}^N, \quad e_i = (u_i, v_i, t_i, p_i),$$4, are concatenated and merged with 3D convolution to form a unified 3D voxel $$\mathcal{E} = \{e_i\}_{i=1}^N, \quad e_i = (u_i, v_i, t_i, p_i),$$5, which is consumed by the detector (Kang et al., 4 Aug 2025).

5. Detection head and object-centric ROI alignment

The global detector is anchor-based and operates on BEV features extracted from $$\mathcal{E} = \{e_i\}_{i=1}^N, \quad e_i = (u_i, v_i, t_i, p_i),$$6 (Kang et al., 4 Aug 2025). Each BEV location has fixed-size class-specific anchors

$$\mathcal{E} = \{e_i\}_{i=1}^N, \quad e_i = (u_i, v_i, t_i, p_i),$$7

and the network predicts offsets

$$\mathcal{E} = \{e_i\}_{i=1}^N, \quad e_i = (u_i, v_i, t_i, p_i),$$8

The decoded global box is

$$\mathcal{E} = \{e_i\}_{i=1}^N, \quad e_i = (u_i, v_i, t_i, p_i),$$9

This stage provides coarse scene-level detection (Kang et al., 4 Aug 2025).

The second major novelty after the dual filter is object-centric ROI alignment. The paper motivates this by noting that global regression can remain ambiguous because objects move, the ego vehicle moves, event data are sparse, and geometric cues alone may not localize exact box boundaries (Kang et al., 4 Aug 2025).

The method uses the semantic BEV feature because it contains strong object boundary evidence. A predicted global box $$L(u,v,t) - L(u,v,t-\Delta t) \geq \mathbf{p}\mathcal{C}, \quad \mathbf{p} \in \{-1,+1\}.$$0 is divided into a $$L(u,v,t) - L(u,v,t-\Delta t) \geq \mathbf{p}\mathcal{C}, \quad \mathbf{p} \in \{-1,+1\}.$$1 grid, with $$L(u,v,t) - L(u,v,t-\Delta t) \geq \mathbf{p}\mathcal{C}, \quad \mathbf{p} \in \{-1,+1\}.$$2 in the implementation. Each grid cell is pooled from the semantic BEV feature map to gather local evidence around the object. The pooled feature is

$$L(u,v,t) - L(u,v,t-\Delta t) \geq \mathbf{p}\mathcal{C}, \quad \mathbf{p} \in \{-1,+1\}.$$3

and is passed through an MLP to predict local offsets

$$L(u,v,t) - L(u,v,t-\Delta t) \geq \mathbf{p}\mathcal{C}, \quad \mathbf{p} \in \{-1,+1\}.$$4

The final refined box is

$$L(u,v,t) - L(u,v,t-\Delta t) \geq \mathbf{p}\mathcal{C}, \quad \mathbf{p} \in \{-1,+1\}.$$5

The stated role is coarse-to-fine detection using object-centric semantic context, improving box regression under motion blur, dynamic scenes, and sparse events (Kang et al., 4 Aug 2025).

A plausible implication is that ROI alignment partially compensates for the limited precision of global voxelized detection when event evidence is discontinuous. That interpretation is supported by the ablation table, where adding ROI alignment on top of semantic features or on top of the dual filter improves both 3D mAP and BEV mAP (Kang et al., 4 Aug 2025).

6. Optimization, evaluation protocol, and empirical findings

The total loss is

$$L(u,v,t) - L(u,v,t-\Delta t) \geq \mathbf{p}\mathcal{C}, \quad \mathbf{p} \in \{-1,+1\}.$$6

where the terms correspond to initial depth supervision, refined depth supervision, auxiliary 2D detection, classification, global 3D regression, and local ROI alignment regression (Kang et al., 4 Aug 2025). The refined depth loss is

$$L(u,v,t) - L(u,v,t-\Delta t) \geq \mathbf{p}\mathcal{C}, \quad \mathbf{p} \in \{-1,+1\}.$$7

and the regression losses are

$$L(u,v,t) - L(u,v,t-\Delta t) \geq \mathbf{p}\mathcal{C}, \quad \mathbf{p} \in \{-1,+1\}.$$8

with $$L(u,v,t) - L(u,v,t-\Delta t) \geq \mathbf{p}\mathcal{C}, \quad \mathbf{p} \in \{-1,+1\}.$$9 denoting the ground-truth 3D box (Kang et al., 4 Aug 2025).

At inference time, the procedure is: collect a recent event window $E(u, v, b) = \sum _{i=1}^N p_i k_{\mathcal{B}(u - u_i) k_{\mathcal{B}(v - v_i) k_{\mathcal{B} (b - b^*_i)}$0, convert it to left and right voxel grids, extract semantic and geometric stereo features, build the geometric volume, refine depth with dual semantic-geometric filtering, create the fused 3D voxel representation, run the global anchor-based 3D detector, apply ROI alignment for local refinement, and output the final 3D boxes $E(u, v, b) = \sum _{i=1}^N p_i k_{\mathcal{B}(u - u_i) k_{\mathcal{B}(v - v_i) k_{\mathcal{B} (b - b^*_i)}$1 (Kang et al., 4 Aug 2025). The continuous-time aspect is that inference can be run at arbitrary timestamps with varying $E(u, v, b) = \sum _{i=1}^N p_i k_{\mathcal{B}(u - u_i) k_{\mathcal{B}(v - v_i) k_{\mathcal{B} (b - b^*_i)}$2, including 100 FPS blind-time evaluation.

The experiments use DSEC-3DOD, a real-world event-based 3D detection dataset built on DSEC stereo event driving data. It contains manual 10 FPS annotations for active time and 100 FPS blind-time annotations created by interpolation or refinement. The classes are vehicle and pedestrian. Evaluation follows the KITTI 3D detection metric using AP$E(u, v, b) = \sum _{i=1}^N p_i k_{\mathcal{B}(u - u_i) k_{\mathcal{B}(v - v_i) k_{\mathcal{B} (b - b^*_i)}$3 and AP$E(u, v, b) = \sum _{i=1}^N p_i k_{\mathcal{B}(u - u_i) k_{\mathcal{B}(v - v_i) k_{\mathcal{B} (b - b^*_i)}$4, separated by class and easy or moderate difficulty. Events are sliced into 10 ms windows and voxelized into 5 bins. The paper also introduces motion scale (MS) and time slice (TS) to test temporal robustness under longer blind intervals and larger motion, with typical settings MS $E(u, v, b) = \sum _{i=1}^N p_i k_{\mathcal{B}(u - u_i) k_{\mathcal{B}(v - v_i) k_{\mathcal{B} (b - b^*_i)}$5 and TS $E(u, v, b) = \sum _{i=1}^N p_i k_{\mathcal{B}(u - u_i) k_{\mathcal{B}(v - v_i) k_{\mathcal{B} (b - b^*_i)}$6 (Kang et al., 4 Aug 2025).

The following table summarizes the main quantitative figures explicitly reported.

Setting	AP$E(u, v, b) = \sum _{i=1}^N p_i k_{\mathcal{B}(u - u_i) k_{\mathcal{B}(v - v_i) k_{\mathcal{B} (b - b^*_i)}$7	AP$E(u, v, b) = \sum _{i=1}^N p_i k_{\mathcal{B}(u - u_i) k_{\mathcal{B}(v - v_i) k_{\mathcal{B} (b - b^*_i)}$8
Vehicle Easy	23.47	40.13
Vehicle Moderate	19.62	33.03
Pedestrian Easy	19.86	22.91
Pedestrian Moderate	12.93	14.34

On the main benchmark table, these are the results of the proposed Event-Stereo method (Kang et al., 4 Aug 2025). The paper states that it clearly outperforms frame-based stereo methods such as DSGN and LIGA, is competitive with or better than some LiDAR-based methods in blind time, and is better than prior event-stereo or event-only stereo baselines especially on pedestrians (Kang et al., 4 Aug 2025).

The fast-motion evaluation is central to the paper’s argument. Under easy difficulty with MS $E(u, v, b) = \sum _{i=1}^N p_i k_{\mathcal{B}(u - u_i) k_{\mathcal{B}(v - v_i) k_{\mathcal{B} (b - b^*_i)}$9, TS $b^*_i = (B - 1)(t_i - t_1)/(t_N - t_1).$0, the reported results are:

• Ours VEH: 23.47 / 40.13
• Ev-3DOD VEH: 15.96 / 31.13
• Ours PED: 19.86 / 22.91
• Ev-3DOD PED: 1.47 / 2.63

Under MS $b^*_i = (B - 1)(t_i - t_1)/(t_N - t_1).$1, TS $b^*_i = (B - 1)(t_i - t_1)/(t_N - t_1).$2:

• Ours VEH: 20.70 / 37.08
• Ev-3DOD VEH: 6.24 / 11.62
• Ours PED: 16.71 / 21.93
• Ev-3DOD PED: 3.04 / 4.09

The paper interprets these results as evidence that Ev-3DOD deteriorates as motion increases because it depends on past LiDAR geometry, whereas Ev-Stereo3D remains stable because it computes geometry directly from current event stereo data (Kang et al., 4 Aug 2025).

Ablation studies further isolate the proposed components. Starting from only the geometric plane-sweep volume, the baseline yields 3D mAP $b^*_i = (B - 1)(t_i - t_1)/(t_N - t_1).$3 and BEV mAP $b^*_i = (B - 1)(t_i - t_1)/(t_N - t_1).$4. Adding semantic information only raises this to $b^*_i = (B - 1)(t_i - t_1)/(t_N - t_1).$5. Adding object-centric ROI alignment with the semantic voxel gives $b^*_i = (B - 1)(t_i - t_1)/(t_N - t_1).$6. With dual semantic-geometric filtering, SDR only yields $b^*_i = (B - 1)(t_i - t_1)/(t_N - t_1).$7, GSV only yields $b^*_i = (B - 1)(t_i - t_1)/(t_N - t_1).$8, and both SDR + GSV yield $b^*_i = (B - 1)(t_i - t_1)/(t_N - t_1).$9. Adding ROI alignment on top produces the best reported ablation result, SDR + GSV + OCRA: $E_L, E_R$0 (Kang et al., 4 Aug 2025).

The paper also reports that semantic-guided depth refinement improves depth accuracy across all outlier thresholds, with example values changing from $E_L, E_R$1 without SDR to $E_L, E_R$2 with SDR (Kang et al., 4 Aug 2025). This is presented as evidence that the refinement module improves geometric quality, not only detection.

7. Significance, misconceptions, and limitations

Ev-Stereo3D is best understood as a stereo event-only 3D detection system rather than a generic event-stereo depth estimator. A common misconception would be to equate it with event-based stereo depth or disparity methods. By design, it includes a stereo geometric backbone, but its task is object detection with 3D boxes at arbitrary timestamps, not only depth reconstruction (Kang et al., 4 Aug 2025). This distinguishes it from correspondence-free ego-motion stereo depth estimation based on ray-density fusion (Ghosh et al., 2022) and from disparity networks such as EV-MGDispNet (Jiang et al., 2024).

Another potential misconception is that the method is simply replacing RGB or LiDAR with events while keeping the rest of the pipeline unchanged. The paper’s actual claim is narrower and more specific: it combines geometric stereo reasoning, semantic event cues, cross-filtering between the two, and object-centric local refinement. The dual semantic-geometric filter and ROI alignment are presented as essential components rather than optional additions (Kang et al., 4 Aug 2025).

The paper explicitly notes or implies several limitations. It does not match LiDAR-fusion methods in absolute performance in some settings because it lacks direct range sensing. Event data are sparse, making learning harder than from RGB/LiDAR. Stereo event detection still depends on good correspondence estimation, so very challenging textures and occlusions can be difficult. The benchmark is mainly on DSEC-3DOD with two classes, vehicle and pedestrian, so broader generalization remains to be tested (Kang et al., 4 Aug 2025).

The RGB fusion ablation reinforces the method’s main claim about temporal robustness. Under small motion, synchronous RGB helps, with Ours (E): $E_L, E_R$3 and Ours (E+I): $E_L, E_R$4. Under larger motion, RGB fusion hurts, with Ours (E): $E_L, E_R$5 and Ours (E+I): $E_L, E_R$6 (Kang et al., 4 Aug 2025). This suggests that the advantages of event-only stereo are not merely sensor redundancy, but specifically the ability to avoid synchronization liabilities when motion is large.

Within the broader literature, Ev-Stereo3D marks a shift from using events as auxiliary signals to using stereo events as the sole basis for continuous-time 3D perception. The paper’s reported qualitative observations—that conventional sensor-based methods fail during blind time, that Ev-3DOD can propagate detections but deteriorates as time moves away from active timestamps, and that Ev-Stereo3D can detect new objects appearing during blind time while remaining stable under large motion—summarize the intended contribution of the framework (Kang et al., 4 Aug 2025).