EPRBench: A High-Quality Benchmark Dataset for Event Stream Based Visual Place Recognition

Abstract: Event stream-based Visual Place Recognition (VPR) is an emerging research direction that offers a compelling solution to the instability of conventional visible-light cameras under challenging conditions such as low illumination, overexposure, and high-speed motion. Recognizing the current scarcity of dedicated datasets in this domain, we introduce EPRBench, a high-quality benchmark specifically designed for event stream-based VPR. EPRBench comprises 10K event sequences and 65K event frames, collected using both handheld and vehicle-mounted setups to comprehensively capture real-world challenges across diverse viewpoints, weather conditions, and lighting scenarios. To support semantic-aware and language-integrated VPR research, we provide LLM-generated scene descriptions, subsequently refined through human annotation, establishing a solid foundation for integrating LLMs into event-based perception pipelines. To facilitate systematic evaluation, we implement and benchmark 15 state-of-the-art VPR algorithms on EPRBench, offering a strong baseline for future algorithmic comparisons. Furthermore, we propose a novel multi-modal fusion paradigm for VPR: leveraging LLMs to generate textual scene descriptions from raw event streams, which then guide spatially attentive token selection, cross-modal feature fusion, and multi-scale representation learning. This framework not only achieves highly accurate place recognition but also produces interpretable reasoning processes alongside its predictions, significantly enhancing model transparency and explainability. The dataset and source code will be released on https://github.com/Event-AHU/Neuromorphic_ReID

• The paper introduces EPRBench, a 13,109-sample event-camera dataset spanning 1,022 scenes, three environments, varied weather, lighting, and viewpoints, with rich textual annotations.
• The paper presents SG-VPR, which combines DINOv2 event features, CLIP text embeddings, selective local semantic fusion, and multi-scale pooling to achieve 94.3% Recall@1 on EPRBench.
• The paper shows that semantic guidance improves robustness to event sparsity and noise, while its LLM-generated explanations enhance interpretability but remain limited by dataset-specific scene-expert training.

This paper introduces EPRBench, a large-scale benchmark dataset for event stream–based Visual Place Recognition (VPR), together with SG-VPR, a semantic-guided multi-modal retrieval framework that incorporates LLM-generated scene descriptions and chain-of-thought reasoning. The work addresses three gaps the authors identify in event-based VPR: dataset scarcity, insufficient semantic exploitation, and poor interpretability of retrieval decisions.

Motivation and positioning

Conventional RGB cameras degrade under low illumination, overexposure, high dynamic range demands, and motion blur at typical frame rates (~30 FPS). Event cameras mitigate these issues with 120–140 dB dynamic range, asynchronous acquisition, low latency, and low power consumption. Prior event-based VPR datasets — Brisbane-Event-VPR (2020), NeuroGPR (2023), and NYC-Event-VPR (2024) — are limited in scale and diversity; none provide textual annotations. EPRBench is positioned as the first event-based VPR dataset with rich semantic annotations, enabling cross-modal alignment between sparse event streams and language.

EPRBench dataset

The data were captured with a Prophesee EVK4-HD sensor at $1280 \times 720$ resolution using both handheld and vehicle-mounted setups, covering Campus, Park, and Road scene categories across multiple viewpoints, weather conditions (clear/rainy), and lighting conditions (day/night). The dataset comprises 13,109 event scene samples (each with five event frames) spanning 1,022 scenes: 315 Campus scenes (2,766 samples), 351 Park scenes (5,176 samples), and 356 Road scenes (5,167 samples). A 7:1:2 train/validation/test split yields 9,220 / 1,273 / 2,616 samples. Relative to prior benchmarks, EPRBench offers roughly an order-of-magnitude more scenes than NYC-Event-VPR (16 scenes) or Brisbane-Event-VPR (6 scenes).

Text annotation pipeline. Scene descriptions are produced via a three-stage pipeline: (1) template-prompted generation of Chinese chain-of-thought (CoT) text over 50,000 balanced positive/negative image pairs using Doubao Pro-1.6; (2) translation to English with NLLB-200-3B followed by refinement with DeepSeek; (3) LoRA-based supervised fine-tuning of Qwen2.5-VL-7B as a "scene description expert" that generates descriptions for the dataset. Human verification is applied to ensure accuracy. One caveat: the resulting expert model is tuned to this specific data distribution, which constrains transferability (see limitations below).

Benchmark baselines

The authors retrain and evaluate 15 representative VPR algorithms on EPRBench, spanning CNN-based global descriptors (CosPlace, MixVPR, EigenPlaces), Transformer-based aggregators (CricaVPR, BoQ, SALAD, SuperVLAD, EMVP, SelaVPR, ImAge, PRGS), and re-ranking methods (R2Former, Pair-VPR, FOL). This establishes a systematic reference point for future comparisons.

SG-VPR methodology

SG-VPR is a dual-stream framework processing event frames and text in parallel:

• Dual-stream encoding: events within a fixed interval $\Delta T$ are accumulated by polarity into a 2D event frame, encoded by a frozen DINOv2 backbone into patch tokens $v_{patch}$ and a class token $v_{cls}$. Text is encoded by a frozen CLIP Text Encoder into word-level ($t_w$) and sentence-level ($t_s$) features.
• Global fusion: concatenation of $v_{cls}$ and $t_s$ through an MLP produces a multi-modal semantic anchor $\mathbf{f}_{global}$.
• Local fusion: each visual patch receives a relevance score equal to its maximum cosine similarity against word tokens; the top $\rho = 0.25$ fraction of patches is retained, and semantic injection is applied only to selected patches via element-wise interaction with their most similar word token, scaled by a learnable factor $\Delta T$0. Unselected background patches remain unchanged.
• Spatial pyramid aggregation: patch tokens are reshaped into a feature map; GeM pooling over $\Delta T$1 and $\Delta T$2 grids is concatenated with the global anchor.
• Auxiliary reasoning: an optional LLM decoder generates CoT explanations justifying match/reject decisions, providing human-readable interpretability beyond activation heatmaps.

Training uses Multi-Similarity loss for retrieval plus a symmetric InfoNCE contrastive loss for cross-modal alignment, weighted as $\Delta T$3 with $\Delta T$4. Only adapters, projection layers, and the fusion module are trained; backbones remain frozen.

Experimental results

Evaluation uses Recall@N on EPRBench and both modalities of NYC-Event-VPR. Key results:

Method	NYC-Event R@1	NYC-RGB R@1	EPRBench R@1
CricaVPR	30.3	88.3	92.6
MixVPR	45.7	88.3	90.2
R2Former	61.2	86.3	80.0
Pair-VPR	64.0	89.4	91.9
FOL	55.5	85.1	93.8
SG-VPR	57.5	86.5	94.3

On EPRBench, SG-VPR achieves state-of-the-art performance with 94.3% R@1, exceeding SuperVLAD (93.3%) and CricaVPR (92.6%). On the more challenging NYC-Event-VPR event modality, SG-VPR ranks second behind two-stage re-rankers (Pair-VPR at 64.0%), but outperforms single-stage specialized methods by large margins — +27.2 points over CricaVPR and +11.8 over MixVPR — indicating that semantic guidance substantially mitigates event sparsity and noise where purely geometric approaches fail. On the RGB split it remains competitive (86.5% R@1), suggesting generalization across modalities.

Ablations confirm the design choices: full global+local fusion improves R@1 from 92.6 (vision-only baseline) to 94.3; the selection ratio peaks at $\Delta T$5 (93.1% at 0.20 due to lost structural cues, 94.2% at 0.30 due to reintroduced noise); and the contrastive weight peaks at $\Delta T$6, with higher values degrading performance because the auxiliary loss overwhelms retrieval supervision. Qualitative results show correct retrievals under viewpoint shifts where baselines fail, and the reasoning branch produces step-by-step landmark-based justifications for matches and rejections.

Limitations and open questions

The paper concedes that its reliance on a fine-tuned scene expert model for text generation is a core bottleneck: the expert is optimized for the authors' data distribution and does not directly transfer to open-world RGB scenarios. Additionally, the CoT annotations derive from an LLM pipeline (generation, translation, refinement) rather than fully independent human authorship, so annotation quality depends on the fidelity of Doubao Pro-1.6, NLLB-200, DeepSeek, and the human verification step. Whether the semantic-guided gains persist when descriptions are noisier or unavailable at inference time remains unexamined, as does scaling the scene expert to open-vocabulary conditions.

Conclusion

EPRBench provides the largest and most diverse event-based VPR benchmark to date, distinguished by high resolution, broad condition coverage, and — uniquely — semantic annotations suitable for vision-language VPR. The accompanying SG-VPR baseline demonstrates that LLM-derived semantics can be integrated into event-stream retrieval through text-guided token selection and multi-scale aggregation, yielding state-of-the-art accuracy on EPRBench and substantial gains over comparable single-stage methods on NYC-Event-VPR, while offering interpretable CoT justifications. The main open question left by the work is how to build a general-purpose scene expert whose semantic guidance transfers beyond the distribution on which it was fine-tuned.