HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification (1803.07913v1)

Abstract: Event-based cameras have recently drawn the attention of the Computer Vision community thanks to their advantages in terms of high temporal resolution, low power consumption and high dynamic range, compared to traditional frame-based cameras. These properties make event-based cameras an ideal choice for autonomous vehicles, robot navigation or UAV vision, among others. However, the accuracy of event-based object classification algorithms, which is of crucial importance for any reliable system working in real-world conditions, is still far behind their frame-based counterparts. Two main reasons for this performance gap are: 1. The lack of effective low-level representations and architectures for event-based object classification and 2. The absence of large real-world event-based datasets. In this paper we address both problems. First, we introduce a novel event-based feature representation together with a new machine learning architecture. Compared to previous approaches, we use local memory units to efficiently leverage past temporal information and build a robust event-based representation. Second, we release the first large real-world event-based dataset for object classification. We compare our method to the state-of-the-art with extensive experiments, showing better classification performance and real-time computation.

• The paper introduces HATS, a novel method using histograms of averaged time surfaces to achieve robust event-based object classification.
• It leverages local memory units to effectively capture historical events, thereby reducing noise and enhancing temporal sensitivity.
• Experiments on the extensive N-CARS dataset show that HATS outperforms existing approaches with up to twenty times faster computation and greater accuracy.

Essay on "HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification"

The paper "HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification" addresses significant gaps in the utilization of event-based cameras for object classification. Event-based cameras, which trigger events asynchronously in response to changes in visual scenes, offer advantages such as high temporal resolution, low power usage, and an impressive dynamic range. Yet, the development of robust algorithms for event-based object classification has been hampered by the absence of efficacious low-level feature representations and the lack of substantial real-world datasets.

This research innovatively proposes a novel event-based feature representation through Histograms of Averaged Time Surfaces (HATS) and introduces a new machine learning architecture. Unlike prior methods that fail to optimally leverage past temporal information, the proposed methodology uses local memory units to store and access historical events effectively. This approach provides a robust and efficient event-based representation, overcoming limitations of noise and temporal sensitivity that plagued earlier techniques such as time surfaces.

The introduction of HATS is a notable advancement, where the local memory time surfaces are spatially averaged to form histograms that offer an accurate, compact representation of event streams. This is crucial, given the absence of a direct applicability of standard Computer Vision methods to event-based data due to its asynchronous and sparse nature.

A pivotal contribution of this paper is the creation of the N-CARS dataset, a large-scale, real-world dataset specifically tailored for event-based vision. This dataset comprises over 24,000 samples captured from a vehicle in diverse urban and motorway environments, annotated with a semi-automatic protocol to ensure accuracy. In the context of classification, the HATS method outperforms existing methods such as HOTS and SNNs, exhibiting superior classification performance and significantly reduced computation times.

Numerical results demonstrate the effectiveness of the proposed approach, with the HATS methodology surpassing competing algorithms across various benchmarks, including the novel N-CARS dataset and other artificial datasets generated from frame-based counterparts. The HATS method is up to twenty times faster than some of the baselines and boasts greater accuracy, underscoring its potential for real-time applications.

The implications of this research are broad, providing a framework for more efficient and accurate event-based object recognition systems with applications in autonomous navigation, robot vision, and other domains requiring high-speed visual processing. This could stimulate further investigations into memory-efficient architectures and the broader application of neuromorphic sensors in machine learning.

In conclusion, the paper lays a robust foundation for future advancements in event-based camera technology and object classification. It highlights the necessity of scalable datasets and efficient feature representations in advancing the field, potentially opening avenues for new machine learning algorithms that align more closely with the high temporal resolution capabilities of neuromorphic cameras. Future developments may involve exploring more advanced network architectures that evolve beyond spiking networks, incorporating learned feature representations for archival event data, and further enhancing real-world dataset curation methodologies.