Radar and Event Camera Fusion for Agile Robot Ego-Motion Estimation

Abstract: Achieving reliable ego motion estimation for agile robots, e.g., aerobatic aircraft, remains challenging because most robot sensors fail to respond timely and clearly to highly dynamic robot motions, often resulting in measurement blurring, distortion, and delays. In this paper, we propose an IMU-free and feature-association-free framework to achieve aggressive ego-motion velocity estimation of a robot platform in highly dynamic scenarios by combining two types of exteroceptive sensors, an event camera and a millimeter wave radar, First, we used instantaneous raw events and Doppler measurements to derive rotational and translational velocities directly. Without a sophisticated association process between measurement frames, the proposed method is more robust in texture-less and structureless environments and is more computationally efficient for edge computing devices. Then, in the back-end, we propose a continuous-time state-space model to fuse the hybrid time-based and event-based measurements to estimate the ego-motion velocity in a fixed-lagged smoother fashion. In the end, we validate our velometer framework extensively in self-collected experiment datasets. The results indicate that our IMU-free and association-free ego motion estimation framework can achieve reliable and efficient velocity output in challenging environments. The source code, illustrative video and dataset are available at https://github.com/ZzhYgwh/TwistEstimator.

• The paper presents an IMU-free, association-free algorithm that fuses radar Doppler and event camera data for direct 6-DoF velocity estimation.
• It employs a lightweight front-end with statistical filtering and a continuous-time B-spline back-end for robust sensor fusion and smoothing.
• Experimental results on UAV platforms demonstrate competitive performance with state-of-the-art methods in dynamic, low-texture, and high-speed scenarios.

Fusion of Radar and Event Camera for IMU-Free, Association-Free Ego-Motion Estimation in Agile Robotics

This paper presents a novel framework for ego-motion velocity estimation in agile robotic platforms by fusing data from a 4D millimeter-wave radar and an event camera, explicitly avoiding the use of IMU data and feature association. The approach is motivated by the limitations of conventional sensors—such as frame-based cameras, LiDARs, and IMUs—when subjected to highly dynamic motions, where issues like motion blur, distortion, and latency degrade performance. The proposed method leverages the complementary characteristics of radar and event cameras to directly estimate 6-DoF velocities, providing robust, low-latency feedback suitable for aggressive maneuvers.

Methodological Overview

The framework is composed of two main components: a lightweight front-end for direct velocity estimation and a continuous-time back-end for sensor fusion and smoothing.

Front-End: Direct Velocity Estimation

• Linear Velocity from Radar Doppler: The radar provides instantaneous radial velocities via the Doppler effect. By aggregating Doppler measurements from multiple radar points and solving a least-squares problem, the system estimates the 3D linear velocity in the radar frame. Outlier rejection is handled via statistical filtering, and uncertainty is quantified based on the spatial distribution and noise characteristics of the radar points.
• Angular Velocity from Event Camera: The event camera supplies high-frequency, asynchronous event streams. The method reconstructs local optical flow by fitting spatio-temporal planes to the event time surface, extracting normal flow, and then solving for angular velocity using a continuous-time epipolar constraint. This process does not require explicit feature tracking or frame-to-frame association.

Back-End: Continuous-Time Fusion

• B-Spline Trajectory Representation: Both translational and rotational velocities are parameterized as cumulative B-splines in the body frame, enabling smooth, continuous-time trajectory estimation.
• Sliding-Window Optimization: The estimator minimizes a cost function composed of radar velocity residuals, event camera angular velocity residuals, and prior constraints over a fixed-lag window. This formulation accommodates the asynchronous, motion-adaptive nature of the sensor measurements and avoids the need for measurement alignment or state inflation.
• Measurement Models: The radar and event camera measurements are modeled as noisy observations, and their covariances are incorporated into the optimization. Prior information ensures consistency across sliding windows and handles slow-varying biases and time offsets.

Experimental Evaluation

The framework is validated on a custom UAV platform equipped with a DAVIS346 event camera, an ARS548 4D MMWR, and a DJI M300 RTK for ground truth. Experiments span three challenging environments: building interiors, urban roads, and semi-open playgrounds, each presenting unique challenges such as low texture, dynamic lighting, and high-speed motion.

Key findings from the evaluation include:

• Linear Velocity Estimation: The proposed method (Twist-Estimator) achieves top or second-best performance in most sequences, often outperforming IMU-based and radar-inertial fusion baselines, especially under high dynamics or radar vibration.
• Angular Velocity Estimation: The event camera-based approach is competitive with IMU-based methods, particularly in semi-open and low-texture environments where feature-based methods degrade.
• Pose Estimation: Despite being IMU-free, the method achieves trajectory reconstruction accuracy comparable to IMU-based systems over short time horizons (10–30 seconds), with absolute and relative errors remaining low in most scenarios.
• Computational Efficiency: The association-free, IMU-free design reduces computational load, making the approach suitable for edge computing platforms.

A summary of the comparative results is provided in the following table (AVE/RVE: Absolute/Relative Velocity Error, APE/RPE: Absolute/Relative Pose Error):

Method	IMU	Radar	Event	Assoc-Free	Linear Vel.	Angular Vel.	Pose Est.	Best/2nd Best (Seq.)
Pure-IMU	✓				–	–	–	–
River	✓	✓			✓	✓	✓	Several
CMax-SLAM			✓		–	✓	–	Building
Twist-Estimator (Ours)		✓	✓	✓	✓	✓	✓	Most
RIO	✓	✓			✓	✓	✓	Some

Notable Claims and Results

• The framework is IMU-free and association-free, yet achieves velocity and pose estimation performance on par with or superior to state-of-the-art IMU-based and feature-based methods in challenging, dynamic environments.
• The method is robust to low-texture, structureless, and high-dynamic-range scenes, where traditional visual odometry and SLAM approaches often fail.
• The continuous-time fusion approach effectively handles asynchronous, high-frequency measurements without requiring explicit alignment or pre-integration.

Implications and Future Directions

The presented approach demonstrates that fusing radar Doppler and event camera data can provide reliable, low-latency ego-motion estimation for agile robots without reliance on inertial sensors or computationally expensive feature association. This has significant implications for the deployment of autonomous systems in environments where IMU data is unreliable or unavailable, or where computational resources are constrained.

Potential future developments include:

• Integration with Global Localization: While the method excels at local velocity and short-term trajectory estimation, integration with global localization modules (e.g., GNSS, map-based SLAM) could address long-term drift.
• Extension to Multi-Robot Systems: The lightweight, robust nature of the approach makes it suitable for swarms or fleets of agile robots operating in GPS-denied or visually degraded environments.
• Hardware Acceleration: Given the computational efficiency, further optimization and deployment on embedded platforms or FPGAs could enable real-time operation on resource-limited hardware.
• Learning-Based Enhancements: Incorporating learning-based modules for outlier rejection, uncertainty estimation, or adaptive sensor fusion could further improve robustness and generalization.

The source code, datasets, and illustrative videos are made available by the authors, facilitating reproducibility and further research in this domain.

Reference:

"Radar and Event Camera Fusion for Agile Robot Ego-Motion Estimation" (2506.18443)