Event-Based Robotic Policies

Updated 4 July 2026

Event-Based Robotic Policies (ERP) are control strategies that use asynchronous event data and triggered decision updates instead of fixed, dense control ticks.
They fuse modalities like event-camera and RGB inputs to capture high-temporal-resolution brightness changes for robust perception in adverse conditions.
These policies enhance performance in low-light, fast-motion, and multi-agent scenarios by preserving temporal event structures in control and simulation.

Event-Based Robotic Policies (ERP) denote learned robot control policies that are organized around events rather than only dense frame sequences or fixed control ticks. In the recent robotics literature, the term covers at least two closely related constructions: policies that consume event-camera observations, either alone or fused with RGB, to output control commands, and policies whose decision updates are themselves event-triggered, for example by macro-action termination or value-based detection of inadequate progress (Ramesh et al., 15 Mar 2026, Vinod et al., 25 Aug 2025, Menda et al., 2017, Du et al., 2024). The unifying motivation is that event structure carries information that conventional synchronous pipelines often discard: event cameras provide asynchronous brightness-change measurements with microsecond-level temporal resolution and high dynamic range, while event-driven decision formulations preserve the timing and ordering of action-relevant occurrences in control and simulation (Ramesh et al., 15 Mar 2026, Vemprala et al., 2021).

1. Conceptual scope and terminology

In the event-vision branch of ERP, the policy is built around dynamic vision sensors that emit events when per-pixel brightness changes exceed a threshold. This sensing model is attractive for robotic navigation because conventional RGB cameras degrade under fast motion and low light through poor contrast, motion blur, and dynamic-range limitations, whereas event cameras remain informative in dim scenes and during rapid target or ego motion (Ramesh et al., 15 Mar 2026, Vemprala et al., 2021). This branch includes end-to-end imitation policies for indoor mobile robots, representation-learning pipelines for visuomotor reinforcement learning, and multimodal RL systems that use event-based perception for pedestrian following and obstacle avoidance (Ramesh et al., 15 Mar 2026, Vemprala et al., 2021, Bugueno-Cordova et al., 12 Jun 2025).

A second branch uses “event-based” in the control-theoretic sense of decision instants tied to meaningful occurrences rather than uniform time discretization. In this formulation, a robot or multi-robot system re-observes and re-decides when a macro-action terminates or when a monitored progress signal indicates that the current strategy has become suboptimal (Menda et al., 2017, Du et al., 2024). This is especially relevant to asynchronous robotics, where action durations are stochastic and different agents or subsystems naturally desynchronize.

The literature therefore does not restrict ERP to a single architectural template. Across the cited works, ERP may mean end-to-end behavioral cloning from synchronized RGB-event observations, RL on latent representations learned from raw event streams, hybrid imitation-learning-plus-RL controllers that use event-based detection as a perception front end, or event-triggered wrappers that alter deployment behavior when failure-relevant events are detected (Ramesh et al., 15 Mar 2026, Vemprala et al., 2021, Bugueno-Cordova et al., 12 Jun 2025, Du et al., 2024).

A terminological caveat is necessary. On arXiv, the acronym “ERP” is not unique to robotics. “Reconstructing ERP Signals Using Generative Adversarial Networks for Mobile Brain-Machine Interface” uses ERP to mean event-related potentials in EEG, specifically N200 and P300, rather than Event-Based Robotic Policies (Lee et al., 2020). In robotics usage, the relevant term is the one made explicit in SEBVS: Event-Based Robotic Policies (Vinod et al., 25 Aug 2025).

2. Sensor models and observation representations

The fundamental event primitive is commonly written as a timestamped polarity tuple. In representation learning for event-based visuomotor control, an event is defined as

$e = (t, x, y, p),$

with timestamp $t$ , pixel location $(x,y)$ , and polarity $p$ ; over a time window $\tau$ , the event stream is

$E_\tau = \{e_i \mid t < i < t+\tau\}.$

This formalization highlights why event data is difficult for conventional pipelines: it is asynchronous, variable-length, sparse, and order-sensitive (Vemprala et al., 2021).

One common strategy is event-to-frame aggregation. In eNavi, RGB frame timestamps are used as the anchor. For each RGB frame $I_i$ at time $t_i$ , the event stream is sliced between consecutive frames,

$\mathcal{E}_i = \{ e_k \in \mathcal{E} \mid t_{i-1} < t_k \leq t_i \},$

and accumulated into a dense 2-channel event frame $\mathbf{E}_i \in \mathbb{R}^{H \times W \times 2}$ by separating positive and negative polarity events:

$t$ 0

$t$ 1

Actions are aligned over the same interval by averaging commands,

$t$ 2

where $t$ 3 (Ramesh et al., 15 Mar 2026). This produces synchronized tuples $t$ 4 suitable for supervised imitation learning.

A different accumulation choice appears in human-robot navigation with RL. There, the asynchronous stream can be grouped either by fixed-time windows,

$t$ 5

or by fixed-event windows,

$t$ 6

and rendered as a Surface of Active Events (SAE) image for event-based pedestrian detection. The detector is trained on the PEDRo event-based dataset using a 10 ms accumulation window (Bugueno-Cordova et al., 12 Jun 2025).

SEBVS adopts another frame-like event representation in simulation. Synthetic events generated by v2e are discretized into ON and OFF channels and concatenated with RGB into a 5-channel tensor. The accumulated event frame is defined as

$t$ 7

with $t$ 8 and separate channels $t$ 9 and $(x,y)$ 0 (Vinod et al., 25 Aug 2025).

By contrast, the eVAE work aims to preserve more of the native asynchronous structure. Rather than reducing observations to event images before learning, it processes raw event sets with an Event Context Network (ECN), adds sinusoidal temporal embeddings, and only reconstructs an event image at the decoder stage (Vemprala et al., 2021). This suggests a central representation tradeoff in ERP: synchronizing and densifying events simplifies downstream supervision and deployment, while set-based or sequence-native encoders preserve the timing structure that originally motivated event sensing.

3. Learning formulations and policy architectures

Work	Training paradigm	ERP mechanism
eNavi (Ramesh et al., 15 Mar 2026)	Behavioral cloning	Late-fusion RGB-Event policy for continuous differential-drive commands
Event-based visuomotor policies (Vemprala et al., 2021)	Unsupervised eVAE + PPO	RL on latent representations learned from raw event streams
Human-robot navigation (Bugueno-Cordova et al., 12 Jun 2025)	Behavior cloning initialization + DDPG	Event-based perception fused with additional sensors for continuous velocity increments
SEBVS (Vinod et al., 25 Aug 2025)	Behavioral cloning	Transformer-based RGB, Event, and RGB+Event policies
Event-driven multi-agent RL (Menda et al., 2017)	PS-TRPO with modified GAE	Event-triggered macro-action policies in Dec-POMDPs
Bellman-Guided Retrials (Du et al., 2024)	Imitation policy + offline value learning	Event-triggered recovery and retry during deployment

The eNavi policy, ENP-Fusion, is a late-fusion RGB-Event navigation policy trained by behavioral cloning to predict continuous differential-drive commands $(x,y)$ 1 from synchronized multimodal observations (Ramesh et al., 15 Mar 2026). Its architecture uses two parallel MobileNetV3-Small encoders, one for RGB and one for event frames. The RGB encoder is pretrained and frozen, while the event encoder is trainable. Each encoder outputs a compact feature token; the tokens are concatenated and processed by a Transformer encoder block for attention-based fusion, and an MLP head predicts

$(x,y)$ 2

Training minimizes an L1 / MAE behavioral cloning objective,

$(x,y)$ 3

with MAE chosen for robustness to outliers and noisy command labels (Ramesh et al., 15 Mar 2026).

The representation-learning line separates perception from control more explicitly. The event variational autoencoder uses a PointNet-style, permutation-invariant ECN with max pooling and sinusoidal temporal embeddings,

$(x,y)$ 4

to encode raw event streams into a latent variable $(x,y)$ 5 (Vemprala et al., 2021). A PPO policy $(x,y)$ 6 is then trained on that latent representation rather than on raw events or event images. The paper’s stated rationale is that end-to-end reward-driven perception from sparse asynchronous observations is sample-inefficient and unstable.

The human-robot navigation controller of (Bugueno-Cordova et al., 12 Jun 2025) is multimodal and actor-critic based. It combines an event camera for fast pedestrian detection, a RGB-D depth sensor for pedestrian distance estimation, a 2D LIDAR for obstacle avoidance, and a DDPG controller that outputs continuous increments

$(x,y)$ 7

every $(x,y)$ 8. The state is a 6-dimensional vector

$(x,y)$ 9

and the actor is initialized by behavior cloning from a simple PD-based expert before RL fine-tuning (Bugueno-Cordova et al., 12 Jun 2025).

SEBVS uses a single shared transformer-style backbone across navigation and manipulation tasks (Vinod et al., 25 Aug 2025). Early fusion is performed by concatenating RGB and ON/OFF event channels into a 5-channel input. The image is split into non-overlapping $p$ 0 patches, each patch is projected to a 64-dimensional embedding, a learnable class token is prepended, and sinusoidal positional encodings are added. The transformer encoder has 4 attention heads, a 256-unit feed-forward block, residual connections, layer normalization, and dropout $p$ 1. Task-specific heads then produce either a navigation command

$p$ 2

or a 6-DoF pre-grasp pose

$p$ 3

The event-driven decision-process line modifies the policy-update machinery itself. In asynchronous multi-agent learning with macro-actions, agents execute temporally extended actions of stochastic duration and re-decide only when relevant events occur (Menda et al., 2017). Standard GAE is replaced by a time-aware estimator over macro-actions:

$p$ 4

$p$ 5

The key change is discounting by elapsed time between decision events rather than by discrete step count.

Bellman-Guided Retrials provides another event-triggered control layer. A base diffusion policy trained on demonstrations is wrapped by a value function that estimates task completion; during deployment, the robot compares expected progress against achieved progress and triggers recovery and retry when progress is too slow (Du et al., 2024). Under the sparse reward setting described in the paper, the comparison becomes

$p$ 6

This is not an event-camera policy, but it is an ERP in the decision-architectural sense because value-based trigger events alter policy execution.

4. Platforms, datasets, and task domains

The most detailed real-world ERP dataset in the provided literature is eNavi. It is collected on a TurtleBot 2 mobile robot with a Kobuki base and an NVIDIA Jetson Orin Nano running ROS 2 (Ramesh et al., 15 Mar 2026). The sensing suite combines a Prophesee Metavision EVK4 event camera with an IMX636 sensor at $p$ 7 and a Teledyne Blackfly S RGB camera with an onsemi Python1300 sensor at $p$ 8, mounted coaxially via a beam splitter and equipped with Kowa 25 mm f/1.4 lenses. Hardware synchronization is performed by an STM32F407 microcontroller that sends a 30 Hz, 50% duty-cycle trigger to both cameras; spatial alignment uses a blinking checkerboard and a homography that maps RGB into the event camera frame. The dataset contains about two hours of teleoperated driving data across more than 175 episodes, collected in three indoor environments with two human subjects under normal-light and low-light / mixed-light conditions (Ramesh et al., 15 Mar 2026).

The same paper uses this platform for indoor person-following and navigation in cluttered real environments. Data are split into a single-map in-distribution condition, with the remaining two environments held out for generalization experiments (Ramesh et al., 15 Mar 2026). Control labels are reconstructed from wheel odometry rather than taken directly from joystick commands so that the labels reflect actual executed motion and avoid dead-zone noise, latency, and operator jitter.

Event-based visuomotor RL has also been studied in simulation for high-speed UAV obstacle avoidance. The eVAE work uses AirSim with a forward-facing event camera of resolution $p$ 9, a drone modeled as a point robot, assumed speed $\tau$ 0, control frequencies from 45 Hz to 400 Hz, and environments containing poles, cones, and gates (Vemprala et al., 2021). The policy observes a stack of the three most recent latent vectors.

The human-robot navigation controller of (Bugueno-Cordova et al., 12 Jun 2025) targets a Pioneer 3-AT mobile robot equipped with a DAVIS346 event camera, RealSense RGB-D camera, and 2D Hokuyo LIDAR. Training and evaluation are conducted in Gazebo and ROS, implemented in Python with PyTorch, on a $\tau$ 1 map containing corridors, obstacles, a moving pedestrian, and a figure-eight trajectory over 100 seconds, with pedestrian speed $\tau$ 2.

SEBVS extends ERP to both navigation and manipulation in synthetic event simulation (Vinod et al., 25 Aug 2025). Its stack uses Gazebo, ROS2, and v2e. The navigation task, ERPNav, is a mobile robot object-following problem in which a four-wheel differential-drive robot follows a moving cube using expert demonstrations from a YOLOv8 + PID pipeline. The manipulation task, ERPArm, uses a UR5 arm with a Robotiq 2F-85 gripper, a wrist-mounted RGB camera, and a co-located v2e emulator to predict a 6-DoF pre-grasp pose for a cube on a conveyor belt in static and moving settings. The training data comprise 70 episodes, about 65 minutes, and 32,033 action labels for ERPNav, and 35 episodes, about 70 minutes, and 18,417 motion waypoints for ERPArm (Vinod et al., 25 Aug 2025).

Beyond event-camera sensing, event-driven ERP has been validated in cooperative bus holding and wildfire fighting with unmanned aircraft (Menda et al., 2017), and event-triggered retry logic has been studied on CaddyGrasp / CaddyLift, SimObjectLift, RealObjectLift, and DoorOpening (Du et al., 2024). These works broaden ERP from sensor modality to decision timing.

5. Empirical behavior and comparative findings

A recurring empirical result is that event information is most useful when appearance degrades, motion is rapid, or task dynamics become more complex. In eNavi, the authors study 12 policy variations formed from 3 architectures, 2 path settings, and 2 lighting settings, all trained with AdamW at learning rate $\tau$ 3, weight decay $\tau$ 4, batch size 64, an 80/10/10 train/val/test split, early stopping patience 8, and a maximum of 50 epochs (Ramesh et al., 15 Mar 2026). In Mixed-Light validation, ENP-Fusion attains lower MAE than RGB-only in both trajectory regimes: Single-Path Mixed-Light gives RGB $\tau$ 5, Event $\tau$ 6, Fusion $\tau$ 7, while Multi-Path Mixed-Light gives RGB $\tau$ 8, Event $\tau$ 9, Fusion $E_\tau = \{e_i \mid t < i < t+\tau\}.$ 0 (Ramesh et al., 15 Mar 2026). For zero-shot low-light testing, the Multi-Path case makes the same pattern clearer: RGB moves from $E_\tau = \{e_i \mid t < i < t+\tau\}.$ 1 to $E_\tau = \{e_i \mid t < i < t+\tau\}.$ 2 in low light, Event from $E_\tau = \{e_i \mid t < i < t+\tau\}.$ 3 to $E_\tau = \{e_i \mid t < i < t+\tau\}.$ 4, and Fusion from $E_\tau = \{e_i \mid t < i < t+\tau\}.$ 5 to $E_\tau = \{e_i \mid t < i < t+\tau\}.$ 6 (Ramesh et al., 15 Mar 2026). The paper also notes that RGB-only models are more likely to saturate during training, converge more slowly, and generalize less reliably to unseen low-light conditions, while event-only and fusion models often stop earlier, around 20–35 epochs, whereas RGB-only models often train all the way to 50 epochs.

The representation-learning results for event-based visuomotor control emphasize sample complexity, control-rate adaptation, and robustness rather than illumination (Vemprala et al., 2021). Policies using eVAE representations train faster than event-image CNN policies, improve reward more strongly over the first 500,000 timesteps, and operate successfully up to 400 Hz, where image-based methods degrade because event images become overly sparse. They are also more robust to unseen obstacle shapes, changed textures, different event thresholds, induced sparsity, and random background activity noise; BRP-full can be more sensitive than BRP-xy under background activity noise because of spurious polarity information (Vemprala et al., 2021).

The human-robot navigation system of (Bugueno-Cordova et al., 12 Jun 2025) shows that an event-based perception front end can support real-time RL control in a social-navigation setting. Its YOLOv5s detector trained on SAE images reports accuracy and recall close to 0.90, with $E_\tau = \{e_i \mid t < i < t+\tau\}.$ 7 and $E_\tau = \{e_i \mid t < i < t+\tau\}.$ 8 reaching above 0.7 for the stricter metric (Bugueno-Cordova et al., 12 Jun 2025). In controller comparisons, PD exhibits large deviations, especially in curves; imitation learning is smoother and more stable than PD; and the DDPG controller gives the best overall trajectory tracking, smoother adaptation, and stronger robustness, with better centering around the target and more stable pedestrian distance near 2 m (Bugueno-Cordova et al., 12 Jun 2025).

SEBVS provides a direct modality comparison across navigation and manipulation (Vinod et al., 25 Aug 2025). In ERPNav, RGB+Event achieves a centroid error of $E_\tau = \{e_i \mid t < i < t+\tau\}.$ 9, versus $I_i$ 0 for RGB and $I_i$ 1 for Event, and a success rate of $I_i$ 2, versus $I_i$ 3 and $I_i$ 4 respectively. In ERPArm single-object evaluation, RGB+Event achieves $I_i$ 5 error, $I_i$ 6 accuracy, and $I_i$ 7 success rate, compared with $I_i$ 8, $I_i$ 9, and $t_i$ 0 for RGB, while Event is the lowest-latency condition at $t_i$ 1 but has lower accuracy and success (Vinod et al., 25 Aug 2025). In multi-object ERPArm, RGB+Event again leads, with $t_i$ 2 error, $t_i$ 3 accuracy, and $t_i$ 4 success rate (Vinod et al., 25 Aug 2025). These results support the paper’s stated conclusion that RGB+Event policies are typically strongest overall, while event-only policies can be faster but less accurate.

Event-driven decision-process work yields a different kind of empirical lesson. In asynchronous multi-agent domains, fixed time-step simulation can obscure the ordering of closely separated events and alter the learned policy (Menda et al., 2017). The paper reports that maintaining a low race-condition probability by shrinking the time step scales poorly with the number of agents, following

$t_i$ 5

Event-driven simulation therefore preserves the event order that decentralized asynchronous policies depend on. Bellman-Guided Retrials shows analogous benefits at deployment time: average absolute success rates increase by more than 20% in simulation and 50% in real-world experiments, and DoorOpening exceeds twice the baseline success rate (Du et al., 2024). Together, these findings suggest that ERP gains can arise not only from event sensing but also from preserving event structure in execution and supervision.

6. Limitations, open problems, and disputed intuitions

A consistent limitation in current event-camera ERP is that many pipelines still collapse asynchronous streams into synchronous tensors. eNavi identifies its time-synchronized event-to-frame aggregation as a computational burden for real-time deployment on edge hardware and explicitly suggests that native asynchronous processing could be more efficient and responsive (Ramesh et al., 15 Mar 2026). The same paper also notes the narrowness of its present scope: the dataset is relatively small, focused on indoor person-following, and broader generalization to other navigation tasks remains open. Proposed extensions include action chunking, diffusion-based policy heads, and DAgger-style online learning (Ramesh et al., 15 Mar 2026).

The simulated social-navigation controller of (Bugueno-Cordova et al., 12 Jun 2025) is also not fully event-based end-to-end. It depends on depth sensing, LIDAR, and a RealSense-based pedestrian-distance estimate. The paper identifies fully event-based operation, real-world testing, more complex pedestrian motion patterns, other RL algorithms such as TD3, alternative reward designs, alternative event representations, and monocular event-based depth and pose estimation as future directions (Bugueno-Cordova et al., 12 Jun 2025).

SEBVS complicates a common simplification in the literature: event-only is not uniformly superior to RGB (Vinod et al., 25 Aug 2025). In the reported experiments, event-only often provides the lowest latency but not the best accuracy or success rate, whereas RGB+Event fusion is typically strongest. eNavi reaches a closely related conclusion: RGB can still suffice on simple, well-lit trajectories, but fusion offers the best tradeoff between accuracy and robustness in realistic low-light indoor navigation (Ramesh et al., 15 Mar 2026). A plausible implication is that ERP should not be treated as a wholesale replacement of frame-based context, but as a design space in which asynchronous motion-sensitive cues and dense appearance cues are complementary.

Another disputed intuition concerns representation. The eVAE work argues explicitly that treating events as accumulated images is inferior for control because it discards temporal precision, becomes sparse at high control rates, and generalizes less well to variable event density (Vemprala et al., 2021). The result is not that event frames are unusable—eNavi, SAE-based navigation, and SEBVS all use frame-like event encodings successfully—but that frame accumulation is an engineering compromise rather than a full exploitation of the modality (Ramesh et al., 15 Mar 2026, Bugueno-Cordova et al., 12 Jun 2025, Vinod et al., 25 Aug 2025).

On the decision-process side, shrinking the control step is not a complete substitute for event-driven execution. The multi-agent work shows that fixed-step simulators can change event ordering and that arbitrarily shrinking the step size scales poorly (Menda et al., 2017). Likewise, event-triggered retry systems do not eliminate the need for a strong base policy: Bellman-Guided Retrials cannot invent strategies absent from the base policy, requires a reasonable recovery policy, and may be less effective when proprioceptive distance is a poor proxy for behavioral diversity in highly dynamic scenes (Du et al., 2024).

Taken together, the cited literature positions ERP as a heterogeneous but increasingly coherent area at the intersection of asynchronous sensing, temporally structured policy optimization, and closed-loop robotic control. The strongest current evidence favors multimodal fusion and event-aware decision structure rather than a single canonical architecture, and the central unresolved question is how much of the original event timing can be preserved without sacrificing tractable training and deployment (Ramesh et al., 15 Mar 2026, Vinod et al., 25 Aug 2025, Menda et al., 2017).