Microsaccade-Enhanced Event Cameras

• Artificial Microsaccade-enhanced Event Cameras (AMI-EV) are sensor systems that integrate a rotating wedge prism to mimic biological microsaccades, ensuring robust, motion-invariant edge detection.
• They employ precise mechanical actuation and per-event warping algorithms to compensate for induced optical flow, enhancing high-dynamic range and accurate edge information in robotics perception.
• Benchmark comparisons show AMI-EV outperforms standard event cameras with superior metrics in edge F1 scores, texture entropy, and reconstruction quality under both static and dynamic conditions.

Artificial Microsaccade-enhanced Event Cameras (AMI-EV) are sensor systems that augment conventional event cameras with an optical mechanism inspired by biological microsaccades—small, involuntary eye movements in primates that counteract perceptual fading. By integrating a precisely controlled rotating wedge prism in front of the event sensor and employing algorithmic compensation, AMI-EV systems achieve robust, motion-invariant edge detection and information-rich event streams, even for scene features aligned parallel to camera motion. These capabilities address a major intrinsic limitation of standard event-based cameras and enable improved perception for a wide range of high-dynamic robotics applications (He et al., 2024).

1. System Architecture and Hardware Design

The AMI-EV integrates a rotating wedge prism optical module immediately in front of a standard event camera lens to actively induce microsaccade-like optical flow. The core components are:

• Wedge Prism: Custom-ground BK7 glass, aperture 12 mm, thickness ~3 mm, with adjustable apex angle (α ∈ {0.5°, 1.0°, 2.0°}). The default configuration uses α = 1.0°. The prism refracts incoming scene rays, and when rotated about the optical axis ($z_c$), produces a 2D circular sweep of optical flow on the focal plane, ensuring all orientation edges traverse the sensor array.
• Actuator and Encoder: A DJI M2006 brushless DC motor combined with a 10:1 planetary gearbox controls rotation. The system achieves up to 1,500 rpm (25 Hz) but typically operates at 720 rpm (12 Hz) for optimal balance between event volume and warping accuracy. An onboard incremental encoder, marked with optical references, provides absolute prism angle measurements $\theta(t)$ with ±0.05° precision.
• Mechanical Layout and Control Electronics: The prism is mounted in a precision-machined aluminum housing ensuring centration within ±0.1 mm and tilt <±0.1°. Motor, encoder, and timestamp synchronization are managed by an MCU (DJI Robomaster, STM32H7), which synchronizes all event data to a common 48 MHz time base. The complete optical module attaches via standard C-mount lens adapters for compatibility. Total power consumption is ≈ 5 W for the motor+MCU and ≈ 0.5 W for the camera.
• Event Camera Sensor: Utilizes an iniVation DVXplorer, featuring 640 × 480 spatial resolution, 3 µs timestamp precision, and high dynamic range (HDR, >120 dB).

2. Geometrical and Computational Modeling

2.1 Optical Model

The AMI-EV system models light path refraction at both prism interfaces, governed by Snell’s law ($n≈1.55$). Key stages:

• Incoming ray $v_{in}$ forms an angle $\Phi_i$ with $z_c$; inside the prism, Snell’s law gives $\Phi_p = \arcsin(\sin\Phi_i / n)$. Rotational transformation $R(a_i, \Phi_i−\Phi_p)$ is applied.
• Upon exiting, a second Snell-based refraction is computed relative to the rotated prism local axis $z_w(\theta)$. The outgoing ray is $v_{out} = R(a_o, \Phi_q−\Phi_o)\cdot v_p$.
• The full deflection is described by $\theta(t)$0, and projection to image coordinates uses the intrinsic $\theta(t)$1:

$\theta(t)$2

As $\theta(t)$3 varies, $\theta(t)$4 traces a near-circular locus whose radius and trajectory depend on geometry.

2.2 Warping and Compensation Algorithm

The prism's induced optical flow, $\theta(t)$5, is a deterministic function of pixel $\theta(t)$6 and prism angle $\theta(t)$7. Algorithmic compensation leverages accurate timestamp-synchronized encoder data to apply a per-event coordinate warp, mapping each event $\theta(t)$8 back to a reference geometry.

Let $\theta(t)$9, the warping operator $n≈1.55$0 is: $n≈1.55$1 In practice, $n≈1.55$2 is approximated by a 2-D rotation about the image center: $n≈1.55$3 where $n≈1.55$4 is calibrated per pixel by fitting its observed circular path.

Pseudo-code:

$v_{in}$1

Synchronization is achieved by sharing both event and encoder data with microsecond-level jitter (<1 µs), ensuring pairing within a global time base.

3. Performance Metrics and Benchmark Comparisons

Quantitative evaluation of AMI-EV employs information-theoretic and task-level metrics, comparing against standard event (S-EV) and conventional frame-based cameras:

Metric	AMI-EV (Static)	S-EV (Static)	AMI-EV (Motion)	S-EV (Motion)
Variance of KDE density	0.196	0.425	0.196	0.425
ODS-F score (mean ± std, 10 s)	0.80 ± 0.02	0.32 ± 0.05	0.78 ± 0.03	0.55 ± 0.04
Entropy (bits)	7.2	5.1	—	—
NIQE (reconstructed image)	2.1	4.5	2.3	2.8

• Information Rate (IR): Events × log$n≈1.55$5(contrastLevels)/time.
• Edge Contrast: F1-score between Canny-detected and ground-truth edges.
• Entropy: Higher $n≈1.55$6 measures richer texture.
• Point-Cloud Uniformity: KDE-based variance $n≈1.55$7; lower is better.
• NIQE: No-reference image quality; lower indicates greater fidelity after reconstruction.

Across static and dynamic conditions, AMI-EV demonstrates higher information rates, F1 edge contrast, texture entropy, and point-cloud uniformity, while reconstructed images achieve lower NIQE scores, signifying significant improvement over S-EV.

4. Applications and Experimental Protocols

4.1 Robotics Perception Tasks

Events recorded with AMI-EV show marked improvements on robotic visual perception benchmarks:

• Corner Detection & Tracking: Median track lifetime of 0.91 s (AMI-EV) vs 0.42 s (S-EV); peak track count of 1,200 vs 650; lower latency at 1.8 ms vs 3.5 ms.
• Motion Segmentation: Optimization on event surfaces yields per-event segmentation accuracy of 92% for AMI-EV/S-EV, with frame-based methods achieving 65%.
• Human Detection & Pose Estimation: E2VID followed by OpenPifPaf delivers intersection-over-union (IoU) at 100 fps of 0.78 (AMI-EV) vs 0.54 (S-EV) vs 0.60 (frame); PDJ @0.2r of 86% vs 60% vs 44%.

4.2 Experimental Platforms

Tests are performed on a 6-DoF robotic arm platform, with controlled translational motions up to 0.5 m/s and rotation up to 60°/s. Datasets encompass structured checkerboards, unstructured office scenes, HDR corridors, and dynamic objects. AMI-EV is also simulated using the WorldGen engine for synthetic benchmarks, with notable improvements (e.g., 35% F1 gain on Neuromorphic-Caltech101 edges).

5. Calibration Procedures and Known Limitations

5.1 Calibration

A 2 s calcium event recording under static view conditions suffices for a coarse-to-fine search minimizing the “sharpness cost”: $n≈1.55$8 where $n≈1.55$9 is the image-wise event (IWE) intensity. The cost landscape is uniquely convex in $v_{in}$0 near the optima, supporting rapid convergence (<1 s on commodity CPUs).

5.2 Failure Modes and Environmental Constraints

• Counteracting Camera Motion: When physical camera movement nearly cancels the prism-induced image flow (~0.2% of operations), the event rate dips. This is recoverable by dithering the prism angle or briefly pausing rotation.
• Environmental: Performance in <10 lux scenes is affected by event noise; temperatures >60 °C may induce glass birefringence, necessitating re-calibration every 2 h.

6. Prospects for Extension and Optimization

Several directions are identified:

• Optical Actuation: Replacement of the rotating prism with MEMS optical phased arrays or LCD beam steerers is proposed, targeting control rates >5 kHz and total power <100 mW.
• Adaptive Control: The system could dynamically slow or halt the artificial microsaccade mechanism if camera motion alone is sufficient for edge stimulation, reducing average power consumption.
• Neural Compensation: End-to-end event-based neural networks may be trained for per-event warping, potentially eliminating residual geometric errors (currently 1.5–2 pixels).
• Integration with Event-Inertial SLAM: Incorporation of prism angle θ(t) as auxiliary SLAM measurements could enhance motion estimation robustness in low-texture environments.

By decoupling edge event generation from external motion, AMI-EV provides uniformly rich, motion-invariant sensory information for downstream vision tasks. The design is compatible with existing event-based pipelines and demonstrates proven efficacy across a spectrum of both low- and high-level perception challenges in robotics (He et al., 2024).