EVIS: Event-Camera Plugin for Isaac Sim

• EVIS is a physics-grounded event-camera plugin that generates high-rate, fully labeled event streams directly within NVIDIA Isaac Sim.
• It integrates event generation with simulator physics, coupling robot motion, scene dynamics, and ground truth for diverse camera setups.
• EVIS employs a faithful log-intensity contrast model with per-pixel asynchronous updates and motion-vector interpolation for efficient simulation.

Searching arXiv for related event camera simulator and downstream papers to support context. EVIS is a physics-grounded event-camera plugin for NVIDIA Isaac Sim that generates high-rate, fully labeled event streams directly inside a physics simulator, with the explicit aim of making event-camera data generation a first-class component of modern robotics simulation workflows (Shi et al., 9 Jul 2026). It addresses a practical bottleneck in event-based robotics: real event-camera datasets are difficult to collect, expensive to label, and typically tied to fixed robot platforms, sensor placements, and environments. By embedding event generation directly into Isaac Sim and Isaac Lab, EVIS couples event streams to the same physics-simulated world as robot motion, contacts, scene dynamics, and simulator-provided ground truth, while retaining a workflow in which an RGB camera configuration can be replaced by an event-camera configuration with minimal changes (Shi et al., 9 Jul 2026).

1. Concept and system role

EVIS is designed around three requirements that its authors identify as essential for useful event-camera simulation: fidelity of the event model, physics consistency with the underlying simulator, and rendering throughput high enough to support training-scale workloads (Shi et al., 9 Jul 2026). The motivation follows directly from the sensing principle of event cameras. Unlike conventional cameras, event cameras do not emit full frames at fixed intervals; each pixel independently reports a change in brightness once that change exceeds a contrast threshold. This yields microsecond-scale temporal resolution, low latency, and high dynamic range, but also makes data collection and labeling unusually demanding because asynchronous event streams must be synchronized with robot state, object motion, geometry, and dense labels such as depth or optical flow.

The distinctive systems contribution of EVIS is its placement inside NVIDIA Isaac Sim and Isaac Lab rather than outside the robotics simulator as an offline converter or scene renderer. Earlier methods discussed in the EVIS paper include video-to-event converters such as v2e, Vid2E, PIX2NVS, DVS-Voltmeter, and V2CE, as well as simulators such as ESIM, Blender-based simulators, EvDNeRF, and Gazebo plugins; EVIS is positioned against these categories by emphasizing direct inheritance of simulator physics, arbitrary robot embodiments, arbitrary camera trajectories, and frame-perfect labels from Isaac workflows (Shi et al., 9 Jul 2026). In practical terms, this means that monocular, stereo, robot-mounted, or user-defined camera trajectories can all be used within the same simulator pipeline.

This simulator-native integration has methodological significance beyond convenience. A plausible implication is that EVIS is intended less as an isolated event-generation utility than as a reusable perception component for end-to-end robotics experimentation, especially in settings where robot embodiment, camera placement, and task dynamics change frequently.

2. Event generation model

EVIS implements what the paper describes as a faithful log-intensity contrast event model with per-pixel asynchronous reference updates (Shi et al., 9 Jul 2026). By default, the plugin uses the renderer’s HDR color buffer before gamma correction, which is described as linear in light intensity and therefore more physically appropriate for event-camera simulation. Luminance is computed from RGB using Rec. 709 weights,

$I = 0.2126\,R + 0.7152\,G + 0.0722\,B,$

and the log response is

$L = \log(I + \epsilon), \qquad \epsilon = 10^{-5}.$

For each pixel, EVIS stores a reference log-intensity $L_{\text{ref}}$. The trigger variable is

$\Delta L = L - L_{\text{ref}},$

and an event is emitted when the contrast crosses a threshold:

$$p = \begin{cases} +1 & \Delta L \ge +C \quad (\text{ON})\ -1 & \Delta L \le -C \quad (\text{OFF}) . \end{cases}$$

The default threshold is $C = 0.15$. This is the standard physically motivated formulation in which events encode brightness changes in log intensity rather than absolute intensity.

A central fidelity detail is the per-pixel asynchronous “fire-and-latch” update. When a pixel fires, its reference state is updated immediately:

$L_{\text{ref}} \leftarrow L.$

This makes each pixel track its own last triggering state independently, which the paper treats as important for matching the asynchronous behavior of real event cameras. EVIS executes these updates as batched tensor operations over all environments and cameras on the GPU.

The paper does not introduce a closed-form continuous-time multi-event solution within a single rendered interval. Instead, EVIS approximates high-rate event generation by providing the event model with a sequence of fine time-step frames. Multiple events per pixel over a coarse interval therefore arise implicitly: if the brightness repeatedly crosses threshold across rendered or synthesized intermediate frames, the same pixel can fire multiple times because $L_{\text{ref}}$ is relatched after each event. This suggests that EVIS adopts a sufficiently fine discrete-time approximation to continuous-time event formation rather than a fundamentally continuous rendering model.

3. Motion-vector interpolation and throughput engineering

A major design problem for event-camera simulation is that faithful event generation requires kHz-scale temporal sampling, while RTX rendering at such rates is prohibitively expensive. EVIS addresses this with a motion-vector interpolation pipeline that renders sparse keyframes at rate $f_k$, synthesizes $K-1$ intermediate frames, and then runs the event trigger model on the effective rate $L = \log(I + \epsilon), \qquad \epsilon = 10^{-5}.$0 stream (Shi et al., 9 Jul 2026).

The interpolation uses bidirectional warping based on motion vectors and depth. If $L = \log(I + \epsilon), \qquad \epsilon = 10^{-5}.$1 is the fractional intermediate time, then for source pixel $L = \log(I + \epsilon), \qquad \epsilon = 10^{-5}.$2,

$L = \log(I + \epsilon), \qquad \epsilon = 10^{-5}.$3

where $L = \log(I + \epsilon), \qquad \epsilon = 10^{-5}.$4 is the exact renderer motion field at keyframe $L = \log(I + \epsilon), \qquad \epsilon = 10^{-5}.$5, and $L = \log(I + \epsilon), \qquad \epsilon = 10^{-5}.$6 is approximately reused from the previous step as a forward velocity estimate. The formulation linearizes motion across the interval, so the interpolation assumes approximately constant velocity across the keyframe gap.

Because many warped source pixels may collide at one output location, EVIS uses softmax splatting. For source frame $L = \log(I + \epsilon), \qquad \epsilon = 10^{-5}.$7,

$L = \log(I + \epsilon), \qquad \epsilon = 10^{-5}.$8

with bilinear kernel

$L = \log(I + \epsilon), \qquad \epsilon = 10^{-5}.$9

inverse-depth weight

$L_{\text{ref}}$0

and $L_{\text{ref}}$1. Nearer surfaces dominate splats when several points project to the same target pixel, and zero-denominator locations are treated as holes.

The compositing rule is asymmetric. The backward-warped $L_{\text{ref}}$2 frame is primary because $L_{\text{ref}}$3 gives the exact $L_{\text{ref}}$4 correspondence; the forward-warped $L_{\text{ref}}$5 frame only fills disocclusion holes:

$L_{\text{ref}}$6

This avoids hallucinating appearance where neither source contributes. The intermediate frames are then processed by the same event trigger model as native rendered frames.

The performance effect of this interpolation is central to EVIS’s practical value. On a single RTX 5090 with one environment and one monocular $L_{\text{ref}}$7 camera, full rendering at 1000 Hz costs 13.88 ms per output frame, about $L_{\text{ref}}$8 slower than real time, whereas a $L_{\text{ref}}$9 configuration, rendering at 30 Hz and synthesizing to 240 Hz, costs 3.36 ms per output frame and runs $\Delta L = L - L_{\text{ref}},$0 faster than real time (Shi et al., 9 Jul 2026).

Configuration	Runtime	Throughput note
Full rendering at 1000 Hz	13.88 ms/output frame	About $\Delta L = L - L_{\text{ref}},$1 slower than real time
$\Delta L = L - L_{\text{ref}},$2 interpolation to 240 Hz	3.36 ms/output frame	$\Delta L = L - L_{\text{ref}},$3 faster than real time
$\Delta L = L - L_{\text{ref}},$4 with 50-pixel over-render margin	Not separately timed in ms in the summary text	$\Delta L = L - L_{\text{ref}},$5 faster than real time

The paper also states that throughput scales further with parallel environments because rendering cost amortizes across the batch. This suggests that EVIS is explicitly engineered for training-scale simulation rather than only for single-scene visualization.

4. Integration, configurability, and realism extensions

EVIS is integrated into the Isaac Sim camera system and can migrate from a standard RGB camera with minimal changes (Shi et al., 9 Jul 2026). Since it inherits camera intrinsics and extrinsics from the simulator setup, it supports monocular and stereo configurations, arbitrary robot-mounted cameras, and user-defined trajectories. Event streams are stored as HDF5 tuples $\Delta L = L - L_{\text{ref}},$6, grouped per camera.

Per-camera configuration includes contrast thresholds, output temporal rate through render rate and interpolation factor, resolution, camera layout, and optional realism features. An over-render margin can be enabled to reduce border artifacts by rendering a larger canvas, warping in that canvas, and cropping afterward. The paper also notes a specific implementation constraint: anti-aliasing must be disabled when interpolation is active, because anti-aliased silhouette fringe pixels tend to inherit background-like near-zero motion vectors rather than foreground motion, producing ghosting after warping.

Beyond the ideal event model, EVIS includes optional sensor noise and motion blur (Shi et al., 9 Jul 2026). The noise model has six configurable components. Threshold mismatch replaces a scalar threshold by per-pixel ON/OFF thresholds sampled once to model fixed-pattern noise. A refractory period suppresses later events until $\Delta L = L - L_{\text{ref}},$7. Leak events are modeled as spurious ON events with probability $\Delta L = L - L_{\text{ref}},$8. Shot noise is darker-region dependent,

$\Delta L = L - L_{\text{ref}},$9

with random polarity. Hot pixels are a fixed random subset $$p = \begin{cases} +1 & \Delta L \ge +C \quad (\text{ON})\ -1 & \Delta L \le -C \quad (\text{OFF}) . \end{cases}$$0 emitting events at elevated rate

$$p = \begin{cases} +1 & \Delta L \ge +C \quad (\text{ON})\ -1 & \Delta L \le -C \quad (\text{OFF}) . \end{cases}$$1

Finite bandwidth is modeled by low-pass filtering the log intensity,

$$p = \begin{cases} +1 & \Delta L \ge +C \quad (\text{ON})\ -1 & \Delta L \le -C \quad (\text{OFF}) . \end{cases}$$2

The Bernoulli sampling with probability $$p = \begin{cases} +1 & \Delta L \ge +C \quad (\text{ON})\ -1 & \Delta L \le -C \quad (\text{OFF}) . \end{cases}$$3 is described as a Poisson approximation at kHz rates, and all randomness is seeded so that noisy synthetic datasets remain exactly reproducible.

Motion blur is modeled by averaging fine-rate HDR frames over an exposure interval:

$$p = \begin{cases} +1 & \Delta L \ge +C \quad (\text{ON})\ -1 & \Delta L \le -C \quad (\text{OFF}) . \end{cases}$$4

The event model then operates on the blurred frames. The paper presents this as a sim-to-real realism option rather than a fully calibrated sensor-identification model.

5. Empirical evaluation and downstream usability

The evaluation of EVIS is organized around two questions: whether interpolation makes runtime practical, and whether the resulting synthetic streams are usable by pretrained event-based networks without retraining or domain adaptation (Shi et al., 9 Jul 2026). The latter point is especially important because it moves beyond visual plausibility toward statistical compatibility with downstream event-based perception models.

Three representative tasks are used. For image reconstruction, pretrained E2VID reconstructs 50 Hz intensity frames from 20 ms event windows in a warehouse scene with figure-eight camera motion and thrown YCB objects. At a common effective 1000 Hz event rate, the full 1000 Hz rendering reference achieves SSIM $$p = \begin{cases} +1 & \Delta L \ge +C \quad (\text{ON})\ -1 & \Delta L \le -C \quad (\text{OFF}) . \end{cases}$$5, MSE $$p = \begin{cases} +1 & \Delta L \ge +C \quad (\text{ON})\ -1 & \Delta L \le -C \quad (\text{OFF}) . \end{cases}$$6, and LPIPS $$p = \begin{cases} +1 & \Delta L \ge +C \quad (\text{ON})\ -1 & \Delta L \le -C \quad (\text{OFF}) . \end{cases}$$7. Interpolated variants degrade gradually: $$p = \begin{cases} +1 & \Delta L \ge +C \quad (\text{ON})\ -1 & \Delta L \le -C \quad (\text{OFF}) . \end{cases}$$8 yields $$p = \begin{cases} +1 & \Delta L \ge +C \quad (\text{ON})\ -1 & \Delta L \le -C \quad (\text{OFF}) . \end{cases}$$9, $C = 0.15$0 yields $C = 0.15$1, and $C = 0.15$2 yields $C = 0.15$3. This indicates that interpolation reduces photometric fidelity for reconstruction, though moderate factors do not collapse performance.

For dense optical flow, pretrained E-RAFT is run on pairs of 20 ms event windows using renderer motion vectors as exact dense screen-space ground truth. The average endpoint error remains subpixel across all evaluated settings: $C = 0.15$4 px for full rendering, $C = 0.15$5 px for $C = 0.15$6, and $C = 0.15$7 px for both $C = 0.15$8 and $C = 0.15$9, with correlation around $L_{\text{ref}} \leftarrow L.$0–$L_{\text{ref}} \leftarrow L.$1. The paper attributes this robustness to the interpolation design, which preserves motion geometry more reliably than photometric detail.

For cross-view event matching, using two event cameras with approximately $L_{\text{ref}} \leftarrow L.$2 view separation, pretrained Match-Any-Events achieves $L_{\text{ref}} \leftarrow L.$3 matching precision and pose AUCs $L_{\text{ref}} \leftarrow L.$4 at $L_{\text{ref}} \leftarrow L.$5 for the full 1000 Hz reference. The $L_{\text{ref}} \leftarrow L.$6 setting remains close at $L_{\text{ref}} \leftarrow L.$7 precision, while $L_{\text{ref}} \leftarrow L.$8 and $L_{\text{ref}} \leftarrow L.$9 decline more clearly to $L_{\text{ref}}$0 and $L_{\text{ref}}$1. The consistent pattern across tasks is that moderate interpolation remains usable, while aggressive interpolation more noticeably affects fine downstream performance.

The qualitative analysis identifies two principal failure modes. Temporal quantization appears when motion is fast enough that an object moves several pixels per synthesized frame, producing banding in the event surface; this can arise even without interpolation if the base sampling rate is too low. Interpolation-specific artifacts appear under rotation and occlusion: newly revealed surfaces lack source pixels and therefore generate pulsed events at the keyframe rate, while rotational motion deviates from the constant-velocity assumption used by the warping model. The recommended mitigation is to increase base render rate when scenes contain severe occlusion or fast nonlinear motion.

6. Limitations, interpretation, and significance

EVIS has several explicit limitations (Shi et al., 9 Jul 2026). Its event generation remains a discrete-time approximation driven by rendered or synthesized frames rather than a fully continuous rendering process, so threshold-crossing fidelity depends on effective temporal resolution. Its interpolation assumes approximately linear motion across keyframe gaps and depends on renderer motion vectors; this creates visible artifacts under strong occlusion, disocclusion, and rotational motion. The compositing rule leaves uncovered regions black rather than hallucinating content, which avoids fabricated appearance but can create holes. Interpolation also requires anti-aliasing to be disabled, and border artifacts may appear without over-render margins.

The realism options are intentionally practical rather than exhaustively calibrated. The paper provides qualitative realism evidence for threshold mismatch, leak events, and motion blur, but does not present a numerical calibration against a specific hardware sensor or a measured sim-to-real transfer benchmark. It likewise does not deeply analyze HDR rendering fidelity, material reflectance, or lighting transients in Isaac Sim, though the paper notes that these issues are likely relevant to event generation.

These constraints delimit the scope of EVIS. It is not presented as a physically complete sensor simulator; rather, it is a practical bridge between event-camera research and modern GPU robotics simulation. Its significance lies in the combination of a physically motivated log-intensity trigger, per-pixel asynchronous state updates, simulator-native physics and labels, and a motion-vector interpolation pipeline that makes high-rate labeled event generation computationally feasible on a single GPU. This suggests that EVIS’s main contribution is infrastructural as much as algorithmic: it brings event cameras into the standard Isaac Sim / Isaac Lab robotics workflow as a directly usable, configurable, and performance-conscious sensing modality (Shi et al., 9 Jul 2026).