ZED Mini Stereo Camera

• ZED Mini Stereo Camera is a compact, gimbal-mountable stereo vision system that provides real-time depth mapping for mobile robotics and UAV applications.
• It employs a 12 cm stereo baseline and a histogram filter for noise reduction, ensuring high accuracy in dynamic, short-to-midrange depth measurements.
• Integrated with NVIDIA Jetson TX2, the sensor delivers synchronized 1280×720 imaging and robust 3D tracking through efficient disparity computation and 3D back-projection.

The ZED Mini Stereo Camera is a compact, gimbal-mountable, passive stereo vision system designed to enable short-to-midrange depth perception in mobile robotics and UAV applications. Its application as the sole depth-sensing module in pursuit-evasion scenarios has been extensively documented, notably as the central perception sensor in a UAV-based interception system capable of reconstructing and pursuing complex, dynamic targets. The ZED Mini integrates factory-calibrated intrinsics, a 12 cm stereo baseline, and wide-angle optics to deliver disparity-derived depth maps in real time, supporting direct integration with neural network-based object detection and robust 3D localization pipelines (Barisic et al., 2021).

1. Hardware Configuration and Sensing Characteristics

The ZED Mini system operates with dual color cameras, each providing synchronized 1280×720 resolution images for both left and right streams. Mounted on a three-axis gimbal and interfaced via an NVIDIA Jetson TX2, it operates in “Ultra” depth mode, delivering depth maps at rates up to 15 Hz; the practical operating point for a full detection-to-depth processing pipeline is approximately 7 Hz. The stereo camera features a 12 cm baseline and offers a horizontal field of view of approximately 90°, with a vertical field of view of 60°. Empirical characterization restricts the operational depth range to 2–15 m to mitigate both computational load and performance degradation due to low SNR at extreme distances; the hardware is nominally capable of depth measurement up to 20 m, but this extended envelope is not exploited in high-reliability settings.

2. Calibration: Intrinsic and Extrinsic Parameters

Intrinsic calibration for the ZED Mini adopts factory-supplied pinhole parameters—focal lengths $f_x, f_y$ and principal points $c_x, c_y$—directly from the ZED SDK, with no additional refinement reported. Extrinsic calibration entails determination of a rigid-body transform $T_{FC}$ relating the camera’s optical frame $C$ to the UAV’s “follower” frame $F$. This calibration, measured once and accounting for gimbal mounting geometry, allows transformation of per-frame 3D points from camera to vehicle coordinates: $P_F = T_{FC}\cdot P_C$. With integration of the UAV’s global pose $T_{GF}$ from onboard GPS/INS, the global point is determined by $P_G = T_{GF}\cdot T_{FC}\cdot P_C$ (Barisic et al., 2021). This multi-stage calibration approach ensures that stereo-derived measurements are appropriately situated for navigation and trajectory estimation.

3. Stereo Disparity to Depth Estimation

The ZED Mini computes per-pixel metric depth by evaluating the disparity $\Delta(u,v)$ between matched pixels in the left and right images, applying the standard pinhole stereo formula: $Z(u, v) = \frac{f \cdot B}{\Delta(u, v)}$ where $c_x, c_y$0 m (stereo baseline), and $c_x, c_y$1 is the focal length in pixels (with $c_x, c_y$2). Given the computed depth $c_x, c_y$3, 2D image coordinates $c_x, c_y$4 are back-projected to 3D camera coordinates by

$c_x, c_y$5

This process generates dense point clouds synchronized with the RGB image stream, underpinning object localization and further trajectory analysis.

4. Depth Measurement Denoising: Histogram-Filter Approach

The sparse, wire-frame morphology of UAVs induces highly non-Gaussian, spiky distributions of raw per-pixel depth values within detected object masks—causing instability in 3D position estimates. To mitigate this, a noise-reducing histogram filter is deployed as follows:

• Depth samples $c_x, c_y$6 within the bounding box (produced by YOLOv3-Tiny object detection) are distributed into $c_x, c_y$7 uniform bins spanning [2 m, 15 m].
• For each depth $c_x, c_y$8, the relevant histogram bin is incremented.
• Local histogram peaks are identified, and the mean value $c_x, c_y$9 over these peaks is computed.
• Only peaks with $T_{FC}$0 are retained.
• The nearest such peak provides the candidate, with $T_{FC}$1 set to the mean of all $T_{FC}$2 in that bin.

This strategy effectively rejects outlier depths in front of or behind the true UAV and reliably extracts the densest, inlier cluster (presumed to correspond to the actual target), providing a single robust depth $T_{FC}$3 per frame (Barisic et al., 2021).

5. Visual Perception and 3D Tracking Pipeline

The ZED Mini’s output feeds into an end-to-end perception pipeline structured as follows:

1. Image acquisition: Dual 1280×720 streams from ZED Mini.
2. Disparity computation: ZED SDK in “Ultra” mode.
3. Object detection: YOLOv3-Tiny inference on 608×608 crops, operated at ≈7 Hz.
4. ROI expansion and data association
5. Histogram-filtered depth computing: As described above, extracting $T_{FC}$4 per frame.
6. Back-projection: 2D centroid and depth combined to yield $T_{FC}$5 in camera coordinates.
7. Coordinate transformation: Application of extrinsic ($T_{FC}$6) and global ($T_{FC}$7) transforms.
8. Temporal filtering: Discrete Kalman filter (constant-velocity model) for trajectory smoothing.
9. Output: 3D global position time series $T_{FC}$8, enabling trajectory fitting (e.g., Bernoulli lemniscate) and real-time interception planning.

This pipeline is computationally tractable for deployment on embedded hardware (NVIDIA Jetson TX2) and is designed for robustness in both simulated and field operational contexts.

6. Depth Accuracy and Performance Evaluation

Empirical evaluation showcases the ZED Mini perception pipeline’s accuracy under both simulated and real-world conditions:

• In software-in-the-loop simulations (Gazebo), mean absolute error (MAE) in global 3D position reconstruction is approximately 0.04 m, with RMSE of 0.05 m.
• Indoor motion-capture (OptiTrack) experiments with a moving target and static camera yield RMSE of 0.15 m (MAE 0.10 m). Under conditions where both camera and target move abruptly, RMSE rises to ≈0.30 m (MAE 0.21 m), with error increasing approximately linearly as target distance extends (up to roughly 0.6 m RMSE at 15 m).
• The histogram filter alone reduces depth outliers by over 80%, and suppresses frame-to-frame jitter to ±0.1–0.2 m at middle distances.

Observed system precision (≲20 cm practical noise) suffices for capturing and reconstructing figure-eight (Bernoulli lemniscate) trajectories and planning interception of UAVs moving up to 30% faster than the interceptor vehicle (Barisic et al., 2021).

7. Research Impact and Context

The ZED Mini Stereo Camera, when augmented with robust depth post-processing and tight integration with neural-network-based detection, enables high-fidelity 3D motion estimation in real-world pursuit-evasion tasks. In “Brain over Brawn,” the sensor’s utilization underscores several core methodologies in modern robotic perception: direct use of factory-supplied intrinsics/extrinsics, exploitation of dense stereo matching for metric localization, and adaptive outlier rejection via nonparametric depth filters. The demonstrated accuracy and reliability in dynamic, unstructured environments affirm the ZED Mini’s viability as a research-grade instrument for 3D vision-driven robotic control, particularly in applications requiring precise trajectory inference and interception within variable depth envelopes (Barisic et al., 2021).

A plausible implication is that future deployment in scenarios exhibiting greater environmental complexity or occlusion may benefit from further algorithmic innovations addressing depth ambiguity and computation load. However, the empirical evidence from the cited research establishes a clear baseline for robust performance in UAV-based interception using commercially available stereo vision hardware.