xView3-SAR: Dark Vessel Detection Benchmark

• xView3-SAR is a comprehensive maritime remote-sensing dataset using Sentinel-1 SAR imagery to detect and analyze dark vessels.
• It provides analysis-ready scenes with dual-polarization data and ancillary environmental layers to support detailed vessel, fishing, and near-shore classification.
• The benchmark frames maritime monitoring as a multitask problem, enabling robust evaluation of detection, classification, and length estimation methodologies.

Searching arXiv for xView3-SAR and closely related benchmark papers to ground the article in the cited literature. arxiv_search(query="xView3-SAR Detecting Dark Fishing Activity Using Synthetic Aperture Radar Imagery", max_results=5) arxiv_search(query="xView3-SAR", max_results=10) Searching for the core xView3-SAR dataset paper and follow-on benchmark papers. xView3-SAR is a large-scale, analysis-ready maritime remote-sensing dataset and benchmark for detecting and characterizing “dark” vessels in synthetic aperture radar imagery, with a central application in monitoring illegal, unreported, and unregulated fishing when optical imagery is unavailable and AIS-based monitoring is incomplete (Paolo et al., 2022). Built from Sentinel-1 SAR imagery and released with annotations, ancillary environmental rasters, and challenge infrastructure, it frames maritime monitoring as a multitask problem comprising maritime object detection, close-to-shore detection, vessel versus fixed-infrastructure classification, fishing versus non-fishing classification, and vessel length estimation rather than ship detection alone (Paolo et al., 2022). Subsequent work has also used xView3 as a benchmark for automated fishing-activity detection from SAR scenes, emphasizing the dataset’s role in SAR-based inference of fishing behavior as well as vessel presence (Bai et al., 2024).

1. Operational motivation and benchmark scope

xView3-SAR was created to address a specific monitoring gap: many fishing vessels do not carry AIS, and some shut it off to hide illicit activity, producing “dark vessels” that are difficult to monitor with conventional systems (Paolo et al., 2022). The motivating application is directly tied to IUU fishing, which the dataset paper describes as accounting for more than 20% of global catch and causing major ecological and economic harm (Paolo et al., 2022). Traditional monitoring by patrols or optical satellite imagery is limited by scale, cost, weather, and daylight, whereas SAR can image the ocean day or night and in nearly all weather conditions (Paolo et al., 2022).

The benchmark was therefore designed around the practical strengths and limitations of SAR. Maritime objects in open-ocean Sentinel-1 scenes are relatively small and sparse, and SAR requires domain-specific preprocessing and calibration that are not standard in mainstream computer vision pipelines (Paolo et al., 2022). xView3-SAR addresses both issues by releasing full, analysis-ready scenes and by combining automated AIS-to-SAR matching with expert human verification to create large-scale labels (Paolo et al., 2022).

A common simplification is to treat xView3-SAR as merely a vessel-detection dataset. The benchmark definition is broader. The official challenge formulation includes object detection, close-to-shore detection, vessel characterization, fishing classification, and length estimation, which makes the benchmark closer to an operational maritime-domain-awareness workflow than to a single-task detector leaderboard (Paolo et al., 2022). A later study on fishing activity detection makes this explicit by describing xView3 as a multi-task maritime detection problem in which the system must detect objects in SAR scenes, decide whether a marine object is a vessel, decide whether it is fishing, and determine whether it is near shore (Bai et al., 2024).

2. Dataset composition and analysis-ready preprocessing

xView3-SAR uses ESA Sentinel-1 imagery in Interferometric Wide swath mode Level-1 Ground Range Detected format (Paolo et al., 2022). The dataset contains 991 full-size analysis-ready SAR images covering around 43.2 million square kilometers, with about 1,421.81 gigapixels in total (Paolo et al., 2022). The images are, on average, about 29,400 × 24,400 pixels each (Paolo et al., 2022). Sentinel-1 contributes two polarization channels, VH and VV; the dataset paper states that the imagery has about 20 m spatial resolution and 10 m × 10 m pixel spacing (Paolo et al., 2022). A later xView3 benchmark study summarizes the same setting as two SAR channels at 10 m/pixel and auxiliary channels at 500 m/pixel (Bai et al., 2024).

The dual-polarization design is operationally significant. The dataset paper states that VH is typically better for vessel detection because sea clutter returns weakly in cross-polarization while ships often stand out strongly, whereas VV is more sensitive to sea-surface texture and contextual phenomena such as wind patterns, oil slicks, sediment plumes, and offshore structures (Paolo et al., 2022). This makes xView3-SAR useful for both target detection and contextual characterization.

Every SAR image is accompanied by ancillary environmental layers. These include bathymetry from GEBCO 2020 and surface-wind information from Sentinel-1 Level-2 Ocean products: wind speed, wind direction, wind quality, and land/ice masks (Paolo et al., 2022). These rasters are clipped to scene extent, reprojected to UTM, and resampled to 500 m pixel spacing (Paolo et al., 2022). The ancillary layers are important for error analysis, especially in near-shore regions where coastlines, islands, rocks, and infrastructure generate difficult clutter (Paolo et al., 2022).

A major contribution of xView3-SAR is that it is analysis-ready rather than raw SAR. The preprocessing pipeline uses ESA SNAP and includes orbit correction, noise artifact removal, radiometric calibration, terrain correction, reprojection to WGS84, distribution in UTM projection, resampling to 10 m pixel spacing, and storage as GeoTIFFs using half-precision floating point to reduce file size by 50% (Paolo et al., 2022). This design removes a substantial portion of the conventional SAR preprocessing burden for machine-learning users.

3. Annotation methodology, label semantics, and task definition

The labeling strategy combines automated and manual sources. Global Fishing Watch produced automated labels using a CFAR-based ship detection pipeline, a ConvNet classifier/regressor for filtering detections and estimating length, and probabilistic AIS-to-SAR matching, yielding more than 161,000 detections from 2020 used in dataset construction (Paolo et al., 2022). Professional labelers then manually annotated 437 SAR images, producing about 176,000 labeled detections with tight bounding boxes and confidence levels (Paolo et al., 2022).

The final dataset contains 243,018 verified maritime objects (Paolo et al., 2022). Of these, 39.1% have both automated and manual annotations, 33.4% are manual-only, and 27.5% are automated-only (Paolo et al., 2022). The primary categories are vessel and non-vessel, but the challenge formulation expands the semantic structure into object detection, close-to-shore object detection, vessel versus fixed-infrastructure classification, fishing versus non-fishing classification, and vessel length estimation (Paolo et al., 2022).

Confidence is part of the label semantics. The dataset uses High, Medium, and Low confidence for vessels, while non-vessels use High and Medium (Paolo et al., 2022). Confidence is derived from agreement between AIS-based automation and human labelers (Paolo et al., 2022). A later xView3 study stresses that the annotations combine vessel detections from the Global Fishing Watch model and AIS/VMS records and include location, vessel length, source record, vessel/fishing labels, distance from coastline, and confidence (Bai et al., 2024).

This label structure produces both scale and difficulty. The xView3 benchmark paper identifies hard cases arising from small and sparse objects, SAR-specific visual statistics, and the challenge of acquiring ground truth for vessels that are absent from AIS records (Paolo et al., 2022). The later benchmarking study adds that SAR interpretation depends on incidence angle, vessel material, radar cross-section, weather and sea state, and noisy or incomplete labels, making xView3 a deliberately difficult benchmark rather than a clean object-detection corpus (Bai et al., 2024).

4. Challenge design, evaluation protocol, and reported baselines

xView3-SAR was released as the basis for the xView3 Computer Vision Challenge, launched in August 2021 to detect and characterize dark fishing activity with SAR (Paolo et al., 2022). The challenge split the data into 554 train images, 50 validation images, 150 public images, and 237 holdout images (Paolo et al., 2022). Only the train and validation sets were given to competitors; the public set supported the leaderboard, and the holdout set was reserved for final evaluation (Paolo et al., 2022).

The official tasks were maritime object detection, close-to-shore object detection, vessel classification, fishing classification, and vessel length estimation (Paolo et al., 2022). The evaluation used four F1 scores and one length-estimation term: $F1_D$ for all object detection, $F1_S$ for objects within 2 km of shore, $F1_V$ for vessel versus fixed infrastructure, $F1_F$ for fishing versus non-fishing, and $PE_L$ as aggregate percent error for length estimation (Paolo et al., 2022). The overall ranking metric was defined as

$$M_R = F_D \times \frac{1 + F1_S + F1_V + F1_F + PE_L}{5}.$$

This multitask metric was intended to keep scores between 0 and 1, ensure that poor detection hurts the overall result, and count improvements on any subtask equally (Paolo et al., 2022).

A reference model based on Faster R-CNN with an ImageNet-pretrained backbone was released with the benchmark (Paolo et al., 2022). It used 800 × 800 chips, normalized pixel values, fixed 10-pixel bounding boxes around detections, and three classes: non-vessel, non-fishing vessel, and fishing vessel (Paolo et al., 2022). Training used 5 epochs with SGD and momentum, learning rate $5 \times 10^{-3}$, momentum 0.9, weight decay $5 \times 10^{-4}$, and a step learning-rate scheduler with step size 3 and $\gamma = 0.1$ (Paolo et al., 2022). On the holdout set, the reference model achieved $F1_D = 0.4302$, $F1_S$0, $F1_S$1, $F1_S$2, $F1_S$3, and an aggregate score of 0.1904 (Paolo et al., 2022).

The top challenge solutions were substantially stronger. The best performer achieved aggregate score 0.6177 with $F1_S$4, $F1_S$5, $F1_S$6, $F1_S$7, and $F1_S$8 (Paolo et al., 2022). The dataset paper reports that winning approaches often used CenterNet-like or Objects-as-Points ideas, U-Net-style decoders, task-specific heads for objectness, length, vessel classification, and fishing classification, and SAR-specific augmentation and apodization to reduce diffraction artifacts (Paolo et al., 2022). Even so, near-shore detection remained weak across models because shorelines introduce clutter, multipath effects, and ambiguity with rocks and man-made structures (Paolo et al., 2022).

The challenge also imposed an explicit efficiency constraint: models had to process a full 29k × 24k SAR scene in under 15 minutes on a single Tesla V100 with 60 GB RAM (Paolo et al., 2022). This requirement situated xView3-SAR as an operational benchmark rather than a purely academic dataset.

5. Benchmark findings from later xView3 studies

A later study, “FAD-SAR: A Novel Fishing Activity Detection System via Synthetic Aperture Radar Images Based on Deep Learning Method,” used xView3 to benchmark six classical object detectors for automated fishing activity detection from SAR imagery: SSD, RetinaNet, FSAF, FCOS, Faster R-CNN, and Cascade R-CNN (Bai et al., 2024). Because original xView3 scenes are very large, that study followed the xView3 cropping convention and cut them into 800 × 800 patches (Bai et al., 2024). It also examined multiple ways of constructing a third input channel, including a single auxiliary channel, mean embedding of VV and VH, difference embedding of VV and VH, mean embedding of auxiliary channels, and mean embedding of all channels (Bai et al., 2024). The reported conclusion was that auxiliary channels did not improve performance, plausibly because their lower resolution introduced spatial mismatch when upsampled; the final choice was the mean embedding of VV and VH as the third channel, mainly for stability rather than adding new signal (Bai et al., 2024).

That study filtered labels heavily by keeping only high-confidence labels to reduce noise, merged is_vessel and is_fishing into fishing, non_fishing, and non_vessel, and created artificial bounding boxes because the original labels are point centroids rather than boxes (Bai et al., 2024). A bounding-box ablation over 10×10, 20×20, 30×30, and 40×40 found that 20×20 gave the best Faster R-CNN result, with $F1_S$9, $F1_V$0, $F1_V$1, $F1_V$2, and Avg-F1 $F1_V$3 (Bai et al., 2024).

For aggregation, the study argued that the original baseline’s treatment of a length-error term was not ideal and instead used

$F1_V$4

Under that metric, the six detector families formed a clear ranking (Bai et al., 2024).

Model	Detector family	Avg-F1
Faster R-CNN	Two-stage	0.21215
Cascade R-CNN	Two-stage	0.20445
SSD	One-stage	0.20295
RetinaNet	One-stage	0.15518
FSAF	Anchor-free	0.12053
FCOS	Anchor-free	0.06312

The same study interpreted the results as evidence that two-stage detectors fit xView3 better because the benchmark contains small, sparse, and noisy targets, and proposals help refine localization and classification (Bai et al., 2024). Cascade R-CNN improved near-shore detection relative to Faster R-CNN but lost some overall recall because stricter IoU thresholds reduced the number of positive samples (Bai et al., 2024). SSD achieved the best $F1_V$5 among all methods, which the authors attributed to multi-scale detection branches that help with small objects such as ships in SAR images (Bai et al., 2024). Anchor-free detectors, particularly FCOS, performed poorly in that setup (Bai et al., 2024).

The main refinement in that study was to enhance Faster R-CNN with data augmentation, Deformable ConvNets v2, IoU-Balanced Sampling, and Online Hard Example Mining (Bai et al., 2024). OHEM was the only variant that improved over the baseline, increasing Avg-F1 from 0.21215 to 0.21631, whereas data augmentation yielded 0.20291, DCNv2 0.20762, and IoU-Balanced Sampling 0.20724 (Bai et al., 2024). The authors also compared their final OHEM-trained Faster R-CNN with the xView3 baseline and reported Avg-F1 values of 0.21631 versus 0.15176 (Bai et al., 2024). A central limitation identified in that paper was near-shore supervision: removing low- and medium-confidence labels reduced noise but also left many near-shore objects unclassified or excluded, contributing to very low $F1_V$6 values across most detectors (Bai et al., 2024).

6. Extensions, deployment, and broader significance

xView3-SAR has also become a benchmark for deployment-oriented SAR vessel detection. A later paper on FPGA-based on-satellite sensing trained and evaluated a customized YOLOv8 architecture on xView3-SAR, explicitly positioning the dataset as the largest and most diverse open SAR vessel dataset and as the best public proxy for operational maritime-surveillance conditions (Laganier et al., 7 Jul 2025). That work used 800 × 800 chips with three channels—VV, VH, and bathymetry—and emphasized that xView3 is not only a detection benchmark but also a classification problem involving vessel versus non-vessel and fishing versus non-fishing decisions under the official label rules (Laganier et al., 7 Jul 2025).

The same deployment study reported that its best floating-point GPU-trained model, YOLOv8s-Ghost-P2-PIoU2 with two-stage training plus adaptive thresholding, achieved 0.703 detection F1, 0.444 near-shore F1, 0.928 vessel-classification F1, and 0.770 fishing-classification F1 on xView3-SAR (Laganier et al., 7 Jul 2025). The corresponding nano model reached 0.704 / 0.447 / 0.925 / 0.769 (Laganier et al., 7 Jul 2025). The deployed INT8 FPGA version of YOLOv8n-Ghost-P2-PIoU2 achieved 0.704 detection F1, 0.445 near-shore F1, 0.921 vessel F1, and 0.765 fishing F1, while operating within a sub-10 W envelope on a Kria KV260 MPSoC (Laganier et al., 7 Jul 2025). The paper further states that the system can process a full 40,000 km² SAR scene in under one minute (Laganier et al., 7 Jul 2025). This indicates that xView3-SAR has matured from a dataset for offline benchmarking into a testbed for low-power, time-critical inference.

Beyond vessel detection, related SAR literature points to broader trajectories without changing the core identity of xView3-SAR itself. A 2018 study on SAR-optical stereogrammetry demonstrated that SAR can serve as a high-accuracy geometric backbone for cross-modal fusion and 3D reconstruction over urban areas, using RPC modeling, block adjustment, and SAR-optical epipolar reasoning (Bagheri et al., 2018). A later neural-surface-reconstruction framework fused aerial imagery with 3D SAR point clouds and argued that radar-derived geometry priors improve accuracy, completeness, and robustness under sparse-view conditions (Li et al., 29 Jan 2026). These works are not xView3 benchmark papers, but they show that SAR’s value extends beyond maritime detection into multimodal geometric inference.

Within maritime monitoring, xView3-SAR remains distinctive because it unifies analysis-ready SAR data, large-scale labels, ancillary context, challenge protocols, and operational constraints around the problem of dark fishing activity (Paolo et al., 2022). Its central technical lesson is that vessel monitoring in SAR is not reducible to bright-target detection alone: performance depends on joint reasoning over detection, proximity to shore, vessel identity, fishing behavior, and, in the original challenge, vessel length (Paolo et al., 2022). Subsequent xView3 studies reinforce this view by showing both the continued competitiveness of classical two-stage detectors under noisy small-target conditions and the feasibility of compact, deployment-oriented architectures that preserve most of the performance of far larger GPU systems (Bai et al., 2024, Laganier et al., 7 Jul 2025).