Maritime Generation Dataset Benchmark

• Maritime Generation Dataset (MGD) is a benchmark featuring 11,900 real maritime images with detailed annotations for controllable image synthesis and object detection.
• It incorporates multi-modality conditioning through water masks, object boxes, textual descriptions, and discrete attribute labels to address data scarcity and imbalance.
• MGD supports training diffusion models and maritime detectors while enabling synthetic augmentation to improve performance under rare and underrepresented conditions.

The Maritime Generation Dataset (MGD) is a generation-oriented maritime benchmark introduced in “Neptune-X: Active X-to-Maritime Generation for Universal Maritime Object Detection” as “the first generation dataset for maritime scenarios” and “the first generative benchmark for maritime detection.” It contains 11,900 real maritime images with annotations structured for both multi-modality-conditioned image synthesis and maritime object detection: water masks, object boxes and masks, multi-level textual descriptions, and attribute labels spanning viewpoint, location, imaging environment, and object category (Guo et al., 25 Sep 2025). Within Neptune-X, MGD functions simultaneously as the training substrate for the X-to-Maritime generator and as the reference dataset used to filter, condition, and evaluate synthetic data for downstream maritime detection.

1. Definition and design rationale

MGD was created to address two deficits identified in existing maritime datasets: the scarcity of annotated maritime data and weak generalization across maritime attributes such as object category, viewpoint, location, and imaging environment. Existing datasets were characterized as detection-focused, costly to annotate, severely imbalanced across these attributes, and generally lacking labels suitable for controllable image synthesis, such as water masks and scene-level or object-level textual descriptions (Guo et al., 25 Sep 2025).

The dataset is therefore designed around the requirements of generative maritime learning. Each sample is intended to support three coupled functions: training a layout- and text-conditioned diffusion model, providing the attribute labels needed by Attribute-correlated Active Sampling (AAS), and enabling unified evaluation of both image generation and maritime object detection. In that sense, MGD is not merely a collection of maritime images; it is an annotation regime aligned to the conditioning interfaces actually consumed by Neptune-X.

A plausible implication is that MGD formalizes maritime generation as a multi-condition supervision problem rather than a purely synthetic-data problem. The benchmark is built from real images, but its annotation schema is optimized for controllable synthesis, attribute-aware sampling, and downstream detector improvement, rather than for conventional single-task detection alone (Guo et al., 25 Sep 2025).

2. Corpus composition and annotation schema

MGD contains 11,900 real RGB maritime images. For training the generator, images are used as 512×512 crops/patches, while raw resolutions vary according to source datasets. The official split is 3:1:1, with validation and test combined for generation evaluation (Guo et al., 25 Sep 2025).

Split	Images	Role
Train	7,140	Generator and detector training
Val	2,380	Held-out evaluation
Test	2,380	Held-out evaluation

Each sample includes five annotation layers. First is the RGB image itself. Second is a water surface mask $\mathcal{M}_w$, a binary segmentation mask of the water region, from which a bounding rectangle $P_w$ is derived. Third are object annotations: bounding boxes, 5-class labels, binary object masks $\mathcal{M}_o^i$, and textual object labels $L_o^i$. Fourth are multi-level textual descriptions, including an image-level description, a water-surface description, and object descriptions. Fifth are the discrete attribute labels used by AAS: viewpoint, location, imaging environment, and object category (Guo et al., 25 Sep 2025).

The five object categories are:

• ship: 72.44% of instances (29,313)
• buoy: 13.16% (5,326)
• person: 11.97% (4,843)
• floating object: 1.53% (618)
• fixed object: 0.90% (366)

The viewpoint attribute has 3 classes: shore-based (50.77%, 6,042 images), shipboard (20.66%, 2,459), and aerial (28.56%, 3,399). The location attribute has 4 classes: sea (48.98%, 5,829), river (46.48%, 5,531), harbor (2.37%, 282), and lake (2.17%, 258). The imaging environment attribute has 6 classes: sunny (54.55%, 6,491), cloudy (23.48%, 2,794), foggy (10.29%, 1,225), rainy (4.33%, 515), dawn/dusk (4.90%, 583), and night (2.45%, 292) (Guo et al., 25 Sep 2025).

These distributions expose pronounced long-tail structure. Harbor and lake scenes are rare; night and rainy imagery are underrepresented; non-ship classes are sparse. The paper explicitly treats this imbalance not as noise in the benchmark design but as the empirical basis for targeted generation and active sample selection.

3. Construction and labeling workflow

MGD aggregates imagery from eight named maritime datasets plus additional captured data. The dataset composition is as follows (Guo et al., 25 Sep 2025):

Source	Images
MaSTr1325	800
USVInland	1000
MIT Sea Grant	100
SMD	400
Seaships	1500
Seagull	2996
Fvessel	1500
LaRS	1973
Other captured data	1631

The “other captured data” includes surveillance, UAV, smartphone, and DSLR imagery. The full corpus spans shore / ship / aerial viewpoints, and the collection procedure explicitly sought diversity in geographies, times, weather, and traffic (Guo et al., 25 Sep 2025).

Annotation is semi-automatic and multi-stage. For geometric labels, T-Rex2 proposes object bounding boxes and SAM2 segments the water surface given point prompts; human annotators then verify and correct both boxes and water masks. For textual and attribute labels, LLaVA-Next generates image descriptions, water descriptions, and object descriptions using carefully designed prompts, after which human validation ensures correctness. Discrete labels for viewpoint, location, imaging environment, and category are then extracted from these descriptions (Guo et al., 25 Sep 2025).

This workflow is significant because it yields annotation layers that conventional maritime detection datasets generally do not provide in a harmonized form. MGD retains the realism of heterogeneous source imagery while imposing a common taxonomy, common attribute axes, and common conditioning modalities suitable for generation.

4. Role in X-to-Maritime and attribute-aware selection

Within Neptune-X, each MGD training sample provides an image $I$, a caption $\mathcal{C}$, object conditions, and a water condition. For an object $i$ with class label $L_o^i$ and coordinates $P_o^i$, the conditioning pathway is defined as (Guo et al., 25 Sep 2025):

$$\mathbf{e}_o^i = \Phi(P_o^i),\quad \mathbf{t}_o^i = \xi(L_o^i),\quad \mathcal{C}_o^i = \mathrm{MLP}([\mathbf{e}_o^i;\mathbf{t}_o^i]).$$

The water description is encoded analogously:

$P_w$0

The generator is trained with a multi-condition denoising objective:

$P_w$1

MGD is also the basis for Attribute-correlated Training Difficulty Factors (ATDFs). For a matched ground-truth box $P_w$2 and prediction $P_w$3 with confidence $P_w$4, the paper defines prediction accuracy as:

$P_w$5

For attribute $P_w$6 at iteration $P_w$7, the initial difficulty is:

$P_w$8

Using X-to-Maritime trained on MGD, the authors generate about 100k synthetic images conditioned on layouts and text derived and randomized from MGD labels. After filtering with a ResNet classifier for layout and CLIP similarity for semantics, AAS ranks candidates by difficulty and task relevance. The selected subset is then merged with the original MGD train split to fine-tune the detector (Guo et al., 25 Sep 2025).

A plausible implication is that MGD operationalizes a closed loop between benchmark design and synthetic augmentation: the same attribute axes that describe the real dataset also govern difficulty estimation, synthetic candidate generation, and selection of high-value synthetic samples.

5. Benchmarks and empirical performance

MGD defines the reference distribution for both generation and detection evaluation. For image generation, all experiments are evaluated against the held-out MGD val+test split using FID, Classification Accuracy Score (CAS), and YOLO Score. CAS is obtained by training a ResNet-101 classifier on MGD, while YOLO Score is obtained by training YOLOv10 on MGD and reporting mAP, $P_w$9, and $\mathcal{M}_o^i$0 on generated images (Guo et al., 25 Sep 2025).

On this benchmark, X-to-Maritime trained on MGD achieves FID 18.05, CAS 79.34, and YOLO Score 17.08 / 39.14 / 13.52 for mAP / $\mathcal{M}_o^i$1 / $\mathcal{M}_o^i$2, outperforming LayoutDiff, GLIGEN, InstDiff, and RC-L2I (Guo et al., 25 Sep 2025). The benchmark is therefore not limited to human-perceptual realism; it explicitly measures whether synthetic imagery is useful for a maritime detector trained on MGD’s own distribution.

For downstream maritime object detection on the MGD test split, the paper evaluates YOLOv10, YOLOv11, YOLOv12, and Grounding DINO. The strongest reported gains come from augmenting MGD with AAS-selected synthetic samples. For YOLOv10, mAP improves from 39.99 to 43.62, and $\mathcal{M}_o^i$3 from 61.13 to 65.50. For YOLOv11, mAP improves from 41.29 to 44.43; for YOLOv12, from 39.06 to 42.91. Grounding DINO improves from 65.03 / 86.12 to 68.04 / 89.86 in mAP / $\mathcal{M}_o^i$4 (Guo et al., 25 Sep 2025).

The attribute-wise analysis reported in Figure 1 shows that average detection accuracy increases by 13.77%, while variance across attributes decreases by 6.39%. This indicates that synthetic data targeted to the rare and difficult attribute combinations already encoded in MGD improves robustness under underrepresented scenarios such as open sea or extreme illumination (Guo et al., 25 Sep 2025).

6. Relation to adjacent maritime generation resources and limitations

MGD differs from earlier maritime datasets such as MaSTr1325, USVInland, Seaships, Seagull, and LaRS, which are described as primarily detection or segmentation datasets with limited or no water masks, no standardized multi-scale text descriptions, and incomplete or inconsistent labeling of viewpoint, location, and imaging environment (Guo et al., 25 Sep 2025). Its novelty lies less in raw image acquisition than in harmonizing real imagery into a conditioning-ready benchmark for controllable generation.

At the same time, adjacent work shows that the phrase maritime generation dataset now covers several distinct dataset paradigms. MariData is a 7,000-image domain-labeled corpus for Day-to-Foggy, Day-to-Sunset, and Day-to-Night unpaired translation, emphasizing structure-preserving atmospheric synthesis rather than layout-conditioned object generation (Henriksson et al., 2 Jun 2026). SafeSea is a 69,694-image semi-synthetic corpus built by replacing calm sea backgrounds with synthetic Sea State 1–4 backgrounds while preserving real boats, targeting robustness of object detection under rough water conditions (Tran et al., 2023). AIMO is a 66,626-image synthetic classification dataset generated with diffusion models across 15 maritime classes and 5 weather/illumination domains, intended as a labeled source domain for unsupervised adaptation to real maritime imagery (Song et al., 26 Jan 2025). MSC supplies video–text–segmentation triplets for underwater marine video understanding and text-to-video evaluation rather than above-water navigation scenes (Truong et al., 6 Aug 2025). This suggests that MGD represents one specific lineage within a broader family: real-image, attribute-rich, generation-oriented maritime benchmarks.

The limitations identified for MGD are structural rather than incidental. The dataset relies on a fixed, discrete attribute set—viewpoint, location, imaging environment, and object type—which may constrain finer-grained modeling such as continuous sea state or visibility. The object taxonomy remains coarse at five categories, even though fine-grained descriptions exist textually. The benchmark is also built from real images, while augmentation is synthetic, so a synthetic–real domain gap may persist for extremely rare conditions not present in MGD itself (Guo et al., 25 Sep 2025). The paper points toward extensions with continuous or hierarchical attributes, deeper use of fine-grained textual object descriptions, and additional modalities such as thermal or radar as plausible next steps in maritime generative benchmarking.