---
title: Maritime Generation Dataset Benchmark
url: https://www.emergentmind.com/topics/maritime-generation-dataset
type: topic
---

# Maritime Generation Dataset Benchmark

The **Maritime Generation Dataset (MGD)** is a generation-oriented maritime benchmark introduced in “Neptune-X: Active X-to-Maritime Generation for Universal Maritime Object Detection” as “the first generation dataset for maritime scenarios” and “the first generative benchmark for maritime detection.” It contains **11,900 real maritime images** with annotations structured for both **multi-modality-conditioned image synthesis** and **maritime object detection**: water masks, object boxes and masks, multi-level textual descriptions, and attribute labels spanning viewpoint, location, imaging environment, and object category [2509.20745]. Within Neptune-X, MGD functions simultaneously as the training substrate for the **X-to-Maritime** generator and as the reference dataset used to filter, condition, and evaluate synthetic data for downstream maritime detection.

## 1. Definition and design rationale

MGD was created to address two deficits identified in existing maritime datasets: the scarcity of annotated maritime data and weak generalization across maritime attributes such as **object category, viewpoint, location, and imaging environment**. Existing datasets were characterized as detection-focused, costly to annotate, severely imbalanced across these attributes, and generally lacking labels suitable for **controllable image synthesis**, such as water masks and scene-level or object-level textual descriptions [2509.20745].

The dataset is therefore designed around the requirements of **generative maritime learning**. Each sample is intended to support three coupled functions: training a **layout- and text-conditioned diffusion model**, providing the attribute labels needed by **Attribute-correlated Active Sampling (AAS)**, and enabling unified evaluation of both image generation and maritime object detection. In that sense, MGD is not merely a collection of maritime images; it is an annotation regime aligned to the conditioning interfaces actually consumed by Neptune-X.

A plausible implication is that MGD formalizes maritime generation as a **multi-condition supervision problem** rather than a purely synthetic-data problem. The benchmark is built from real images, but its annotation schema is optimized for controllable synthesis, attribute-aware sampling, and downstream detector improvement, rather than for conventional single-task detection alone [2509.20745].

## 2. Corpus composition and annotation schema

MGD contains **11,900** real RGB maritime images. For training the generator, images are used as **512×512 crops/patches**, while raw resolutions vary according to source datasets. The official split is **3:1:1**, with validation and test combined for generation evaluation [2509.20745].

| Split | Images | Role |
|---|---:|---|
| Train | 7,140 | Generator and detector training |
| Val | 2,380 | Held-out evaluation |
| Test | 2,380 | Held-out evaluation |

Each sample includes five annotation layers. First is the RGB image itself. Second is a **water surface mask** $\mathcal{M}_w$, a binary segmentation mask of the water region, from which a bounding rectangle $P_w$ is derived. Third are **object annotations**: bounding boxes, 5-class labels, binary object masks $\mathcal{M}_o^i$, and textual object labels $L_o^i$. Fourth are **multi-level textual descriptions**, including an image-level description, a water-surface description, and object descriptions. Fifth are the discrete **attribute labels** used by AAS: viewpoint, location, imaging environment, and object category [2509.20745].

The five object categories are:

- **ship**: 72.44% of instances (29,313)
- **buoy**: 13.16% (5,326)
- **person**: 11.97% (4,843)
- **floating object**: 1.53% (618)
- **fixed object**: 0.90% (366)

The viewpoint attribute has **3 classes**: **shore-based** (50.77%, 6,042 images), **shipboard** (20.66%, 2,459), and **aerial** (28.56%, 3,399). The location attribute has **4 classes**: **sea** (48.98%, 5,829), **river** (46.48%, 5,531), **harbor** (2.37%, 282), and **lake** (2.17%, 258). The imaging environment attribute has **6 classes**: **sunny** (54.55%, 6,491), **cloudy** (23.48%, 2,794), **foggy** (10.29%, 1,225), **rainy** (4.33%, 515), **dawn/dusk** (4.90%, 583), and **night** (2.45%, 292) [2509.20745].

These distributions expose pronounced long-tail structure. Harbor and lake scenes are rare; night and rainy imagery are underrepresented; non-ship classes are sparse. The paper explicitly treats this imbalance not as noise in the benchmark design but as the empirical basis for targeted generation and active sample selection.

## 3. Construction and labeling workflow

MGD aggregates imagery from eight named maritime datasets plus additional captured data. The dataset composition is as follows [2509.20745]:

| Source | Images |
|---|---:|
| MaSTr1325 | 800 |
| USVInland | 1000 |
| MIT Sea Grant | 100 |
| SMD | 400 |
| Seaships | 1500 |
| Seagull | 2996 |
| Fvessel | 1500 |
| LaRS | 1973 |
| Other captured data | 1631 |

The “other captured data” includes surveillance, UAV, smartphone, and DSLR imagery. The full corpus spans **shore / ship / aerial viewpoints**, and the collection procedure explicitly sought diversity in **geographies, times, weather, and traffic** [2509.20745].

Annotation is semi-automatic and multi-stage. For geometric labels, **T-Rex2** proposes object bounding boxes and **SAM2** segments the water surface given point prompts; human annotators then verify and correct both boxes and water masks. For textual and attribute labels, **LLaVA-Next** generates image descriptions, water descriptions, and object descriptions using carefully designed prompts, after which human validation ensures correctness. Discrete labels for viewpoint, location, imaging environment, and category are then extracted from these descriptions [2509.20745].

This workflow is significant because it yields annotation layers that conventional maritime detection datasets generally do not provide in a harmonized form. MGD retains the realism of heterogeneous source imagery while imposing a common taxonomy, common attribute axes, and common conditioning modalities suitable for generation.

## 4. Role in X-to-Maritime and attribute-aware selection

Within Neptune-X, each MGD training sample provides an image $I$, a caption $\mathcal{C}$, object conditions, and a water condition. For an object $i$ with class label $L_o^i$ and coordinates $P_o^i$, the conditioning pathway is defined as [2509.20745]:

$$
\mathbf{e}_o^i = \Phi(P_o^i),\quad \mathbf{t}_o^i = \xi(L_o^i),\quad
\mathcal{C}_o^i = \mathrm{MLP}([\mathbf{e}_o^i;\mathbf{t}_o^i]).
$$

The water description is encoded analogously:

$$
\mathcal{C}_w = \mathrm{MLP}([\Phi(P_w);\xi(L_w)]).
$$

The generator is trained with a multi-condition denoising objective:

$$
\mathcal{L} = \mathbb{E}_{z,t,\epsilon} \Big[ \|\epsilon - g_\theta(z_t, t, \mathcal{C}, \{\mathcal{C}_o^i, \mathcal{M}_o^i\}_{i=1}^{O}, \{\mathcal{C}_w, \mathcal{M}_w\})\|_2^2 \Big].
$$

MGD is also the basis for **Attribute-correlated Training Difficulty Factors (ATDFs)**. For a matched ground-truth box $b$ and prediction $\hat{b}$ with confidence $\hat{p}$, the paper defines prediction accuracy as:

$$
\text{Acc}(b, \hat{b}) = \hat{p}^\gamma \cdot \mathrm{IoU}(b, \hat{b})^{1-\gamma},\quad \hat{p}, \mathrm{IoU} \in [0,1].
$$

For attribute $s$ at iteration $j$, the initial difficulty is:

$$
d_s^j = \frac{1}{N_s^j}\sum_{n=1}^{N_s^j} (1 - \text{Acc}_n).
$$

Using X-to-Maritime trained on MGD, the authors generate **about 100k synthetic images** conditioned on layouts and text derived and randomized from MGD labels. After filtering with a **ResNet classifier** for layout and **CLIP similarity** for semantics, AAS ranks candidates by difficulty and task relevance. The selected subset is then merged with the original MGD train split to fine-tune the detector [2509.20745].

A plausible implication is that MGD operationalizes a closed loop between benchmark design and synthetic augmentation: the same attribute axes that describe the real dataset also govern difficulty estimation, synthetic candidate generation, and selection of high-value synthetic samples.

## 5. Benchmarks and empirical performance

MGD defines the reference distribution for both generation and detection evaluation. For image generation, all experiments are evaluated against the held-out **MGD val+test split** using **FID**, **Classification Accuracy Score (CAS)**, and **YOLO Score**. CAS is obtained by training a **ResNet-101 classifier on MGD**, while YOLO Score is obtained by training **YOLOv10 on MGD** and reporting **mAP**, **$\mathrm{mAP}_{50}$**, and **$\mathrm{mAP}_{75}$** on generated images [2509.20745].

On this benchmark, **X-to-Maritime** trained on MGD achieves **FID 18.05**, **CAS 79.34**, and **YOLO Score 17.08 / 39.14 / 13.52** for **mAP / $\mathrm{mAP}_{50}$ / $\mathrm{mAP}_{75}$**, outperforming **LayoutDiff, GLIGEN, InstDiff,** and **RC-L2I** [2509.20745]. The benchmark is therefore not limited to human-perceptual realism; it explicitly measures whether synthetic imagery is useful for a maritime detector trained on MGD’s own distribution.

For downstream maritime object detection on the **MGD test split**, the paper evaluates **YOLOv10, YOLOv11, YOLOv12,** and **Grounding DINO**. The strongest reported gains come from augmenting MGD with AAS-selected synthetic samples. For **YOLOv10**, mAP improves from **39.99** to **43.62**, and $\mathrm{mAP}_{50}$ from **61.13** to **65.50**. For **YOLOv11**, mAP improves from **41.29** to **44.43**; for **YOLOv12**, from **39.06** to **42.91**. **Grounding DINO** improves from **65.03 / 86.12** to **68.04 / 89.86** in **mAP / $\mathrm{mAP}_{50}$** [2509.20745].

The attribute-wise analysis reported in Figure 8 shows that average detection accuracy increases by **13.77%**, while variance across attributes decreases by **6.39%**. This indicates that synthetic data targeted to the rare and difficult attribute combinations already encoded in MGD improves robustness under underrepresented scenarios such as open sea or extreme illumination [2509.20745].

## 6. Relation to adjacent maritime generation resources and limitations

MGD differs from earlier maritime datasets such as **MaSTr1325, USVInland, Seaships, Seagull,** and **LaRS**, which are described as primarily detection or segmentation datasets with limited or no water masks, no standardized multi-scale text descriptions, and incomplete or inconsistent labeling of viewpoint, location, and imaging environment [2509.20745]. Its novelty lies less in raw image acquisition than in harmonizing real imagery into a conditioning-ready benchmark for controllable generation.

At the same time, adjacent work shows that the phrase *maritime generation dataset* now covers several distinct dataset paradigms. **MariData** is a **7,000-image** domain-labeled corpus for **Day-to-Foggy**, **Day-to-Sunset**, and **Day-to-Night** unpaired translation, emphasizing structure-preserving atmospheric synthesis rather than layout-conditioned object generation [2606.03246]. **SafeSea** is a **69,694-image** semi-synthetic corpus built by replacing calm sea backgrounds with synthetic **Sea State 1–4** backgrounds while preserving real boats, targeting robustness of object detection under rough water conditions [2311.14764]. **AIMO** is a **66,626-image** synthetic classification dataset generated with diffusion models across **15 maritime classes** and **5 weather/illumination domains**, intended as a labeled source domain for unsupervised adaptation to real maritime imagery [2501.15503]. **MSC** supplies **video–text–segmentation** triplets for underwater marine video understanding and text-to-video evaluation rather than above-water navigation scenes [2508.04549]. This suggests that MGD represents one specific lineage within a broader family: real-image, attribute-rich, generation-oriented maritime benchmarks.

The limitations identified for MGD are structural rather than incidental. The dataset relies on a **fixed, discrete attribute set**—viewpoint, location, imaging environment, and object type—which may constrain finer-grained modeling such as continuous sea state or visibility. The object taxonomy remains **coarse at five categories**, even though fine-grained descriptions exist textually. The benchmark is also built from **real images**, while augmentation is synthetic, so a synthetic–real domain gap may persist for extremely rare conditions not present in MGD itself [2509.20745]. The paper points toward extensions with **continuous or hierarchical attributes**, deeper use of fine-grained textual object descriptions, and additional modalities such as **thermal** or **radar** as plausible next steps in maritime generative benchmarking.

Source: https://www.emergentmind.com/topics/maritime-generation-dataset