OceanCorpus: Unified Multimodal Ocean Data

• OceanCorpus is a unified multimodal oceanographic corpus that integrates sonar data, underwater imagery, marine science visuals, and technical texts.
• It employs a standardized CSV schema with rigorous quality control and semantic alignment via the Ocean Concept Knowledge Graph for reliable data integration.
• By pre-training ocean-aware foundation models on billions of tokens and diverse visual data, OceanCorpus drives enhanced performance in marine AI tasks.

Searching arXiv for papers directly relevant to OceanCorpus and closely related ocean data-corpus efforts. OceanCorpus denotes a unified multimodal oceanographic corpus. In the OceanPile framework, it is the component that integrates sonar data, underwater imagery, marine science visuals, and scientific text from diverse authoritative sources, and it was designed to unite heterogeneous streams into a single, open-access multimodal resource for ocean foundation models (Xue et al., 25 Apr 2026). Its stated primary use cases are pre-training ocean-aware foundation models on more than five billion tokens of marine text and bootstrapping cross-modal alignment between underwater imagery, sonar returns, and technical prose for downstream fine-tuning (Xue et al., 25 Apr 2026). In adjacent literature, the same term also appears as a target architecture or extension path for coastal scientific text corpora, deep-sea animal datasets, SAR benchmarks, and analysis-ready environmental data cubes (Delaunay et al., 2024, Patterson et al., 2024, Tuel et al., 12 Jan 2026, Kavouras et al., 2024).

1. Origins and motivating problem

OceanCorpus emerged from a diagnosis that ocean data are highly fragmented across disparate sources and inherently exhibit multi-modal, high-noise, and weakly labeled characteristics, lacking unified schemas and semantic alignment (Xue et al., 25 Apr 2026). The motivating contrast is with the recent success of general Multimodal LLMs, which, in this account, lack the domain-tailored data foundation required to reason about marine phenomena such as acoustic signatures of a wreck or chemical gradients driving plankton blooms (Xue et al., 25 Apr 2026).

Within OceanPile, OceanCorpus is one of three key components, alongside OceanInstruction and OceanBenchmark (Xue et al., 25 Apr 2026). OceanInstruction is described as a high-quality instruction dataset synthesized via a novel pipeline guided by a hierarchical Ocean Concept Knowledge Graph, while OceanBenchmark is a manually curated evaluation benchmark for rigorous assessment (Xue et al., 25 Apr 2026). OceanCorpus therefore serves as the base substrate from which both instruction tuning and evaluation are organized.

This configuration suggests a two-level conception of the term. At the narrow level, OceanCorpus is the specific multimodal collection released with OceanPile (Xue et al., 25 Apr 2026). At the broader level, “OceanCorpus” functions as a recurring design goal in marine AI: a standardized, integrated resource spanning text, imagery, acoustic sensing, and associated metadata (Delaunay et al., 2024, Patterson et al., 2024, Kavouras et al., 2024, Tuel et al., 12 Jan 2026).

2. Modalities, sources, and corpus coverage

OceanCorpus ingests five major data categories, each pre-processed into a unified internal format stored as CSV records linking text passages, image files, and metadata fields (Xue et al., 25 Apr 2026).

Category	Main content	Reported scale
Scientific Text	Textbooks and peer-reviewed articles	$\approx 300{,}000$ PDF documents; over 5 billion tokens
Marine-focused Web Pages	Educational portals, forums, news sites	Not numerically specified
Sonar Imagery	Three public multibeam and side-scan datasets	Tens of thousands of acoustic frames
Underwater Optical Imagery	WildFish, WildFish ++, SCoralDet, CoralVQA, and other benchmarks	On the order of $50{,}000+$ images
Field-Collected AUV Data	Synchronized sonar and optical data from the Zhoushan archipelago	Several thousand paired scenes

The scientific-text component includes authoritative oceanography textbooks spanning chemical, biological, geological, and physical subdisciplines, together with peer-reviewed articles harvested from ArXiv and Nature-portal journals, retaining LaTeX sources when available (Xue et al., 25 Apr 2026). Documents are converted through direct LaTeX or Markdown export and PDF-to-Markdown tools such as MinerU into structurally annotated plain text that preserves headings, figure and table captions, mathematical symbols, and domain-specific notations (Xue et al., 25 Apr 2026).

The web component consists of educational portals, specialized forums, and news sites recommended by experts (Xue et al., 25 Apr 2026). HTML parsers strip navigation menus and boilerplate, textual passages undergo length- and content-based filtering, and associated images are scored for relevance by MLLMs and retained only if they surpass a quality threshold (Xue et al., 25 Apr 2026).

The sonar component integrates three public multibeam and side-scan sonar datasets, with all bounding-box annotations normalized to a common $[x_1,y_1,x_2,y_2]$ coordinate scheme and synonymous class labels merged (Xue et al., 25 Apr 2026). During preprocessing, vision-LLMs generate localized descriptive text for annotated objects or scene-level captions when only image-level tags exist, producing paired acoustic images and natural-language descriptions (Xue et al., 25 Apr 2026).

The underwater optical component follows a parallel normalization process. High-resolution photographs from WildFish, WildFish ++, SCoralDet, CoralVQA, and other benchmarks are unified so that each RGB image carries both fine-grained instance labels and enriched textual descriptions (Xue et al., 25 Apr 2026). The field-collected AUV component adds thousands of realistically varied scenes from the Zhoushan archipelago, with synchronized sonar and optical cameras capturing natural lighting gradients and complex seabed textures (Xue et al., 25 Apr 2026).

Across all categories, OceanCorpus is reported to ensure balanced representation of the four main oceanographic disciplines and to distribute samples so as to avoid heavy long-tail bias in either text or vision classes (Xue et al., 25 Apr 2026).

3. Unified schema and semantic organization

Although the OceanPile paper does not lay out a formal metamodel in mathematical detail, OceanCorpus achieves multimodal integration by mapping every data element into a shared CSV schema (Xue et al., 25 Apr 2026). Each record carries a unique identifier, a modality tag, source provenance, raw and tokenized text fields for textual modalities, image file paths and normalized bounding-box coordinates for visual modalities, unified class labels, and, where available, spatio-temporal metadata such as latitude, longitude, and timestamp (Xue et al., 25 Apr 2026).

This shared representation is supplemented by a vision-language alignment function informally denoted as

$$f_{\mathrm{align}} : (\mathrm{Image}, \mathrm{Box}) \rightarrow \mathrm{Text},$$

where a pretrained VLM ingests a sub-image and returns a concise descriptive caption (Xue et al., 25 Apr 2026). Homogeneity of coordinate systems and vocabulary is verified through automated merging of synonyms and ablation of low-confidence labels, a design choice described as bridging the modality gap without the need for heavyweight ontologies (Xue et al., 25 Apr 2026).

Semantic organization is further informed by the Ocean Concept Knowledge Graph. The graph is described as

$G=(V,E), \qquad V=P\cup \hat S,$

where $P=\{P_k\}_{k=1}^{K}$ are expert-defined primary disciplines and each $\hat S_k$ is a refined set of subcategories under $P_k$ (Xue et al., 25 Apr 2026). Extraction is driven by GPT-4o via two staged mappings,

$$S_k=\mathcal{M}(D_{\text{text}}\mid P_k), \qquad \hat S_k=\mathcal{M}(S_k;\tau_f),$$

with $\tau_f$ filtering subcategories with occurrence counts below a threshold (Xue et al., 25 Apr 2026). Edges $50{,}000+$0 connect primary disciplines to subcategories, yielding a hierarchy that informs metadata tags on corpus records; the paper gives as examples textbook chapters and paper abstracts being mapped into a $50{,}000+$1 tuple (Xue et al., 25 Apr 2026).

A plausible implication is that OceanCorpus combines two complementary alignment regimes: geometric normalization for image-like modalities and hierarchical semantic tagging for textual and cross-modal organization. The former operates through box normalization and label merging; the latter through disciplined placement within the Ocean Concept Knowledge Graph (Xue et al., 25 Apr 2026).

4. Curation and quality control

OceanCorpus uses a multi-stage quality control process intended to ensure scientific validity and alignment across modalities (Xue et al., 25 Apr 2026). For text, structural conversion is followed by rule-based removal of headers and footers and an LLM-assisted semantic deduplication step; passages whose pairwise cosine similarity exceeds a dedup threshold are culled, with retention of the more information-rich example (Xue et al., 25 Apr 2026). For images, each candidate sonar or optical image is scored for clarity and domain relevance by a VLM, low-scoring images are discarded, and duplicate or near-duplicate visuals are removed through perceptual hashing (Xue et al., 25 Apr 2026). For annotations, all bounding boxes are normalized and merged labels are vetted against a controlled vocabulary (Xue et al., 25 Apr 2026).

The paper states a corpus-level vetting mechanism in which each automatic agent $50{,}000+$2 assigns a quality score $50{,}000+$3 to element $50{,}000+$4, and the aggregate score

$50{,}000+$5

must exceed a minimum quality threshold $50{,}000+$6 for inclusion (Xue et al., 25 Apr 2026). Low-scoring items are either automatically dropped or flagged for manual expert review (Xue et al., 25 Apr 2026).

Related ocean-data work highlights compatible but distinct QC traditions. In oceanographic profile quality control, CoTeDe recommends standardizing on the IOC 1–9 flag convention, embedding QC flags in NetCDF/CF metadata, fitting “good-data” PDFs regionally and seasonally, maintaining a small expert-labeled subset, and storing per-feature survival-function values alongside final flags (Castelão, 2015). In ocean SAR, OceanSAR-2 emphasizes quality filters that discard Sentinel-1 Wave Mode vignettes with processing flags or calibration errors and advocates physics-calibrated normalization using $50{,}000+$7 rather than raw DN, together with dynamic pruning and balanced sampling (Tuel et al., 12 Jan 2026). These recommendations are not presented as OceanCorpus internals, but they indicate the broader methodological environment in which multimodal ocean corpora are being engineered.

5. Role in foundation-model training

OceanCorpus was used to pre-train multimodal ocean foundation models on raw text and image-text pairs (Xue et al., 25 Apr 2026). For text, the stated objective is a causal next-token cross-entropy loss,

$50{,}000+$8

and for cross-modal alignment the paper uses a contrastive InfoNCE loss,

$50{,}000+$9

where $[x_1,y_1,x_2,y_2]$0 is cosine similarity and $[x_1,y_1,x_2,y_2]$1 is a temperature hyperparameter (Xue et al., 25 Apr 2026).

The training rationale is explicit: the corpus couples more than five billion tokens of marine text with tens of thousands of sonar and underwater photographs and richly annotated field data (Xue et al., 25 Apr 2026). The reported outcome is that the resulting model exhibits markedly improved performance on both purely textual ocean-science question answering and multimodal VQA tasks, as shown in OceanBenchmark (Xue et al., 25 Apr 2026).

This usage positions OceanCorpus differently from single-modality benchmark datasets. Its primary function is not only evaluation, nor only domain-specific fine-tuning, but large-scale pre-training and alignment across modalities. A plausible implication is that OceanCorpus occupies in marine AI the role that broad web-scale corpora occupy in general-domain multimodal modeling, while remaining constrained by domain-specific validation and controlled curation (Xue et al., 25 Apr 2026).

6. Relationship to adjacent datasets and open problems

The wider literature treats OceanCorpus both as a concrete released dataset and as a unifying direction for otherwise separate marine-data efforts.

Resource	Focus	Stated relation to OceanCorpus
CoastTerm	Coastal scientific literature	Future work proposes fusion with marine and oceanographic corpora to form a unified “OceanCorpus”
FathomVerse	Deep-sea benthic detection imagery	Provides recommendations for merging and enrichment in an OceanCorpus
Ocean-DC	Analysis-ready coastal EO data cube	Described as laying groundwork for a full-blown OceanCorpus
OceanSAR-2	SAR ocean benchmarks and SSL pre-training	Described as releasing a physically-consistent, dynamically curated “OceanCorpus”
CoTeDe	Oceanographic-data QC	Gives recommendations for “OceanCorpus” integration

CoastTerm introduces a specialized corpus of 2,491 sentences from 410 scientific abstracts for Automatic Term Extraction and Classification, with an ARDI-derived label set of Actor, Resource, Process, Quality, and Location, and its future work explicitly proposes fusion with marine and oceanographic corpora to form a unified “OceanCorpus” (Delaunay et al., 2024). This establishes the textual and terminology-extraction dimension of the concept.

FathomVerse contributes a distinct deep-sea visual regime: 3,843 images with 8,092 bounding boxes from 12 morphological groups recorded at Musicians Seamounts and Octopus Garden, together with a gamified consensus-building workflow in which final annotations retain only $[x_1,y_1,x_2,y_2]$2 tuples with at least three player votes from players whose personal $[x_1,y_1,x_2,y_2]$3 (Patterson et al., 2024). Its recommendations for OceanCorpus include a common annotation schema such as COCO JSON, shared metadata fields including latitude, longitude, depth, vehicle, camera specs, and illumination parameters, hierarchical taxonomy, expanded background classes, complementary imagery sources, environmental data streams such as eDNA, CTD profiles, and acoustic recordings, and public leaderboards and evaluation servers (Patterson et al., 2024).

Ocean-DC approaches the problem from Earth observation and data harmonization. It proposes a 4D hypercube with dimensions time, band or product, row, and column, implemented in Python using Xarray and Rasterio, and its summary explicitly states that it lays the groundwork for a full-blown OceanCorpus (Kavouras et al., 2024). The implication is that an OceanCorpus need not be restricted to document-image pairs; it may also encompass analysis-ready spatio-temporal cubes.

OceanSAR-2 contributes a standardized SAR benchmark ecosystem centered on Sentinel-1A Wave Mode data and uses “OceanCorpus” to describe a physically-consistent, dynamically curated SAR resource released with standardized benchmarks, open labels, code, and evaluation scripts (Tuel et al., 12 Jan 2026). Its emphasis on dynamic data curation, imbalance mitigation, and cross-task transfer complements the broader multimodal ambitions of OceanPile.

Related optical datasets also expose unresolved challenges. ReefNet aggregates 76 curated CoralNet sources plus an Al Wajh subset into approximately 925,000 genus-level hard-coral annotations mapped to WoRMS, but supervised performance drops sharply across domains and the reported domain gap reaches up to 41%, while zero-shot models remain low across the board, especially for rare and visually similar genera (Battach et al., 19 Oct 2025). This suggests that scale and taxonomic enrichment do not by themselves eliminate domain shift in marine visual corpora.

Taken together, these works indicate that OceanCorpus is both an implemented multimodal corpus and a broader research program. Its defining technical themes are schema unification, multimodal alignment, quality-controlled aggregation, and benchmarkability; its persistent challenges are fragmented provenance, weak labels, taxonomy harmonization, and severe cross-domain variation across ocean sensing modalities (Xue et al., 25 Apr 2026, Patterson et al., 2024, Delaunay et al., 2024, Battach et al., 19 Oct 2025).