OceanVive: Marine Monitoring & Immersive Systems

• OceanVive is a term for diverse ocean computing systems that combine vision-language models and retrieval techniques to enhance marine monitoring.
• The visualization variant transforms large-scale ocean datasets into interactive, navigable spatial narratives using advanced rendering and data slicing.
• The VR platform leverages Unreal Engine, spline-based motion, and multisensory feedback to deliver immersive undersea exploration experiences.

OceanVive is a name applied in recent arXiv literature to multiple ocean-focused computational systems. One line of work presents OceanVive as “a proposed end-to-end framework for scalable, resilient ocean monitoring and conservation” built on “Bottom-up, open-domain Vision-LLMs (VLMs) such as CLIP and LLaVA” together with “Retrieval-Augmented Generation (RAG) to ground visual inferences in external, domain-specific knowledge” (Dyanatkar et al., 2024). A second line of work presents OceanVive as “an immersive and interactive visualization system that transforms complex ocean datasets into navigable spatial narratives” by combining “an exploratory panel on a table-sized tablet” with “a large screen” (Ouyang et al., 23 Jul 2025). A related VR blueprint, distilled from the Deepsea prototype, states that by combining a modular Unreal-based architecture, a robust asset pipeline, Catmull–Rom spline paths, controller mappings, multisensory feedback, and a 50-user study, “one arrives at OceanVive, a comprehensive undersea VR platform” (Li et al., 2023).

1. Research scope and naming

Recent usage does not identify a single canonical OceanVive implementation. Instead, the name denotes distinct systems for ocean monitoring, immersive scientific visualization, and undersea virtual exploration. A plausible implication is that OceanVive functions less as a single software artifact than as a recurring label for ocean-centered computational environments spanning CV, VLM-RAG, scientific visualization, and VR.

Paper	OceanVive formulation	Technical substrate
(Dyanatkar et al., 2024)	“proposed end-to-end framework for scalable, resilient ocean monitoring and conservation”	frozen CLIP visual encoder, image-keyed vector store, LLaVA
(Ouyang et al., 23 Jul 2025)	“immersive and interactive visualization system” for “complex oceanic phenomena”	Unity3D, 180° curved panoramic display wall, table-sized multi-touch tablet
(Li et al., 2023)	“comprehensive undersea VR platform”	Unreal Engine 4/5, VR headset, tracked controllers, spline-based creature motion

The three usages share a common concern with marine data and marine environments, but they operationalize that concern differently. The monitoring-oriented OceanVive emphasizes open-domain recognition under long-tailed distributions and domain shifts; the visualization-oriented OceanVive emphasizes navigable spatial narratives for hypoxia and acidification; the VR-oriented OceanVive emphasizes immersion, rendering fidelity, motion modeling, and user experience.

2. OceanVive as open-domain vision-language retrieval for ocean monitoring

In the monitoring formulation, OceanVive begins by passing “each input image (or video frame) from a fishing vessel or underwater camera through a frozen CLIP visual encoder,” producing “a fixed-dimensional embedding $q$.” It then performs “a nearest-neighbor search in a prebuilt ‘image-keyed’ vector store,” retrieves “the top-$k$ reference items (each with its associated textual species description),” and concatenates those texts with “a natural-language prompt” for a pretrained multimodal LLM, specifically LLaVA, which “generate[s] a species label or answer any open-ended question about the scene” (Dyanatkar et al., 2024).

The schematic form is explicitly given as:

• Input image $x \rightarrow$ CLIP encoder $\rightarrow$ query embedding $q$
• $q \rightarrow$ similarity search over database embeddings $\{d_i\} \rightarrow$ retrieve texts $\{t_i\}$
• $$[\text{Prompt} \mid t_1 \mid \dots \mid t_k] \rightarrow$$ LLaVA generator $\rightarrow$ final answer $k$0

The component block diagram is also explicit. The “Image Encoder (CLIP ViT)” generates “image embedding $k$1 ($k$2).” The “Vector Store” indexes “database embeddings $k$3, each linked to species description text $k$4.” The “Retriever” is a “similarity search module (e.g., FAISS/HNSW) that ranks $k$5 by $k$6,” and the “Generator (LLaVA multimodal LLM)” consumes “prompt + retrieved texts to produce answer $k$7.”

Retrieval scoring is defined by cosine similarity:

$k$8

The retriever returns “the indices of the top-$k$9 embeddings $x \rightarrow$0,” and their associated texts are concatenated into context. Conditional generation is stated as

$x \rightarrow$1

A possible end-to-end fine-tuning objective is given as a “RAG-style cross-entropy”:

$x \rightarrow$2

where $x \rightarrow$3 denotes “the set of retrieved texts for $x \rightarrow$4.” In the preliminary version, however, the system does “not backpropagate through retrieval” and keeps “all pretrained weights frozen.”

The architectural motivation is framed as a contrast between bottom-up open-domain learning and top-down classification. “Traditional CNNs or vision transformers require retraining when new species or environments appear” and “struggle with long-tailed distributions (rare species) and domain shifts (different lighting, occlusion, vessel decks).” By contrast, OceanVive “leverages a universal visual backbone trained on billions of image–text pairs (CLIP),” “remains task-free at training time—no fine-tuning on marine classes,” and can handle “unseen or rare species by retrieving rich textual descriptions at inference.”

3. Retrieval corpus construction, inference behavior, and benchmark results

The retrieval database is built from “a small ‘reference set’ of labeled fish images (e.g. the Fishnet validation set).” For each reference image, the system computes “its CLIP embedding and store[s] both the embedding and a detailed text description (taxonomic metadata, morphological characteristics).” This makes retrieval operate on “image–image similarity” while returning “rich textual knowledge” (Dyanatkar et al., 2024).

Indexing details are specified directly: the “Embedding model” is “CLIP’s ViT-B/32 producing 512-dim vectors”; the “Index structure” is “FAISS with an HNSW (Hierarchical Navigable Small World) graph for sub-millisecond nearest-neighbor search”; and there is “No additional dimension reduction beyond CLIP’s native output.”

The sample inference sequence on vessel video frames is also concrete. A “low-resolution, partially occluded frame of a fish” is encoded into $x \rightarrow$5 and “matched against $x \rightarrow$6 reference embeddings.” The “top-3 retrieved texts” may describe “torpedo-shaped body, metallic blue color…” for tuna and “dolphinfish with bright green-yellow hue…” for mahi mahi. LLaVA then receives the prompt “Q: What is the species of the fish?” plus retrieved context and emits “The fish in the image is a tuna…”. The paper identifies this behavior as “emergent retrieval and prediction capabilities without domain-specific training or knowledge of the task itself.”

The zero-shot interpretation is explicit. Because CLIP and LLaVA “were never explicitly trained on this specific fish dataset,” correct identification of “bigeye tuna” rather than “an erroneous mahi mahi without retrieval” is taken as evidence of “an emergent ability to generalize across unseen classes simply by grounding on external descriptions.”

Quantitative results are reported on Fishnet benchmarks:

Method	Reported performance
InceptionV3 baseline (ImageNet-pretrained)	Top-1=75.01%, Top-2=83.12%, Top-3=94.08%
OceanVive VLM-RAG Final Prediction	Top-1=84.03%
VLM-RAG Retrieval Step Alone	Top-1=86.84%, Top-2=95.27%, Top-3=97.81%

The reported failure modes are likewise specific. Failure occurs “when retrieved texts misalign (e.g. retrieval nearest neighbor is a similar-looking but incorrect species), or when the input embedding is too noisy due to extreme occlusion.” Conversely, “Rare or unseen species (e.g., certain billfish) often appear correctly in the Top-k retrieval texts,” providing the knowledge needed for classification.

4. OceanVive as immersive visualization architecture for oceanic phenomena

In the visualization formulation, OceanVive targets “communicating the complexity of oceanic phenomena-such as hypoxia and acidification” by transforming “complex ocean datasets into navigable spatial narratives.” The hardware comprises an “Immersive Viewport” implemented as “a 180° curved, high-resolution panoramic display wall (4 K resolution per panel), designed for shared audiences,” and an “Exploratory Panel” implemented as “a table-sized multi-touch tablet (≈27" diagonal)” that acts as “a personal ‘cockpit’ for data configuration, slicing controls, target designation, and storyboarding.” “No head-mounted displays are required” (Ouyang et al., 23 Jul 2025).

The software architecture is modular. “Unity3D (C#) hosts both the Immersive Viewport and Exploratory Panel as synchronized scenes.” A “Data Manager” loads, preprocesses, and caches ocean-grid assets; a “Visualization Module” implements “GPU-accelerated volume rendering, dynamic slicing, and charting components”; an “Interaction Controller” maps tablet inputs to “camera poses, slicing parameters, and target commands”; a “Narrative Manager” records “target” events and drives guided playback; and a “Recommendation Engine” computes “points of interest (e.g., maxima/minima, spatial/temporal gradients) on demand.”

The workflow is explicitly ordered as:

1. Data Configuration
2. 3D Miniature Overview
3. Volumetric Slicing & Navigation
4. Analytical Target Designation
5. Charting & Recording
6. Immersive Story Playback

The underlying data pipeline is also specified. The raw data consist of “Four daily oceanographic variables (temperature $x \rightarrow$7, salinity $x \rightarrow$8, dissolved oxygen $x \rightarrow$9, nitrate $\rightarrow$0) over a 5-year span, sampled on a 400×441 grid at 90 depth layers,” together with “A binary mask” encoding “coastal bathymetry (surface → –90 m).” Preprocessing includes “Bathymetry mesh generation from mask (marching cubes)” and per-variable normalization:

$\rightarrow$1

where $\rightarrow$2 are “expert-provided clamping bounds.”

Volume rendering uses trilinear interpolation at arbitrary sample position $\rightarrow$3:

$\rightarrow$4

with “the standard barycentric weights.” The transfer function is defined as a mapping $\rightarrow$5 and implemented as “a piecewise linear colormap (e.g., blue → green → yellow → red) with alpha tuned for translucency”:

$\rightarrow$6

Each day’s “4D block” is saved “as a Unity 3D texture array, with metadata for temporal interpolation.”

5. Adaptive encodings, spatial narrative, and expert evaluation

Adaptive visual encoding is central to the visualization-oriented OceanVive. Users can switch among “expert-validated palettes,” and “Dynamic Slicing” is parameterized by a plane $\rightarrow$7 with “a normal $\rightarrow$8 and depth range $\rightarrow$9.” “All samples satisfying $q$0 are rendered with 2D color mapping,” and slider adjustments update the plane instantly (Ouyang et al., 23 Jul 2025).

Three analytical chart types are defined. A “Spatial Point → Bar chart” uses $q$1 over time. A “Spatial Path → Line chart” uses $q$2 along sampled polyline points $q$3. A “Volumetric Area → Box plot” summarizes the distribution of $q$4 within “cylindrical volume $q$5.” Adaptation rules include real-time updates to “the transfer function’s $q$6 sliders” when the “POI selection changes,” and a threshold-triggered highlight rule for critical events:

$q$7

One cited example is “$q$8 mg/L.”

The narrative model is spatial rather than purely temporal. A “story segment” is “a recorded observational target (point, path, or volume) with associated camera pose and chart snapshot.” The “Storyboard” is “a linear list on the tablet” whose thumbnails can be reordered or annotated. During playback, the viewport moves “along a smooth spline through $q$9,” where

$q \rightarrow$0

and $q \rightarrow$1 with $q \rightarrow$2 the chart data.

The exploratory panel exposes the visualization primitives directly: “Dropdowns for dataset, date range, variables”; a “Rotatable/pannable 3D model of the bay”; “Sliders for plane orientation and depth bounds”; “Toggle buttons for Point/Path/Area POI selection plus ‘Apply Recommendation’ menu (Max/Min/Gradient)”; “Buttons to add current target to Record List or compare multiple records side by side”; and a “master time slider” for animated playback. Interaction metaphors are stated verbatim as “Drag to slice,” “tap to place a point,” “draw to trace a path,” and “pinch to zoom.” Synchronization is implemented through “Unity’s networking layer,” which “replicates scene-graph state.”

Validation was conducted with “Eight oceanographers (five in the formative study, three new in the evaluation) with 5+ years of domain experience.” The procedure was “15 min tutorial → 15–20 min free exploration with a sample dataset → 15–20 min semi-structured interview.” Analysis used thematic encoding into “Tool Perception, Workflow Adaptation, Immersive Experience Concerns.” The reported findings were that “All experts praised the intuitive spatial narratives and real-time responsiveness,” that the pipeline was seen as “lowering the barrier for non-coding experts,” and that experts raised warnings about “cognitive overload” and “potential misinterpretation of abstract visuals.” A few “still valued switching back to static 2D plots for clarity.”

6. OceanVive as a VR undersea-exploration platform

The VR-oriented OceanVive is described as “an end-to-end blueprint for building and evaluating ‘OceanVive,’ distilled from the Deepsea prototype.” Its hardware includes a “High-performance PC workstation,” a “Dedicated GPU,” a “VR headset (HTC Vive Pro or Oculus Rift S/Quest 2 with link cable),” “Two tracked handheld controllers,” “Room-scale tracking,” and optional “haptic feedback devices (vest, gloves).” Its software stack includes “Unreal Engine 4/5 (C++/Blueprints),” “SteamVR or OpenXR,” the “Vive Input Utility (VIU) plug-in,” “HTTP/WebSocket or custom TCP streaming service,” Maya, Substance Painter, FBX export, Wwise or Unreal’s native audio engine, and Unreal UMG (Li et al., 2023).

The real-time control flow is also explicit: “detect VR hardware, calibrate headset and controllers, initialize Unreal VR camera rig”; then “UI menu driven by laser-pointer raycasts”; then asynchronous asset streaming; then a “Real-time rendering & interaction loop (90 Hz)” that reads controller poses, updates the player pawn, updates creature animations via spline paths, processes UI raycasts and collision events, and submits frames to the VR compositor.

Rendering and asset preparation follow a standard but detailed pipeline. The system uses “Reference gathering: underwater photographs of coral reefs, rock formations, sediment profiles for Indonesia and Pacific scenes”; “Low/medium/high-poly modeling in Maya → UV unwrapping → FBX export”; and “Physically based textures in Substance Painter.” Level-of-detail management uses “Three LOD tiers per mesh,” “Unreal’s hierarchical LOD (HLOD) system,” and “Runtime culling: frustum and occlusion queries.” Lighting and shading include “Underwater ambient light: blue-green tint, exponential fog with depth attenuation,” “God-rays,” “Dynamic point lights on bioluminescent creatures and interactive UI panels,” “subsurface scattering for corals,” “depth-based color shift,” and “Screen-space reflections/refractions on helmet visor or diving mask.”

Marine-creature motion is modeled using an “Optimized Catmull–Rom Spline.” Control points are defined as $q \rightarrow$3 in 3D with a per-segment speed parameter. For $q \rightarrow$4, the path is

$q \rightarrow$5

with boundary conditions

$q \rightarrow$6

and

$q \rightarrow$7

where $q \rightarrow$8 is the tension parameter and “$q \rightarrow$9” is given “for uniform Catmull–Rom.” The stated benefits are that the curve “passes exactly through $\{d_i\} \rightarrow$0 and $\{d_i\} \rightarrow$1” and yields “Smooth tangents at segment boundaries → fluid, natural swimming paths.” Implementation guidance specifies precomputing the coefficient matrix per segment, evaluating $\{d_i\} \rightarrow$2 on GPU or CPU, orienting creature meshes “by Frenet frame or by billboarded tangent vector,” and adapting $\{d_i\} \rightarrow$3 based on a per-control-point speed field.

Interaction design uses asymmetric controller roles: “Left controller: locomotion (swim forward/back, turn) via thumbstick + tilt gestures”; “Right controller: laser-pointer raycast to select UI widgets and trigger creature info panels”; and “Grip buttons: scale movement up/down (‘ascend/descend’).” Collision avoidance uses a “Simple capsule collider around player pawn” and “Creature colliders” that “trigger informational popups on first contact, but creatures swim away to avoid overlap.” Feedback includes “spatialized sounds,” “short controller vibrations,” “semi-transparent panels,” and an “Energy bar HUD” with the warning “Energy Low, Please Surface” after “5 minutes continuous swim.”

The user study included “N=50 healthy adults (22 M, 28 F), mean age 25.2 yrs” with stated exclusions. Tasks required navigation along “a predefined undersea route (six keypoints)” in “both Indonesia and Pacific scenes,” while answering “pop-quiz UI questions via raycast” and inspecting “three target creatures.” Metrics recorded were “time to complete route (mean=213.4 s, SD≈18 s), number of collisions (mean=2.3, SD≈1.1), UI-raycast accuracy (mean=75%).” Questionnaire scores were “Enjoyment … mean=20.7” and “Engagement/Presence … mean=18.9.” Key findings included “Strong positive correlation enjoyment↔engagement (Pearson r=0.73, p<0.001),” negative correlations between engagement and task time/collisions, positive correlation with ray-cast accuracy, and reports that participants found the controls “learnable though naturalness can improve with more haptic cues.”

7. Limitations, misconceptions, and future trajectories

The current literature already identifies several limitations. In the monitoring-oriented OceanVive, the preliminary implementation keeps “all weights in CLIP and LLaVA” frozen, applies “No LoRA adapters or instruction-tuning,” and exhibits failure “when retrieved texts misalign” or when the “input embedding is too noisy due to extreme occlusion” (Dyanatkar et al., 2024). In the visualization-oriented OceanVive, limitations include a “Steep learning curve for novices,” the fact that “reliance on a flat-screen immersive setup limits portability,” and the absence of “broad user study beyond oceanographers,” so “public comprehension remains to be measured” (Ouyang et al., 23 Jul 2025). In the VR-oriented OceanVive blueprint, participants judged the controls learnable, but “naturalness can improve with more haptic cues” (Li et al., 2023).

Recent work also suggests that the name OceanVive can be a source of conceptual ambiguity. A common misconception is to treat OceanVive as a single integrated platform. The publications instead apply the name to distinct systems: one for “classifying fish from video onboard fishing vessels,” one for “communicating complex oceanic phenomena,” and one for “undersea exploration.” This suggests a family resemblance rather than a unified codebase.

Future directions are concrete but domain-specific. For monitoring, proposed directions include “larger CLIP variants,” “custom finetuning with LoRA,” “multi-query search over different image crops,” “prompt optimization,” “re-ranking of retrieved results,” “dynamic context window sizing,” “Hierarchical Reasoning,” and “multi-modal keys (combining text, acoustic, depth cues)” (Dyanatkar et al., 2024). For immersive visualization, proposed extensions include “Integration of head-mounted AR for mobile outreach,” “Template-based story modes,” “isosurface extraction for pH fronts,” and “animated vector fields for current visualization” (Ouyang et al., 23 Jul 2025). For the VR blueprint, future extensions “might include adaptive AI-driven creature behaviors, multi-user synchronous exploration via networked replication, and advanced bio-feedback integration (heart‐rate, gaze)” (Li et al., 2023).

Taken together, these systems position OceanVive at the intersection of open-domain perception, scientific visualization, and immersive media. The monitoring variant treats external textual knowledge as a mechanism for species recognition under long-tailed and shifted conditions; the visualization variant turns multiyear ocean grids into interactive spatial narratives; the VR variant formalizes a full-stack undersea simulation pipeline with empirically studied interaction. A plausible implication is that future research could connect these strands more tightly, but the present literature documents them as separate instantiations under a shared name.