CoralVQA: Visual Q&A for Coral Reefs
- CoralVQA is a domain-specific VQA dataset focused on coral reef health and ecological monitoring, featuring over 12,800 underwater images.
- It integrates taxonomic rigor across 67 coral genera with 277,653 QA pairs, spanning 16 dimensions to capture both visual and ecological attributes.
- Benchmark evaluations reveal strengths and challenges in LVLM performance, emphasizing issues in domain adaptation and fine-grained numeric reasoning.
Searching arXiv for CoralVQA and closely related entries to ground the article in current papers. CoralVQA is a large-scale, domain-specific visual question answering dataset for coral reef image understanding, introduced as the first large-scale VQA dataset dedicated to coral reef analysis. It contains 12,805 real-world underwater images, 67 coral genera from 20 families, and 277,653 question–answer pairs spanning 16 question dimensions that jointly target basic visual perception and ecological or health-related interpretation. The dataset is explicitly oriented toward ecological monitoring and conservation, and it is coupled with a semi-automatic, expert-in-the-loop construction pipeline and benchmark evaluations of several large vision–LLMs (LVLMs), which expose both current capabilities and important failure modes in specialized marine imagery (Han et al., 14 Jul 2025).
1. Scope, motivation, and defining characteristics
CoralVQA is framed around a practical bottleneck in coral reef monitoring: reef images are abundant, but their interpretation requires specialized marine biology expertise. The dataset addresses this by treating coral reef analysis as a VQA problem in which a system answers natural-language questions about taxonomic identity, morphology, spatial arrangement, bleaching, disease, algal competition, structural integrity, and habitat conditions. Its stated purpose is not generic object recognition, but support for ecological monitoring and conservation workflows (Han et al., 14 Jul 2025).
The dataset’s scope is unusually broad for coral imagery. It comprises 12,805 real-world underwater images collected from three oceans—Atlantic, Indian, and Pacific—and multiple marine regions, including the Island of Moorea, ATL, IND_CHA, IND_MDV, PAC_USA, PAC_IDN_PHL, PAC_SLB, PAC_TWN, and PAC_TLS. The coral labels are re-annotated at genus level under Linnaean taxonomy, and the resulting benchmark spans 67 genera from 20 families. This combination of taxonomic specificity, cross-region coverage, and question answering is the central feature that distinguishes CoralVQA from earlier coral datasets, which were primarily organized around classification or segmentation and did not provide multidimensional ecological QA (Han et al., 14 Jul 2025).
Its motivation is also methodological. Existing VQA resources are generic or focused on other specialized domains, whereas coral monitoring requires questions such as genus identification, location-specific bleaching detection, algal overgrowth assessment, and counting diseased genera. CoralVQA encodes these demands directly, thereby positioning coral reef understanding as a benchmark for vision-language reasoning under domain-specific visual ambiguity, subtle morphology, underwater image degradation, and regional distribution shift.
2. Dataset composition and annotation schema
CoralVQA contains 277,653 QA pairs, averaging 21.6 questions per image. Questions have a mean length of about 14.4 words and are organized into 16 dimensions divided evenly between basic visual attributes and ecological or health-related attributes. Open-ended questions account for 42.1% of all questions, corresponding to 116,960 QA pairs, while the remaining questions are closed yes/no items, with 73,095 “yes” answers and 87,598 “no” answers (Han et al., 14 Jul 2025).
The images have high resolution on average, approximately pixels, and are sourced from XL Catlin Seaview Survey, the Moorea Labeled Corals dataset, and Corals of the World. The annotation schema is taxonomically restrictive: all coral instances are re-annotated at genus level, non-standard categories such as “branching Acroporidae” are removed, and non-coral classes such as sand, fishing gear, and sediments are deleted. The final dataset includes 97,395 coral-related annotations, while 50,200 non-coral or non-standard labels are removed during cleaning (Han et al., 14 Jul 2025).
| Group | Dimension | Focus |
|---|---|---|
| Basic visual | Category | Coral genus at a given region |
| Basic visual | Color | Coral color at a specific location |
| Basic visual | Presence | Presence of coral or a specific genus |
| Basic visual | Position | Location of a specified coral |
| Basic visual | Quantity | Number of coral genera in the image |
| Basic visual | Size | Dominant or smallest coral genus |
| Basic visual | Shape | Morphology such as branching or massive |
| Basic visual | Scene type | Benthic habitat type |
| Ecological/health | Coral growth condition | Whether the coral is actively growing |
| Ecological/health | Algal presence | Algae surrounding or overgrowing coral |
| Ecological/health | Symbiotic relationship | Stable symbiosis with zooxanthellae |
| Ecological/health | Health status existence | Bleaching or disease at a location |
| Ecological/health | Structural integrity | Skeleton intact or damaged |
| Ecological/health | Health status quantity | Count of genera showing bleaching or disease |
| Ecological/health | Bleaching susceptibility | Whether the genus is susceptible |
| Ecological/health | Habitat environment | Water clarity, debris, contaminants |
The annotation ontology is hybrid. Basic visual attributes such as genus labels, approximate positions, and counts are extracted from cleaned annotation files using scripts, while ecological and health attributes are manually annotated under the guidance of marine biologists. These structured labels are then supplied to GPT-4o during question generation so that the generated QA pairs reflect domain knowledge rather than generic VQA templates. This suggests that CoralVQA is not only a collection of images and questions, but also a curated interface between biological ontology and multimodal language supervision.
3. Semi-automatic data construction pipeline
CoralVQA is built through a six-stage, semi-automatic, expert-supervised pipeline designed to balance scalability and professional-grade data quality. The initial collection stage aggregates 16,862 images from MLC, XL Catlin Seaview Survey, and Corals of the World, and then filters them using the underwater image quality metric UICQE from Yang and Sowmya (2015), leaving 12,805 images in the final set (Han et al., 14 Jul 2025).
Label cleaning and re-annotation form the second stage. Marine biology experts, led by the fourth author, remove inconsistent or non-biological labels and re-annotate coral instances at genus level according to biological taxonomy. The third stage extracts textual attributes: scripts derive basic visual properties from cleaned annotations, while domain experts manually annotate bleaching or disease presence, algal cover, growth status, structural integrity, symbiosis, and environment quality (Han et al., 14 Jul 2025).
Prompt engineering and QA generation constitute the fourth and fifth stages. Prompt templates for GPT-4o provide the image, structured attributes, desired question dimensions, and language style. The system uses GPT-4o’s multimodal input capability through the image API. Spatial referencing is standardized by dividing each image into a grid and assigning coral instances to descriptors such as “upper left,” “center,” or “lower right” based on coordinates. Each question type is asked at most 3 times per image, the generation temperature is set to 0.3, prompts require definitive answers rather than “unknown” or “uncertain,” and GPT-4o is also used for paraphrasing to increase linguistic diversity (Han et al., 14 Jul 2025).
The final stage is human verification. Twelve annotators with marine science backgrounds manually review and correct hallucinated or irrelevant questions, incorrect spatial references, and inaccurate answers. Each QA set is then double-checked by a second annotator, and any subset with accuracy below 95% is re-verified. Marine biologists additionally sample 10% of each annotator’s work, and if sampled accuracy is below 95%, that portion is fully revalidated. The result is a dataset that is both large-scale and scientifically credible, with explicit quality-control thresholds embedded into the construction process (Han et al., 14 Jul 2025).
4. Task design, reasoning demands, and evaluation protocol
CoralVQA is designed to probe a wide spectrum of reasoning skills. At the lower end are color recognition, morphology description, presence detection, position reporting, and simple counting. At the higher end are fine-grained genus recognition across 67 genera, ecological reasoning about algal competition and growth state, health assessment involving bleaching, disease, and structural damage, and environmental reasoning about water clarity, debris, or contaminants. The dataset therefore operationalizes coral reef understanding as a layered multimodal reasoning problem rather than a single-label classification task (Han et al., 14 Jul 2025).
The benchmark uses three non-overlapping subsets: a train split with 10,537 images and 226,726 QA pairs, a test split with 1,274 images and 27,984 QA pairs, and a cross-region split with 994 images and 22,943 QA pairs drawn from PAC_USA as a Hawaii-like region. Models are fine-tuned on the train split and evaluated on the standard test set, the cross-region set, and a Bleaching-Coverage sub-dataset focused on bleaching coverage estimation. The cross-region split is specifically intended to test generalization under real-world domain shift (Han et al., 14 Jul 2025).
For the main VQA tasks, the reported metric is accuracy, defined as the proportion of questions correctly answered. Because many answers are open-ended and may involve synonyms, GPT-4o is used as an automatic judge to determine semantic equivalence between model answers and ground truth, in a setup described as similar in spirit to LLM-as-a-judge evaluation such as MT-Bench. For bleaching coverage estimation, ground truth is computed by segmenting bleached regions with LabelMe and calculating the ratio of bleached coral area to total surveyed area; performance is then measured by MAE and MASE (Han et al., 14 Jul 2025).
The task design also induces an empirical notion of difficulty. Binary ecological questions such as bleaching presence or skeleton integrity are reported as easier, whereas fine-grained category recognition, quantity estimation, shape questions, and bleaching coverage estimation are harder. This suggests that CoralVQA is not only a dataset but also a structured stress test for the interaction among taxonomy, spatial grounding, ecological interpretation, and numeric reasoning.
5. Benchmarks, empirical findings, and failure modes
Four representative LVLMs are benchmarked on CoralVQA: Mini-Gemini (7B), Qwen2.5-VL (7B), BLIP3/xGen-MM, and InternVL2.5 (8B). All are initialized with their official pre-trained weights and fine-tuned on the CoralVQA training split. The benchmark does not propose a new architecture or loss function; its purpose is to use CoralVQA as a testbed for systematic evaluation (Han et al., 14 Jul 2025).
On the standard test set, InternVL2.5 is the strongest reported model, achieving 71.35% overall accuracy on the basic visual group and 81.42% on the ecological and health group. Qwen2.5-VL reaches 58.60% and 72.39%, BLIP3 reaches 50.37% and 80.32%, and Mini-Gemini is substantially lower at 12.76% and 34.04%. Closed-ended ecological questions are generally easier than open-ended visual ones. For InternVL2.5, growth reaches 86.96%, health presence 88.73%, structural integrity 95.90%, symbiosis 88.81%, and environment 80.20%, whereas open-ended tasks remain much harder: category 81.69%, quantity 35.54%, shape 62.85%, and position 62.18%. The reported gap between open-ended and closed-ended questions exceeds 10% in many cases (Han et al., 14 Jul 2025).
Cross-region generalization is markedly weak. On the PAC_USA cross-region set, InternVL2.5 drops from 71.35% to 18.75% on visual “All” and from 81.42% to 28.25% on ecological “All.” Qwen2.5-VL falls to 16.95% and 25.70%, respectively, and similar declines are reported for BLIP3 and Mini-Gemini. InternVL2.5’s visual quantity score falls to 2.14%. The interpretation given is that regional variation in species composition, color morphs, and environmental conditions produces severe domain shift, even within the same ocean. CoralVQA is therefore valuable partly because it makes that domain shift visible rather than suppressing it (Han et al., 14 Jul 2025).
Bleaching coverage estimation remains especially difficult. On the Bleaching-Coverage dataset, Qwen2.5-VL reports MAE and MASE , while InternVL2.5 reports MAE and MASE . Mini-Gemini and BLIP3 are not reported for this task because they often generate irrelevant text instead of meaningful numeric answers. The reported error levels indicate that complex quantitative ecological reasoning remains unresolved for current LVLMs without explicit segmentation (Han et al., 14 Jul 2025).
The failure analysis is concrete. For “largest coral genus” questions, InternVL2.5 sometimes selects non-coral structures or algae as the largest coral, and for counting tasks it fails to localize distinct colonies and their genera. These observations are used to argue for tighter integration among segmentation, taxonomy, and VQA. A plausible implication is that generic multimodal pretraining remains insufficient when the target domain requires marine-specific visual discrimination and regionally robust ecological reasoning.
6. Relation to prior datasets, nomenclature, limitations, and applications
Relative to prior coral datasets, CoralVQA is distinguished by its combination of taxonomic rigor, QA structure, and ecological scope. The comparison set includes TasCPC with 1,258 images and 13 benthic categories, RSMAS with 766 patches and 8 genera, Benthoz15 with 9,874 images and 148 categories but only 2 coral genera following Linnaean taxonomy, EILAT with 1,123 patches and 4 morphological classes, ATCRC with 147 hyperspectral images and 6 genera, HSCR16K with 16,659 patches and 10 genera with textual knowledge, MLC with 2,055 images and 5 coral genera plus 4 non-coral classes, and CoralSCOP with 41,297 images focused on segmentation. None of these provide VQA or multidimensional ecological QA, whereas CoralVQA is explicitly designed for vision-language reasoning over coral ecology and health (Han et al., 14 Jul 2025).
CoralVQA also occupies a different niche from general VQA benchmarks and from other domain-specific VQA datasets. General VQA resources emphasize everyday scenes, while domain-specific VQA datasets in areas such as medical imaging and remote sensing do not cover marine ecology. A frequent source of confusion is the similarly named CORAL benchmark, which is not a VQA resource at all but a multi-turn, text-only conversational retrieval-augmented generation benchmark built from Wikipedia; it contains no visual inputs, images, or multimodal signals (Cheng et al., 2024). CoralVQA should therefore be understood as a distinct benchmark centered on reef imagery rather than conversational text retrieval.
The dataset’s limitations are explicitly acknowledged. Geographic coverage spans three oceans and multiple regions, but many coral regions remain absent. Underwater imagery retains variable lighting, turbidity, and color distortion despite quality filtering. Sixty-seven genera constitute a large scope relative to prior coral datasets, but not full global coral diversity, and rare genera may be underrepresented. Many questions are initially generated by GPT-4o, so stylistic or semantic patterns may not fully match practitioner questioning, and evaluation via GPT-4o as judge introduces dependence on another model’s judgments rather than direct human expert grading (Han et al., 14 Jul 2025).
The practical relevance of CoralVQA follows directly from its task design. It can support VQA-based assistants for reef surveys, rapid triage of large image collections, conservation decision support, educational tools for students and citizen scientists, and research on new LVLM architectures, domain adaptation, and knowledge-aware multimodal reasoning. The dataset, QA labels, and scripts are publicly released on HuggingFace, and the release is intended to be open to the research community, with code and documentation provided to reproduce experiments (Han et al., 14 Jul 2025).