FishDet-M: Underwater Fish Detection Benchmark

• FishDet-M is a comprehensive benchmark integrating 13 underwater datasets with COCO-style annotations for reliable fish detection.
• It evaluates 28 state-of-the-art detectors using detailed COCO metrics including mAP, AP50, and AP75 to measure performance.
• The framework introduces a CLIP-guided zero-shot model selection method that adaptively assigns optimal detectors for real-time applications.

FishDet-M is a unified large-scale benchmark specifically designed for robust fish detection in underwater imagery, addressing the challenges posed by fragmented data sources, heterogeneous acquisition conditions, and inconsistent evaluation standards. It integrates 13 publicly available datasets, standardizing annotation formats and protocols, and evaluates a diverse suite of state-of-the-art object detectors. FishDet-M also introduces a CLIP-guided, zero-shot model selection framework that adaptively selects the optimal detector per image using vision-language alignment, supporting both static benchmarking and adaptive, real-time deployment in complex aquatic environments (Abujabal et al., 23 Jul 2025).

1. Dataset Composition and Coverage

FishDet-M consolidates the largest diversity of underwater fish-detection datasets into a unified COCO-style benchmark. The selected sources span various habitat types—marine, brackish, occluded/debris, and aquarium settings—providing extensive coverage of practical imaging conditions. Key sources and statistics include:

Dataset	Images/Frames	Domain/Notes
Brackish-MOT	14,000 frames	Brackish, video; multi-object tracking
UIIS	4,628 images	Underwater instance segmentation
TrashCan 1.0	7,212 images	Debris + fish, cluttered scenes
Fish4Knowledge-2023	1,897 images	Marine, field videos
FishDataset	1,850 images	Aquarium, with masks
FishNet	94,532 images	Multi-species, ≈1,000,000 bboxes
DeepFish	3,820 images	Tropical Australia, masks

Summing across all sources, FishDet-M provides 105,556 images containing 296,885 fish instances, with splits for training, validation, and testing. Each annotation corresponds to a single, species-agnostic “Fish” class, facilitating cross-domain and cross-dataset evaluation.

2. Annotation Harmonization and Quality Control

All source datasets undergo rigorous conversion to the COCO object-detection and instance segmentation schema:

• Category simplification: All instances relabeled to a single “Fish” class.
• Bounding box encoding: Standardized to $$[x_\text{min}, y_\text{min}, \mathrm{width}, \mathrm{height}]$$ pixelwise.
• Segmentation masks: Provided as RLE or polygonal masks where available.
• Quality control: Correction of coordinate formats, removal of erroneous or duplicate annotations, manual review of out-of-bounds and ambiguous entries, and split harmonization.

Annotation sets strictly conform to COCO’s JSON schema, enhancing interoperability and reducing evaluation inconsistencies.

3. Evaluation Protocol and Performance Metrics

Detection results are assessed using COCO-style metrics, ensuring comparability with broader object detection literature. The key indicators are:

• Intersection-over-Union (IoU): $$\mathrm{IoU}(\hat{b}, b) = \frac{\mathrm{area}(\hat{b} \cap b)}{\mathrm{area}(\hat{b} \cup b)}$$.
• Average Precision at IoU threshold $\tau$:

$\mathrm{AP}(\tau) = \int_0^1 P(r;\tau) dr$

where $P(r;\tau)$ is precision at recall $r$, for predictions at IoU $\geq \tau$.

• Mean Average Precision (mAP): $$\mathrm{mAP} = \frac{1}{K} \sum_{k=1}^K \mathrm{AP}(\tau_k)$$ for thresholds $\{0.50, 0.55, ..., 0.95\}$.
• mAP@50, mAP@75: $\mathrm{AP}(0.50)$ and $$\mathrm{IoU}(\hat{b}, b) = \frac{\mathrm{area}(\hat{b} \cap b)}{\mathrm{area}(\hat{b} \cup b)}$$0.
• Scale-specific AP:
  • $$\mathrm{IoU}(\hat{b}, b) = \frac{\mathrm{area}(\hat{b} \cap b)}{\mathrm{area}(\hat{b} \cup b)}$$1 (small, area $$\mathrm{IoU}(\hat{b}, b) = \frac{\mathrm{area}(\hat{b} \cap b)}{\mathrm{area}(\hat{b} \cup b)}$$2 px)
  • $$\mathrm{IoU}(\hat{b}, b) = \frac{\mathrm{area}(\hat{b} \cap b)}{\mathrm{area}(\hat{b} \cup b)}$$3 (medium, $$\mathrm{IoU}(\hat{b}, b) = \frac{\mathrm{area}(\hat{b} \cap b)}{\mathrm{area}(\hat{b} \cup b)}$$4 area $$\mathrm{IoU}(\hat{b}, b) = \frac{\mathrm{area}(\hat{b} \cap b)}{\mathrm{area}(\hat{b} \cup b)}$$5 px)
  • $$\mathrm{IoU}(\hat{b}, b) = \frac{\mathrm{area}(\hat{b} \cap b)}{\mathrm{area}(\hat{b} \cup b)}$$6 (large, area $$\mathrm{IoU}(\hat{b}, b) = \frac{\mathrm{area}(\hat{b} \cap b)}{\mathrm{area}(\hat{b} \cup b)}$$7 px)

These metrics provide comprehensive insight into both detection performance and scale sensitivity.

4. Model Benchmarking and Inference Profiling

FishDet-M benchmarks 28 contemporary detectors, spanning the YOLOv8–YOLOv12 series, R-CNN derivatives, and transformer-based architectures (e.g. DETR, Sparse R-CNN). All models are trained from scratch and evaluated under identical conditions. Key performance and efficiency statistics:

Model	mAP	mAP₅₀	mAP₇₅	FPS	Params (M)
YOLOv8n	0.433	0.776	0.445	251	3.2
YOLOv8m	0.471	0.828	0.490	196	25.9
YOLO12l	0.487	0.847	0.516	73	26.4
YOLO12x	0.491	0.848	0.521	75	59.1
RetinaNet	0.448	0.764	0.465	38	34.0
FCOS	0.475	0.838	0.472	34	32.1
Faster R-CNN	0.379	0.690	0.374	29	41.0
DETR	0.390	0.708	0.390	28	41.0

YOLO12x achieves the highest accuracy ($$\mathrm{IoU}(\hat{b}, b) = \frac{\mathrm{area}(\hat{b} \cap b)}{\mathrm{area}(\hat{b} \cup b)}$$8), but with higher latency and parameter count. YOLOv8n/v8s deliver superior throughput ($$\mathrm{IoU}(\hat{b}, b) = \frac{\mathrm{area}(\hat{b} \cap b)}{\mathrm{area}(\hat{b} \cup b)}$$9 FPS) with moderate accuracy. Single-stage detectors (RetinaNet, FCOS) achieve balanced performance at moderate frame rates. Transformer-based methods demonstrate higher latency and underperform notably on small object regimes.

5. CLIP-Guided Zero-Shot Model Selection

FishDet-M introduces a CLIP-based model selection approach for adaptive, per-image detector assignment. This selector computes cosine similarity between the CLIP-encoded input image and CLIP text embeddings of model-specific prompts (e.g., “YOLOv12x model for fish detection”). The inference pseudocode:

$\tau$8

Here $\tau$0 uses L₂-normalized CLIP embeddings. No additional training is required. This zero-shot selection achieves $\tau$1, $\tau$2, $\tau$3, and $\tau$4, $\tau$5, $\tau$6, at $\tau$712.46 ms per image (80 FPS). The architecture outperforms several fixed detectors in both efficiency and mean accuracy (Abujabal et al., 23 Jul 2025).

6. Reproducibility and Resource Availability

Comprehensive resources are publicly available to ensure full reproducibility:

• Data: COCO-style annotation JSONs for train/val/test splits of all 13 unified datasets.
• Scripts: Data conversion, harmonization, and evaluation tools.
• Models: Pretrained weights for all 28 benchmarked detectors.
• Code: CLIP selection module, prompts, and standard evaluation pipelines (PyTorch, Ultralytics YOLO, MMDetection, SuperGradients).
• Setup: Environment and dependency specifications (e.g., RTX 4090 GPU, i9-14900K CPU, 64 GB RAM, Windows 11).
• Documentation: Step-by-step instructions for reproducing splits, launching model training, and running metric evaluations with pycocotools.

These assets support transparent comparison, rapid prototyping, and iteration, underpinning research into robust, real-time fish detection and adaptive deployment across aquatic domains.