Tuna Tuning Pipeline: Automated Fisheries Analytics

• The paper introduces the Tuna Tuning Pipeline, a robust vision workflow that achieves automated catch composition estimation with <5% MAE under real-world conditions.
• It details a comprehensive approach that integrates high-fidelity data collection, multi-model segmentation (YOLOv9, SAM 2, Mask R-CNN), and sophisticated tracking using ByteTrack and optical-flow analysis.
• The study demonstrates superior segmentation performance with YOLOv9 + SAM 2 (Recall = 0.85) and offers practical guidelines for hardware, software, and validation for EM-based fisheries monitoring.

The Tuna Tuning Pipeline is a data-centric, multi-stage computer vision workflow for quantifying and automating catch composition estimation in tropical tuna purse seine fisheries. Developed for large-scale video streams collected via electronic monitoring (EM) systems on industrial fishing vessels, Tuna Tuning integrates robust segmentation, tracking, and species classification tailored to the challenge of distinguishing closely related tuna species under realistic conditions. The pipeline combines state-of-the-art models (YOLOv9, SAM 2, RegNetX_400) with hierarchical ensemble design and rigorous validation on ground-truth data, representing a significant advance in standardized EM-based fisheries analytics (Lekunberri et al., 19 Nov 2025).

1. Data Acquisition, Annotation, and Preprocessing

The initial stage centers on high-fidelity data collection via “Artificial Fishing Operations” (AFOs) performed on board. Catch is sorted by species, loaded onto conveyor belts, and imaged using stereoscopic RGB-D cameras (ZED 2i/X) at 1920×1080 pixels and 30 FPS. Preprocessing routines include belt-region cropping, geometric perspective correction, and CLAHE-based contrast enhancement to mitigate variable illumination and geometry.

Manual annotation produces a comprehensive dataset: 1,395 frames yielding 24,078 fish masks, labeled at both fine (species: SKJ, BET, YFT, NOT_TARGET) and coarse (FISH, BET_OR_YFT) granularity. Annotation is performed with CVAT. Crucially, a ground-truth reference for testing is constructed using observer-identified monospecific batches, as well as expert image-only inter-annotator agreement studies (BET 42.9 ± 35.6 %, YFT 57.1 ± 35.6 %). This reveals substantial ambiguity even for taxonomic experts, necessitating robust model uncertainty estimation.

2. Segmentation Architectures and Comparative Evaluation

Segmentation is approached through three alternatives:

• Mask R-CNN: Standard instance segmentation (Faster R-CNN backbone plus spatial mask head) implemented via PyTorch’s “maskrcnn_resnet50_fpn.” Trained for 40 epochs, batch size 2, SGD optimizer (LR=0.01, momentum=0.95, weight_decay=5×10⁻⁴, Exponential scheduler γ=0.96).
• YOLOv9 + SAM 2: Bounding boxes are detected by YOLOv9 (trained for 700 epochs, batch size 2, SGD, LR=0.01, γ=0.96), then used as prompts for mask generation by SAM 2 (“prompt by box” mode, no further fine-tuning).
• DINOv2 + SAM 2 (zero-shot): DINOv2 encodes semantic similarity maps which then prompt SAM 2; this method is not suited for real-time inference (~10 min/frame), and is included as a proof-of-concept only.

Validation results (mean ± sd, CV-folded):

• Mask R-CNN: mAP = 0.66 ± 0.01, Recall = 0.69 ± 0.01
• YOLOv9 + SAM 2: mAP = 0.66 ± 0.03, Recall = 0.85 ± 0.03

YOLOv9 + SAM 2 demonstrates superior segmentation yield (68.5 ± 16.0 % of fish segmented) compared to Mask R-CNN (52.1 ± 11.7 %, p < 0.01). The mAP is computed as: $$\text{mAP} = \frac{1}{|C|}\sum_{c\in C}\mathrm{AP}_c,$$ recall as

$\text{Recall} = \frac{TP}{TP+FN}.$

3. Tracking and Spatio-Temporal Integration

Multi-object associations across video frames are managed using ByteTrack. All frame-level detections are linked via the Hungarian algorithm backed by Kalman filtering. Low-score bounding boxes are retained for association to reduce identity switches during partial occlusions. Temporal logic is incorporated by analyzing the conveyor belt’s movement through optical-flow: track maintenance is suspended when direction reverses or halts, preventing duplication of fish identifications on belt re-entry.

4. Hierarchical and Multiclass Species Classification

Fish identities, post-tracking, are classified using RegNetX_400-based convolutional neural networks:

• Standard Multiclass: A single RegNetX_400 predicts one of {BET, SKJ, YFT, NOT_TARGET} via four-way softmax. Training uses SGD (LR=0.01, momentum=0.95, weight_decay=5×10⁻⁴), StepLR (step_size=7, γ=0.1), in a 10×5 stratified CV repeated split (70%/20%/10% train/val/test).
• Hierarchical Cascade: Sequential binary classifiers (also RegNetX_400) are stacked: (1) TARGET vs NOT_TARGET, (2) SKJ vs (BET+YFT), (3) BET vs YFT. Each stage is trained independently under the same regimen, yielding finer discrimination for confounded classes.

For both paradigms, the per-fish label is assigned by averaging all frame-level softmax outputs associated with a track. Hierarchical strategies produce improved test-time generalization, particularly in BET vs YFT disambiguation, reflecting the class-imbalance and expert-level difficulty observed in ground-truth datasets.

5. Validation Against Observer Ground Truth and Metrics

Final model assessments are run end-to-end on held-out AFOs (21 sets; known composition). Segmented/classified proportions achieved 84.8%, a practical upper bound for automated EM-based reporting. Mean absolute error (MAE) per species (computed as

$$\text{MAE} = \frac{1}{N}\sum_{i=1}^N |y_i - \hat{y}_i|$$

) on the second trip was BET 2.1 ± 2.1%, SKJ 5.0 ± 4.7%, YFT 5.6 ± 8.0%, NOT_TARGET 5.4 ± 2.8%, with total MAE ≈ 4.5%. Use of global-shutter cameras is critical (MAE 4.5%) versus rolling-shutter (MAE 12.3%).

6. Implementation and Deployment Guidelines

Practical reproduction requires:

• Hardware: 2× Intel Xeon Silver 4509Y, NVIDIA H100 NVL (94 GB), 768 GB RAM.
• Software: Python 3, PyTorch, Torchvision, OpenCV, CVAT, SAM 2 SDK, DINOv2 weights.
• Model Training: Segment hyper-parameter tuning over epochs (Mask R-CNN: 30–60, YOLOv9: 500–900), batch sizes {2,4}; augmentation for brightness/contrast/saturation adjustment to match on-vessel lighting; rigorous repeated 5-fold CV.
• Data Preparation: Controlled AFOs with known composition, top-down global-shutter imaging, standardized imaging geometry and lighting.
• Validation: Cross-validation and testing on mixed-batch AFOs; Wilcoxon signed-rank tests to compare model vs manual observers.

Operational adaptations for new fleets or target species involve fine-tuning augmentation and classifier architectures, and expanding the AFO library.

7. Broader Significance, Uncertainty, and Outlook

This pipeline formally quantifies the limits of both AI and human image-based species discrimination under real-world conditions, revealing that even domain experts exhibit low consensus on species-level ID. The end-to-end design achieves reliable (<5% MAE) EM-based catch composition under standardized settings, a notable improvement for regulatory and scientific compliance in major tuna fisheries. Extensions such as leveraging RGB-D for biometric analysis (length/weight estimation), and adaptation to new fleets or regions, are facilitated by the modularity of the annotation, segmentation, tracking, and classification stages. The approach establishes a reproducible benchmark for fast, accurate, and explainable automated fisheries monitoring (Lekunberri et al., 19 Nov 2025).