AutoFish Dataset for Fine-Grained Fish Analysis

• AutoFish is a dataset for fine-grained fish analysis, featuring high-resolution images and detailed annotations for instance segmentation, specimen identity, and morphometric measurements.
• It supports sustainable fisheries management by enabling automated fish documentation and addressing challenges such as occlusion and viewpoint variation.
• The benchmark includes structured splits and rigorous calibration, providing tasks for segmentation, length regression, and re-identification with robust performance metrics.

Searching arXiv for AutoFish and related fish re-identification papers to ground the article in published work. AutoFish is a publicly available dataset and benchmark for fine-grained fish analysis, introduced to support automated fish documentation in settings relevant to sustainable fisheries management and overfishing mitigation. It was presented in “AutoFish: Dataset and Benchmark for Fine-grained Analysis of Fish” (Bengtson et al., 7 Jan 2025) as a collection of high-resolution RGB images of visually similar fish arranged on a white conveyor belt under controlled capture conditions, with annotations for instance segmentation, specimen identity, and length. Subsequent work extended its use to visual re-identification in electronic-monitoring scenarios, treating AutoFish as a testbed for identity persistence across occlusion and viewpoint variation (Thilakarathna et al., 9 Dec 2025).

1. Definition and scope

AutoFish is a dataset for fine-grained analysis of fish specimens in conveyor-belt imagery, with an emphasis on tasks that require instance-level rather than merely species-level discrimination. The dataset comprises 1,500 images of 454 individual fish and 18,160 total fish masks, with each specimen assigned a unique ID and a manual length measurement to the nearest 5 mm by a marine biologist (Bengtson et al., 7 Jan 2025). The species represented are cod (Gadus morhua), haddock (Melanogrammus aeglefinus), whiting (Merlangius merlangus), hake (Merluccius merluccius), horse mackerel (Trachurus trachurus), and an “Other” category grouping several less-common species (Bengtson et al., 7 Jan 2025).

The dataset was designed around repeated appearances of the same individuals across controlled arrangements. Fish are organized into groups of 7–24 individuals and imaged in three arrangement subsets per group: Set1, Set2, and All. Set1 and Set2 each contain half the fish in isolated, non-overlapping placement, while All contains all fish in deliberately touching and occluded configurations. Each fish appears exactly 40 times in total, with 20 images per side (Bengtson et al., 7 Jan 2025).

Later re-identification work describes AutoFish in terms of four condition categories derived from arrangement and lateral orientation: Separated-Initial, Separated-Flipped, Touched-Initial, and Touched-Flipped, with 10 images per specimen in each condition (Thilakarathna et al., 9 Dec 2025). This framing emphasizes identity persistence under two primary nuisance factors: partial occlusion and left-right viewpoint inconsistency.

2. Acquisition protocol and physical setup

AutoFish was collected in a controlled laboratory environment using an RGB camera positioned over a white conveyor belt. The capture hardware is specified as a Jai GO-5100C-USB camera with a KOWA LM12HC lens, mounted 1.5 m above a 1 m × 1 m section of belt. Images have resolution 2,464 × 2,056 px, and the optical configuration used $f/11$ and a 1 m focus distance to obtain sharp detail across the belt (Bengtson et al., 7 Jan 2025). Illumination combined controlled artificial lighting with ambient daylight, introducing minor variation across sessions (Bengtson et al., 7 Jan 2025).

Calibration is an integral component of the dataset design. For each of the 25 groups, 20 checkerboard images with 20 mm × 20 mm squares were captured in varied poses to estimate lens distortion and a homography mapping image pixels to real-world centimeters (Bengtson et al., 7 Jan 2025). This calibration is directly relevant to downstream morphometric estimation, especially fish length prediction.

The re-identification study characterizes the setup as an electronic-monitoring conveyor-belt simulation in which live fish were placed on a moving belt under controlled illumination and imaged repeatedly from a fixed overhead camera (Thilakarathna et al., 9 Dec 2025). It further notes that training inputs were cropped and resized to $224 \times 224$, and that viewpoint variation arises from fish orientation rather than camera motion (Thilakarathna et al., 9 Dec 2025). This suggests that AutoFish isolates specimen appearance, occlusion, and side-dependent morphology while suppressing environmental clutter and camera-induced nuisance variation.

3. Annotation design and data representation

The annotation pipeline is explicitly staged. During image capture, each fish was point-annotated with its unique ID by a click on the fish body. These point annotations were then used as prompts for the Segment Anything Model (SAM) to generate initial segmentation masks. All SAM-proposed masks were subsequently inspected and manually corrected in LabelMe to ensure precise boundaries, with special care taken to merge multiple mask fragments caused by heavy occlusion under a single specimen ID (Bengtson et al., 7 Jan 2025).

Annotations are distributed in a single MS COCO-style JSON file per split. The fields include polygon-list “segmentation” masks, “category_id” for species, “instance_id” for fish identity, “length_cm” for the manual ground-truth length, bounding-box coordinates, and other standard COCO fields (Bengtson et al., 7 Jan 2025). This representation allows joint use for instance segmentation, species-aware analysis, re-identification, and length regression.

The re-identification work uses the identity label and the instance-segmentation mask to produce tight crops with 2 px padding (Thilakarathna et al., 9 Dec 2025). In that formulation, each image is annotated with a unique fish ID shared across all 40 images of the same specimen and a tight instance mask (Thilakarathna et al., 9 Dec 2025). The result is a dataset that can support both full-image structured prediction and crop-based metric learning pipelines.

A common misconception is that AutoFish is only a species-classification dataset. The annotation schema contradicts that interpretation: specimen-level IDs and per-instance lengths make it an identity-aware and morphometric dataset in addition to a segmentation benchmark (Bengtson et al., 7 Jan 2025).

4. Splits, leakage control, and recommended experimental use

AutoFish uses group-based splits to avoid specimen leakage. Groups $[10, 14, 20, 21, 22]$ are held out as a strict test set, while the remaining 20 groups form the training pool (Bengtson et al., 7 Jan 2025). For length-regression experiments, those 20 groups are further split into 15 groups for training and 5 groups for validation, specifically groups $[1, 6, 11, 17, 25]$ (Bengtson et al., 7 Jan 2025). This split design is notable because repeated appearances of the same fish across multiple placements would otherwise make naive image-level random splitting invalid.

The re-identification study likewise enforces identity-disjoint splits. It uses predefined train, validation, and test partitions with no overlap of fish IDs across splits, maintaining similar per-species distributions. The test set comprises 94 fish IDs, approximately 20% of the total, and a single fixed split is used for reproducibility rather than $k$-fold cross-validation (Thilakarathna et al., 9 Dec 2025).

Recommended training augmentations differ by task. For instance segmentation, random horizontal flips with $p_h = 0.5$, random vertical flips with $p_v = 0.5$, and brightness, contrast, and saturation jitter sampled from $\mathrm{Uniform}[0.75, 1.25]$ were used (Bengtson et al., 7 Jan 2025). For length regression, only color augmentations were applied—contrast $[0.5, 1.5]$, brightness $[0.8, 1.2]$, and saturation $224 \times 224$0—with no geometric transforms, specifically to preserve the pixel-to-centimeter mapping (Bengtson et al., 7 Jan 2025).

In the re-identification pipeline, crops are generated using ground-truth masks plus 2 px padding and then transformed by a custom resize-and-pad-to-square operation to $224 \times 224$1: the larger side is resized to fill 224 px while preserving aspect ratio, and the shorter side is padded to square shape (Thilakarathna et al., 9 Dec 2025). Channel-wise normalization uses means $224 \times 224$2 and standard deviations $224 \times 224$3, with $224 \times 224$4 (Thilakarathna et al., 9 Dec 2025).

5. Baseline tasks and reported performance

AutoFish was introduced with baseline results for instance segmentation and fish-length estimation (Bengtson et al., 7 Jan 2025). A later study added a re-identification benchmark centered on retrieval metrics and metric learning (Thilakarathna et al., 9 Dec 2025).

Task	Baseline method(s)	Reported best result
Instance segmentation	Mask2Former with ResNet-50 and Swin-Base backbones	Swin-Base: mAP $224 \times 224$5
Length estimation	SKL and REG	REG: MAE $224 \times 224$6 cm without occlusion; $224 \times 224$7 cm with occlusion
Re-identification	ResNet-50 and Swin-T with hard triplet mining	Swin-T: $224 \times 224$8 mAP@k and $224 \times 224$9 Rank-1

For instance segmentation, the benchmark uses Mask2Former with ResNet-50 and Swin-Base backbones, both pre-trained on COCO and fine-tuned for 1,000 steps with batch size 8, AdamW, and a multi-step learning-rate schedule from 0.1 to 0.0001 (Bengtson et al., 7 Jan 2025). The evaluation metric is mean Average Precision over IoU thresholds from 0.50 to 0.95 in increments of 0.05:

$[10, 14, 20, 21, 22]$0

On the combined test set, the ResNet-50 backbone attains $[10, 14, 20, 21, 22]$1 mAP and the Swin-Base backbone attains $[10, 14, 20, 21, 22]$2 mAP (Bengtson et al., 7 Jan 2025). The same study reports approximately $[10, 14, 20, 21, 22]$3 mAP for touching specimens with Swin-B and nearly $[10, 14, 20, 21, 22]$4 mAP when fish are isolated, indicating strong but not uniform robustness across occlusion regimes (Bengtson et al., 7 Jan 2025).

For length estimation, two approaches are benchmarked. The SKL method is skeletonization-based: it extracts a central skeleton using Zhang–Suen thinning, fits a 4th-degree polynomial to handle curvature and split fragments, maps pixel coordinates to centimeters through the pre-computed homography, and sums segment lengths (Bengtson et al., 7 Jan 2025). The REG method is learning-based: it uses an ImageNet-pretrained MobileNetV2 with a two-layer fully connected regression head, taking as input a tight RGB crop around the mask plus normalized bounding-box coordinates, and is trained with $[10, 14, 20, 21, 22]$5 loss, batch size 32, 200 epochs, and fixed learning rate 0.001 (Bengtson et al., 7 Jan 2025). The Mean Absolute Error is

$[10, 14, 20, 21, 22]$6

On predicted masks with confidence at least $[10, 14, 20, 21, 22]$7, REG achieves $[10, 14, 20, 21, 22]$8 cm MAE for no occlusion and $[10, 14, 20, 21, 22]$9 cm MAE under occlusion (Bengtson et al., 7 Jan 2025). As a reference point, SKL achieves $[1, 6, 11, 17, 25]$0 cm MAE on perfect ground-truth masks but degrades to approximately $[1, 6, 11, 17, 25]$1 cm under occlusion on predicted masks (Bengtson et al., 7 Jan 2025).

For re-identification, the later study employs $[1, 6, 11, 17, 25]$2-D $[1, 6, 11, 17, 25]$3-normalized embeddings, Euclidean distance, Triplet Margin Loss with fixed margin $[1, 6, 11, 17, 25]$4, a PK sampler, and hard-triplet mining from PyTorch Metric Learning (Thilakarathna et al., 9 Dec 2025). The loss is

$[1, 6, 11, 17, 25]$5

Performance is reported with Rank-1 Accuracy and mAP@39, because each query has exactly 39 true matches in the gallery (Thilakarathna et al., 9 Dec 2025). The Vision Transformer-based Swin-T consistently outperforms the CNN-based ResNet-50, achieving peak performance of $[1, 6, 11, 17, 25]$6 mAP@k and $[1, 6, 11, 17, 25]$7 Rank-1 accuracy (Thilakarathna et al., 9 Dec 2025).

6. Error structure, limitations, and interpretive significance

The benchmark results indicate that different nuisance factors affect different tasks in distinct ways. In instance segmentation and length estimation, heavy occlusion remains a principal failure mode. AutoFish explicitly notes that heavy occlusions degrade mask quality and skeletonization-based approaches, and that extreme overlap remains challenging even though the CNN-based regressor is more robust to missing head or fin segments (Bengtson et al., 7 Jan 2025). This establishes occlusion not merely as a dataset property but as a central benchmark variable.

For re-identification, the dominant error source is not inter-species ambiguity but intra-species confusion. Nearly all Rank-1 mistakes occur between different individuals of the same species, while inter-species confusion is almost zero (Thilakarathna et al., 9 Dec 2025). The same study further reports that viewpoint inconsistency is more damaging than partial occlusion: Separated-Initial to Separated-Flipped yields an mAP@39 drop from about $[1, 6, 11, 17, 25]$8 to about $[1, 6, 11, 17, 25]$9, whereas Separated-Initial to Touched-Initial remains high at about $k$0, and combined flips plus occlusion reduce performance further to about $k$1 mAP@39 (Thilakarathna et al., 9 Dec 2025). A non-occluded query against an occluded gallery also performs better than the reverse, suggesting that complete query views provide more discriminative identity cues (Thilakarathna et al., 9 Dec 2025).

AutoFish has several stated limitations. Its environment is a controlled laboratory conveyor belt, so models may require re-calibration, including a new homography and re-training, for in-field deployment or highly variable lighting (Bengtson et al., 7 Jan 2025). Species diversity is concentrated on Gadidae and a small set of common North Sea catches, and generalization to other taxa or underwater settings would require additional data (Bengtson et al., 7 Jan 2025). Length ground truths are rounded to the nearest 5 mm, which limits the theoretical floor of achievable MAE to approximately $k$2 cm (Bengtson et al., 7 Jan 2025).

A plausible implication is that AutoFish is best understood as a high-control benchmark for fine-grained perception under identity-preserving repeated observations, rather than as a direct proxy for unconstrained marine imagery. Its strength lies in isolating hard problems—touching instances, lateral-side changes, and within-species individual discrimination—while holding many environmental variables fixed.

7. Research uses and broader relevance

The primary applications identified for AutoFish are real-time or post-processing analysis of catch landings, including automated species counts, length measurements, and individual-level tracking across multiple conveyor-belt passes (Bengtson et al., 7 Jan 2025). The dataset is also positioned for integration with vessel-mounted electronic-monitoring systems to improve compliance and data fidelity for sustainable fisheries management and overfishing detection (Bengtson et al., 7 Jan 2025). Ecological monitoring applications include fine-grained size-distribution analysis and individual growth studies (Bengtson et al., 7 Jan 2025).

The re-identification study refines this application picture by framing AutoFish as a surrogate for identity loss in operational EM systems, where fish may overlap, flip, or re-enter the field of view in a new pose (Thilakarathna et al., 9 Dec 2025). It argues, through its error analysis, for collecting hard negatives among visually similar individuals and under multiple viewpoints, and it suggests that future EM systems could prioritize “best shot” selection of complete, unobstructed views to reduce identity ambiguity (Thilakarathna et al., 9 Dec 2025). This suggests that AutoFish is not only a dataset but also a structured benchmark for studying the interaction between acquisition protocol and downstream identity preservation.

Within the broader landscape of computer vision datasets, AutoFish occupies a specialized niche at the intersection of instance segmentation, morphometric regression, and re-identification. Its defining contribution is the coexistence of specimen IDs, precise length annotations, and repeated observations under controlled but nontrivial occlusion and viewpoint variation (Bengtson et al., 7 Jan 2025). That combination makes it especially relevant for research on automated catch documentation pipelines in which segmentation, measurement, and identity association must operate jointly rather than as isolated subtasks.