BIOSCAN-1M Insect Dataset

• BIOSCAN-1M is a multimodal insect dataset that combines high-resolution microscope images, detailed taxonomic labels, and mitochondrial DNA barcodes.
• The dataset comprises over 1.1 million records with a long-tailed class distribution and hierarchical taxonomic structure for refined biodiversity analyses.
• It supports robust benchmarks in image-based taxonomic assessment and multimodal learning, offering standardized splits for order-level and Diptera family classification.

The BIOSCAN-1M Dataset, introduced as the BIOSCAN-1M Insect Dataset and also referred to as the BIOSCAN-1M Insect image dataset or BIOSCAN-1M-Insect dataset, is a curated dataset of 1,128,313 insect records designed for image-based taxonomic assessment and broader biodiversity research (Gharaee et al., 2023). Each record combines high-quality microscope imagery with hand-labelled taxonomic annotations, a raw nucleotide barcode sequence for mitochondrial COI, and a Barcode Index Number (BIN), providing a joint image–taxonomy–DNA resource. The dataset was presented as a step toward a comprehensive survey of global biodiversity, while also exposing machine-learning properties such as strong long-tailed imbalance and hierarchical, highly fine-grained taxonomic structure at lower ranks (Gharaee et al., 2023).

1. Dataset scope and modality structure

BIOSCAN-1M contains approximately 1.1M high-quality microscope images, released as RGB JPEG files at 2880×2160 px. All 1,128,313 records have an associated BIN, and the dataset contains 90,918 unique BINs in total. In addition to images, every record includes expert taxonomic labels and the raw DNA barcode sequence stored as COI. The metadata file is released as BIOSCAN_Insect_Dataset_metadata in TSV and JSON-LD formats.

The dataset was assembled primarily to train computer-vision models capable of image-based taxonomic assessment. At the same time, its construction makes it relevant to multimodal learning, long-tailed recognition, hierarchical classification, and fine-grained recognition. A common misconception is that BIOSCAN-1M is only an image corpus; in fact, the dataset is explicitly multimodal because each specimen is linked to both DNA and taxonomy. Another common misconception is that BINs are equivalent to formal taxonomic species labels. In the dataset definition, a BIN is a URI assigned to Operational Taxonomic Units based on clustering mitochondrial COI sequences and serves as a species-like proxy rather than a formal species designation.

The released data are accessible through the dataset portal at https://biodiversitygenomics.net/1M_insects/, with accompanying code and tools for download, reading, cropping, split generation, and baselines at https://github.com/zahrag/BIOSCAN-1M. The linkage to BOLD Systems (https://boldsystems.org/) situates the dataset within a broader biodiversity informatics infrastructure.

2. Taxonomic hierarchy, BINs, and metadata schema

The taxonomic hierarchy follows Linnean ranks together with auxiliary fields. The rank fields are Phylum, Class, Order, Family, Subfamily, Tribe, Genus, Species, and Subspecies, while the name field stores the most specific available label for each record. When a label is absent at a given rank, the dataset uses not_classified.

The coverage across ranks is highly uneven, reflecting the practical difficulty of fine-grained annotation. Phylum and Class each have one category, with all 1,128,313 records labelled; Order has 16 categories and also covers all records; Family has 491 categories with 1,112,968 labelled records; the lower ranks are markedly sparser. The BIN field, by contrast, covers all records.

Rank	Categories	Labelled records
Phylum	1	1,128,313
Class	1	1,128,313
Order	16	1,128,313
Family	491	1,112,968
Subfamily	760	265,492
Tribe	535	60,477
Genus	3,441	254,096
Species	8,355	84,397
BIN	90,918	1,128,313
Name	10,952 unique names	—

This rank profile is central to the dataset’s scientific character. There are 3,441 genera and 8,355 species represented, but only 254,096 genus-level labels and 84,397 species-level labels. This suggests that BIOSCAN-1M is not simply a large-scale fine-grained classification benchmark in the conventional fully labelled sense; rather, it combines complete coverage at higher ranks with sparse lower-rank annotation and complete BIN coverage.

The metadata schema contains 22 fields. Core identifiers are sampleid, which is the collector’s identifier, and processid, which is the unique BOLD record identifier. The field uri stores the BIN, nucraw stores the COI nucleotide barcode sequence, and image_file and chunk_number link metadata to the image packages. Taxonomic fields include name, phylum, class, order, family, subfamily, tribe, genus, species, and subspecies. Benchmark split assignments are stored in large_diptera_family, medium_diptera_family, small_diptera_family, large_insect_order, medium_insect_order, and small_insect_order.

Each sample is uniquely identified by the pair of BOLD processid and collector sampleid. These fields connect images to taxonomy, DNA, and BINs, enabling exact joins across modalities and reproducible reconstruction of the official benchmarks.

3. Collection, imaging, sequencing, and curation

The specimens were mostly collected from Costa Rica, Canada, and South Africa using Malaise traps (Gharaee et al., 2023). Imaging was performed with a Keyence VHX-7000 microscope system, producing high-resolution RGB JPEGs with white or light backgrounds being common but varied. Depth stacking was used to ensure in-focus imagery. The workflow was organized around 96-well microplates, with 96 plates per sequencing run, corresponding to 9,120 samples per run.

DNA sequencing was performed on the Pacific Biosystems Sequel platform using SMRT long-read sequencing for mitochondrial COI barcodes. The resulting barcodes were matched to the BOLD reference library at the Centre for Biodiversity Genomics in Canada. Expert taxonomists—approximately 15–20 personnel from the Centre for Biodiversity Genomics—provided taxonomic labels, primarily by morphology for higher ranks such as order and family. At finer ranks such as genus and species, annotation often required combined visual verification and BIN support, and many samples remained without fine-grained labels because of time and complexity as well as barcode ambiguities.

The paper does not report explicit inter-annotator agreement metrics or numerical error rates. That omission is important for interpreting the dataset: the curation process is expert-led and multimodal, but the released documentation does not quantify agreement or annotation uncertainty. The JSON-LD metadata release was validated using Google’s Inspection Tool.

For preprocessing, the release includes optional cropping and resizing. A DETR-based cropping tool was trained to auto-crop insects. The model was DETR with a ResNet-50 backbone pretrained on MSCOCO, fine-tuned for 10 epochs with AdamW using learning rate $= 0.0001$, weight decay $= 0.0001$, batch size $= 8$, and a binary class setup of “insect vs. no object.” At inference time, the top-confidence bounding box is selected and then extended equally in width and height by 0.4 of the maximum dimension, equivalently with parameter $R = 1.4$, using

$$\text{ExtendSize} = (R - 1) \times \max(\text{width}, \text{height}).$$

Runtime was reported as approximately 2h40m per 10k images on CPU with 16 cores and 4 workers, and approximately 30m per 10k images on an RTX 2080 Ti GPU. The mixed training split consisting of IP-1000 and IW-1000 yielded the highest AP/AR, and cropped images generally improved downstream classifier accuracy and convergence.

4. Packaging, class imbalance, and official benchmarks

The released image data are packaged at multiple scales to support different storage and training regimes. Original JPEG images are distributed across 113 zip files, each containing 10,000 images except the last smaller chunk, for a total size of approximately 2.3 TB. Cropped images are also released in 113 zip files, totaling approximately 151 GB. Resized original images with smaller side $= 256$ px occupy approximately 26 GB and are available as zip and HDF5. Resized cropped images with smaller side $= 256$ px occupy approximately 7 GB and are likewise available as zip and HDF5. The resized original and cropped sets are also organized in HDF5 archive format.

A defining statistical property of BIOSCAN-1M is its strong long-tailed class-imbalance distribution, described as reflecting true biological diversity and sampling realities: many rare taxa and a few very common ones. At the order level, the counts are highly skewed: Diptera accounts for 896,324 images, followed by Hymenoptera with 89,311, Coleoptera with 47,328, Hemiptera with 46,970, and Lepidoptera with 32,538; at the extreme tail are Ephemeroptera with 96, Dermaptera with 66, Archaeognatha with 63, Plecoptera with 30, and Embioptera with 6. Siphonaptera, Strepsiptera, and Zoraptera were removed from experiments due to insufficient samples.

The imbalance factor $\beta$, defined as the ratio of the largest to the smallest class, emphasizes the severity of this skew:

Dataset or benchmark	$\beta$
BIOSCAN-1M (by BIN)	12,491.00
BIOSCAN-Order	156,856.75
BIOSCAN-Diptera	1,092.61
Pl@ntNet-300K	3,604.00
iNaturalist-Insect	1.97

The official benchmarking tasks are BIOSCAN-Order for order-level classification and BIOSCAN-Diptera for Diptera family-level classification. Splits are created by stratified class-based sampling with 70% train, 10% validation, and 20% test in each class; classes without samples for a split are omitted. The recommended evaluation metrics are micro and macro top-$K$ accuracy for $= 0.0001$0, micro and macro F1, together with per-class accuracy curves and confusion matrices. No hierarchical loss or hierarchical evaluation metrics were used in the baseline experiments, and future work was suggested in that direction (Gharaee et al., 2023).

Benchmark subset	Total / Train / Val / Test	Classes
BIOSCAN-Order	1,128,308 / 789,813 / 112,835 / 225,660	16
BIOSCAN-Diptera	891,338 / 623,937 / 89,135 / 178,266	40
Medium subsets	200,000 / 140,000 / 20,000 / 40,000	16 / 40
Small subsets	50,000 / 35,000 / 5,000 / 10,000	16 / 40

5. Baseline models and reported performance

The paper evaluates baseline classifiers based on ResNet-50 and ViT-Base-Patch16-224 (ViT-B/16) (Gharaee et al., 2023). Images were resized to 256 px on the shorter side; during training, random 224×224 crops and random horizontal flips were used, while validation and test used 224×224 center crops. The loss functions were Cross-Entropy and Focal loss. Under the reported settings, Cross-Entropy slightly outperformed Focal loss, and the focal hyperparameters $= 0.0001$1 and $= 0.0001$2 were not extensively tuned.

Optimization used SGD with learning rate $= 0.0001$3, momentum $= 0.0001$4, weight decay $= 0.0001$5, batch size $= 0.0001$6, and 100 epochs, with the large setting trained for fewer epochs upon convergence. Training used one node, one GPU, 10 CPUs per task, and 128 GB RAM on the Narval and Beluga clusters. Model selection was based on multiple seeds, with the best average validation model selected for test inference. ViT-B/16 with Cross-Entropy was chosen for most final test evaluations.

The best reported results are as follows.

Benchmark	Micro Top-1/3/5; Micro/Macro F1	Macro Top-1/3/5
Insect-Order Small	97.86 / 99.35 / 99.66; 97.86 / 85.84	85.01 / 91.68 / 99.23
Insect-Order Medium	99.14 / 99.77 / 99.88; 99.14 / 87.36	85.58 / 97.68 / 98.22
Insect-Order Large	99.69 / 99.96 / 99.98; 99.62 / 92.65	90.61 / 98.14 / 99.32
Diptera-Family Small	94.01 / 97.26 / 98.01; 94.01 / 93.03	92.37 / 96.53 / 97.42
Diptera-Family Medium	96.66 / 98.34 / 98.77; 96.66 / 92.77	91.81 / 96.37 / 97.20
Diptera-Family Large	97.59 / 98.85 / 99.23; 97.59 / 91.45	91.20 / 95.86 / 96.72

The performance profile is not uniform across the label space. Per-class top-1 accuracy decreases primarily in classes with very few samples, especially tail classes. Qualitative error analysis further suggests that a large proportion of misclassifications are linked to low-quality images—approximately 57% of misclassifications on the Small setting and approximately 45% on the Large setting—and that the classifiers tend to overpredict the dominant class, Diptera. This suggests that aggregate micro metrics should be interpreted alongside macro metrics and classwise analyses, particularly for imbalanced taxonomic benchmarks.

6. Limitations, licensing, and research directions

Several constraints define the current scope of BIOSCAN-1M. Many samples lack fine-grained labels because lower-rank annotation is time-consuming and complex; consequently, genus- and species-level coverage is limited relative to the number of represented classes. Image quality is also variable: backgrounds and lighting vary, some specimens are fragmented, and cropping helps but is not flawless. The absence of reported inter-annotator agreement and explicit label error rates further limits direct quantification of annotation reliability.

The dataset is released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 license. The photographer is listed as CBG Robotic Imager; the copyright holder is the CBG Photography Group; the institution is the Centre for Biodiversity Genomics; the copyright contact is [email protected]; and the copyright year is 2021. The release notes also state that, to the authors’ knowledge, no personal data are included. The recommended citation practice is to cite the paper "A Step Towards Worldwide Biodiversity Assessment: The BIOSCAN-1M Insect Dataset" (Gharaee et al., 2023) together with the dataset URL https://biodiversitygenomics.net/1M_insects/.

The paper explicitly frames BIOSCAN-1M as a platform for several research directions. These include long-tailed recognition with re-weighting losses, minority class oversampling, and advanced debiasing; hierarchical classifiers with uncertainty estimates across ranks; multimodal learning that integrates image features with nucraw and uri; and zero-shot or few-shot recognition using BINs and DNA barcodes as side information. The authors also suggest semi-parametric methods that use reference libraries at test time, set-valued classification, and domain adaptation or generalization. Because the dataset is linked to BOLD, a plausible implication is that BIOSCAN-1M can support broader biodiversity analyses that combine specimen imaging with taxonomic and genetic reference systems, including species discovery and monitoring.

The practical workflow recommended for use is straightforward: download from the dataset portal, select among full-resolution or cropped/resized packages depending on storage and compute constraints, use the TSV or JSON-LD metadata to map images to taxonomy, DNA, and BINs via processid and sampleid, and replicate official train/validation/test splits via the split fields. The provided baselines start from ViT-B/16 with Cross-Entropy, standard 256→224 crops, and SGD with learning rate $= 0.0001$7, momentum $= 0.0001$8, weight decay $= 0.0001$9, and batch size $= 8$0, evaluated with micro and macro top-$= 8$1 accuracy and F1. In that sense, BIOSCAN-1M functions both as a biodiversity resource and as a technically demanding benchmark for multimodal, long-tailed, and hierarchical recognition.