MATANet: Multi-context & Taxonomy-Aware Network

• The paper demonstrates that MATANet fuses multi-scale context with biological taxonomy, outperforming baseline models in marine species recognition.
• Its methodology employs a three-stage pipeline: ROI extraction with multi-scale cropping, multi-context environmental attention, and hierarchical taxonomic supervision.
• Results show superior hierarchical consistency and lower Hierarchical Distance across datasets like FathomNet2025, FishCLEF2015, and FAIR1M.

MATANet (Multi-context Attention and Taxonomy-Aware Network) is a deep learning architecture for fine-grained hierarchical classification, specifically developed for the recognition of marine species in underwater imagery. It is designed to leverage both environmental context and biological taxonomy, combining multi-scale spatial cues and taxonomic hierarchies in a unified framework. MATANet outperforms previous vision models and hierarchical supervision strategies on several large-scale benchmarks by fusing environmental information with taxonomy-aligned representations, closely mimicking expert taxonomist practices (Lee et al., 7 Jan 2026).

1. Architectural Overview

MATANet consists of a three-stage pipeline: (i) input preparation with multi-scale context cropping, (ii) feature extraction and fusion via the Multi-Context Environmental Attention Module (MCEAM), and (iii) taxonomy-aware supervision with the Hierarchical Separation-Induced Learning Module (HSLM).

Given an image region of interest (ROI) around a target marine organism, MATANet extracts the ROI and three concentric square context crops (3×, 5×, and full image) centered on the ROI. All inputs are resized to $256 \times 256$ pixels. A Vision Transformer (ViT, specifically DiNOv2 in the primary implementation) encodes both the ROI and context crops to produce embeddings. Cross-attention is performed between the ROI embedding and the patch embeddings from each context crop, fusing environment and instance-level cues into a single feature vector. This fused embedding is supervised using both flat and hierarchical classification losses across all levels of the marine taxonomic tree, enforcing semantic consistency with known biological relationships (Lee et al., 7 Jan 2026).

2. Multi-Context Environmental Attention Module (MCEAM)

MCEAM integrates information from both the focal object and its environment using multi-head cross-attention. The module operates as follows:

• For each image, the ViT backbone yields a [CLS] token embedding $g \in \mathbb{R}^d$ (encoding the ROI) and patch embeddings $P_{cr} \in \mathbb{R}^{N \times d}$ for each context scale $r$.
• Cross-attention is computed for each context: the ROI embedding $g$ serves as the query, while $P_{cr}$ provides keys and values. Concretely:

$$Q_r = g W_r^Q, \quad K_r = P_{cr} W_r^K, \quad V_r = P_{cr} W_r^V$$

$$A_r = \mathrm{softmax} \left( \frac{Q_r K_r^T}{\sqrt{d_k}} \right)$$

$z_r = A_r V_r$

• These attended vectors $z_{3\times}$, $z_{5\times}$, $z_{\mathrm{full}}$ are concatenated with $g$ and projected by a two-layer MLP:

$$z = \mathrm{Proj}\left( [g ; z_{3\times} ; z_{5\times} ; z_{\mathrm{full}}] \right)$$

• The number of stacked cross-attention blocks is four, each with four heads.

This module captures complementary spatial cues, context-specific habitat information, and patterns such as “separated attention,” “complementary attention,” and “clustered attention,” which are observed in both animal-centric regions and their associated environments (Lee et al., 7 Jan 2026).

3. Hierarchical Separation-Induced Learning Module (HSLM)

HSLM enforces explicit alignment between learned representations and biological taxonomic hierarchies. For a given set $H$ of taxonomic levels (e.g., phylum, class, order, family, genus, species):

• Each level $l \in H$ is assigned a dedicated two-layer MLP classifier $f_l(z)$ with logits for all classes at level $l$.
• Supervision is enforced by combining a standard species-level cross-entropy loss:

$$L_{\mathrm{cls}} = \mathrm{CE}(f_c(z), y_{\mathrm{species}})$$

with auxiliary losses for all hierarchy levels:

$$L_{\mathrm{hier}} = \sum_{l \in H} \mathrm{CE}(f_l(z), y_l)$$

• The final objective is the sum:

$$L_{\mathrm{total}} = L_{\mathrm{cls}} + L_{\mathrm{hier}}$$

• If lower-level annotations are absent, parent-level labels are used as targets (label interpolation).

This strategy produces embeddings that reflect taxonomy-consistent clustering, as evidenced by t-SNE visualizations where related phyla are adjacent, and distant phyla are well-separated (Lee et al., 7 Jan 2026). Studies with randomized taxonomic labels degrade performance, confirming the necessity of structured hierarchical supervision.

4. Training Protocol and Implementation

• Input crops: All ROIs and context regions are extracted as squares; context sides are 3×, 5×, and full image, each centered on the ROI’s bounding box.
• Image size: All inputs are resized to $256 \times 256$.
• Data augmentation: Random horizontal/vertical flip, color jitter, and rotation.
• Backbone: DiNOv2-pretrained ViT; all classifiers/MLPs have two layers.
• Optimization: AdamW with learning rates $10^{-6}$ (FathomNet2025, FAIR1M), $10^{-3}$ (FishCLEF2015). CNN and ViT baselines used $10^{-4}$ and $10^{-5}$, respectively. Batch size: 32; epochs: 30; fixed random seed (Lee et al., 7 Jan 2026).

5. Quantitative Results and Ablation Analyses

MATANet demonstrates state-of-the-art performance across FathomNet2025, FishCLEF2015, and FAIR1M. The primary metric for FathomNet2025 is Hierarchical Distance (HD; lower is better), which quantifies semantic distance between predicted and true taxon in the hierarchy.

Method	FathomNet2025 (WgtAvg HD)	FishCLEF2015 (ACC)	FishCLEF2015 (HD)	FAIR1M (ACC)	FAIR1M (HD)
VGG19	4.20	0.671	1.81	0.661	1.26
ResNet50	3.68	0.684	1.86	0.710	1.08
SWIN-B	3.12	0.770	1.37	0.728	1.01
DINOv2-B	2.84	0.744	1.42	0.726	1.02
MATANet (final)	1.54	0.789	1.13	0.740	0.97

Ablation studies show that:

• Including additional context scales (3×, 5×, full) incrementally improves HD.
• Adding HSLM reduces HD by ~7%.
• Upgrading from ViT-Base to ViT-Large provides an additional ~13% gain.
• Among hierarchical auxiliary strategies, HSLM yields the lowest HD compared to contrastive or probabilistic label-chain alternatives.
• Randomized hierarchy supervision decreases performance below the “no hierarchy” baseline, underscoring the importance of accurate taxonomic structure (Lee et al., 7 Jan 2026).

6. Qualitative Insights and Attention Behavior

MCEAM visualizations reveal three primary attention behaviors when querying with the ROI over context crops:

1. Evenly distributed scene-level attention in the absence of salient cues.
2. Focused attention on meaningful habitat features (such as reefs).
3. Suppression of irrelevant regions (e.g., anthropogenic objects).

On the 3× and 5× crops:

• “Separated attention” allows the model to distinguish animal from background.
• “Complementary attention” integrates animal and habitat cues.
• “Clustered attention” highlights conspecific groups, relevant for schooling taxa.

A failure mode in attention drift toward image borders, potentially due to positional encoding bias, is noted as a future direction (Lee et al., 7 Jan 2026).

7. Significance and Comparative Perspective

MATANet’s fusion of environmental cues with taxonomy-aware supervision enables the model to capture subtle inter- and intra-species variations, reflecting both morphological traits and ecological context. The use of lightweight feature fusion and level-wise classification heads adds minimal computational burden relative to the backbone. Empirical superiority across metrics and datasets, coupled with robustness to domain shifts (notably in FAIR1M), positions MATANet as a robust solution for fine-grained hierarchical recognition tasks involving taxonomically structured categories (Lee et al., 7 Jan 2026).

A plausible implication is that approaches combining multi-scale context integration and accurate taxonomy-aware supervision may offer similar advantages in other domains featuring hierarchical class structures and context-dependency, such as botanical, entomological, or document taxonomy applications. MATANet’s publicly available implementation facilitates further research in broad ecological and remote sensing contexts.