Fursee: Hybrid YOLO-DINOv3 Framework for Fursuit Identity Retrieval and Clustering

Abstract: Global furry conventions produce massive fursuit photographs, while manual sorting brings heavy labor costs and calls for automatic identity retrieval and clustering solutions. General multimodal models lack dedicated optimization for complex fursuit scenes, and no public benchmark dataset exists for this task. To fill this gap, we build a specialized fursuit image dataset and present a three-stage hybrid pipeline Fursee for fursuit identity retrieval and clustering. First, YOLO detects and crops high-resolution fursuit head patches to improve localization of small and overlapping targets. Second, ArcFace optimizes DINOv3 embeddings to enlarge angular separation between different identities on the feature hypersphere. Third, DBSCAN performs unsupervised clustering, with silhouette-coefficient-driven search automatically selecting optimal hyperparameters rather than fixed manual radius. Retrieval and clustering experiments verify that our pipeline outperforms mainstream multimodal models including GPT5.5, Claude Opus 4.8 and Qwen3.7-Plus on all evaluation metrics, achieving competitive performance for fursuit head retrieval and grouping.

• The paper introduces a three-stage pipeline integrating YOLO detection and fine-tuned DINOv3 embeddings, achieving a 93.33% hit rate and robust clustering performance.
• It leverages attention module fine-tuning with segmentation masks and ArcFace loss to enhance discriminative feature extraction in challenging fursuit imagery.
• DBSCAN clustering with silhouette optimization delivers improved precision, recall, and F1-score compared to baseline multimodal models.

Authoritative Essay on "Fursee: Hybrid YOLO-DINOv3 Framework for Fursuit Identity Retrieval and Clustering"

Motivation and Problem Statement

The proliferation of fursuit photographs from global furry conventions introduces a pressing need for automated, scalable solutions to fursuit identity retrieval and clustering. Traditional manual sorting is inefficient, while general multimodal models, lacking domain-specific optimization, demonstrate limited accuracy in fine-grained fursuit recognition. Unique challenges are posed by fursuit imagery: most heads are constructed from identical base molds, making color, pattern, and texture the primary discriminative features rather than geometric structure or facial landmarks. Existing benchmarks are absent, and face recognition methodologies developed for humans fail given the low geometric variation and significant occlusion.

Hybrid Framework: YOLO Detection and DINOv3 Embedding

The proposed Fursee pipeline addresses these complexities with a three-stage cascade:

1. YOLO Detection: A YOLO26l detector, trained on a custom dataset incorporating hard negative human samples, identifies and crops high-resolution fursuit head patches. This transforms multi-object images into single-character patches, effectively mitigating background interference and improving localization for small and overlapping targets.

   Figure 1: The Fursee pipeline—YOLO detects and crops fursuit heads; DINOv3 attention layers are selectively fine-tuned for embedding extraction; projection and ArcFace heads are updated exclusively during training, supporting cosine similarity-based retrieval and clustering.

2. DINOv3 Embedding Extraction: Fine-tuning is confined to upper attention layers (18–23), guided by segmentation masks and bounding boxes to enforce attention on discriminative regions. The backbone remains frozen during embedding learning; only the projection and ArcFace heads are optimized with a hybrid loss (ArcFace plus supervised contrastive loss), maximizing angular separation and generalizable identity discrimination.
3. DBSCAN Clustering with Silhouette Optimization: Density-based clustering leveraging cosine similarity and silhouette coefficients enables unsupervised partitioning of unknown identity counts without manual $\epsilon$ radius selection. Silhouette-guided grid search for optimal hyperparameters ensures tight intra-cluster organization and clear inter-cluster boundaries.

Attention Module Fine-tuning

Fine-tuning the ViT attention layers is critical for suppression of residual background and enhancement of key fursuit head features. Training supervision employs a heatmap constructed from segmentation masks and bounding boxes, enforced with KL-divergence loss. The result is a redistribution of model attention, exclusively concentrating on the fursuit head after fine-tuning.

Figure 2: Visualization of attention heatmaps—before fine-tuning, attention is diffused; after fine-tuning, the model accurately focuses on the fursuit head region.

Embedding and Metric Learning

ArcFace loss is adopted to enforce angular margin penalties, supplementing supervised contrastive loss for robust identity segregation on the embedding hypersphere. Data augmentation (rotations, distortion, color perturbations) counters overfitting. PK Batch sampling ensures training batches cover diverse identities and variations. The training curves demonstrate convergence and high embedding discrimination.

Figure 3: DINOv3-based embedding model training curves, indicating stable convergence and robust loss minimization.

Identity Retrieval and Clustering: Experimental Results

Comprehensive experiments validate the superiority of Fursee over mainstream multimodal vision-LLMs (GPT5.5, Claude Opus 4.8, Qwen3.7-Plus). On the fursuit retrieval task, Fursee achieves a hit rate of 93.33%, surpassing all baselines. For clustering, Fursee yields weighted final F1-score of 0.8755, with high raw precision (0.8986) and recall (0.8720), demonstrating consistent improvements across overlapping multi-label datasets. Notably, core claims assert Fursee outperforms all tested baselines, including large language vision models, on retrieval and clustering metrics for fursuit identity discrimination.

Ablation Studies and Critical Analysis

Ablation experiments highlight the indispensability of the YOLO detection stage: removing YOLO and utilizing DINOv3 alone results in clear degradation of recall and final clustering F1-score. Weak localization for small, densely-packed heads and limited input resolution are the principal limitations of the DINO-only variant, validating the detect-then-embed cascade.

Implications and Future Directions

The work defines a robust, scalable pipeline for domain-specific identity retrieval and clustering, applicable to other visually subtle, multi-object datasets where geometric cues are minimal or occlusion is rampant. The hybrid detect-then-embed paradigm, attention fine-tuning, and adaptive clustering provide a template for future advances in fine-grained visual subgroup classification. Extending dataset diversity, hard case coverage, and accessory-invariant embedding learning are promising future directions. Addressing clustering instability under extreme occlusion or variable accessory presence is critical for broader generalization.

Conclusion

Fursee introduces a domain-specialized, three-stage hybrid pipeline for accurate fursuit identity retrieval and clustering, outperforming state-of-the-art multimodal vision-language baselines on all reported metrics. Attention fine-tuning, ArcFace-optimized embeddings, and silhouette-net clustering collectively address the core challenges posed by fursuit imagery. Ablation underscores the necessity of detection-guided cropping, central to resolving small and dense target localization. The methodology establishes a foundation for automated classification in highly overlapping, low-geometry fine-grained visual domains and signals productive directions for future research in adaptive identity retrieval and clustering architectures.