Fursee: Fursuit Identity Retrieval & Clustering

• The paper introduces a three-stage pipeline that first detects fursuit heads, then extracts 512-dimensional identity embeddings, and finally clusters them using DBSCAN.
• It leverages a hybrid architecture combining YOLO for detection, DINOv3 with ArcFace and supervised contrastive learning, enhancing fine-grained discrimination in cluttered images.
• By integrating a grid search for DBSCAN hyperparameters and an overlapping-cluster evaluation protocol, Fursee effectively addresses open-set, multi-identity challenges in convention photography.

Searching arXiv for the Fursee paper and closely related technical components. Fursee is a three-stage vision pipeline for fursuit identity retrieval and clustering in crowded convention photography, proposed to address a setting in which one must retrieve all images of a queried identity and group photos by identity without knowing in advance how many identities exist (Wu, 22 Jun 2026). The method is explicitly domain-specific: the paper argues that fursuit heads are visually unusual, that discriminative cues often lie in colors, patterns, and fur textures rather than facial geometry, and that convention photos are cluttered with occlusion, overlap, and multiple identities in the same frame. In this formulation, the task is closer to fine-grained identity retrieval and open-set clustering than to ordinary object classification.

1. Problem setting and benchmark construction

The paper motivates Fursee by identifying a dataset and task mismatch. It states that there was no public standard dataset for fursuit-related visual tasks, and that existing face, animal, or character-recognition datasets do not reflect the real convention-photo setting: multiple fursuits in one image, small heads, severe overlap, and identity classes that are not closed or known ahead of time (Wu, 22 Jun 2026). To address this, the authors collect a custom dataset of 379 images from furry conventions, manually split into 83 identity categories, and also build a detection dataset with bounding boxes and segmentation masks for fursuit heads. Human-face negatives are included to reduce false positives when the detector confuses people for fursuit heads.

This problem formulation differs from standard single-label recognition. Let $\mathcal{P} = \{p_1, \dots, p_n\}$ be all photos and $\mathcal{F} = \{f_1, \dots, f_m\}$ the set of identities. For an identity $f_j$, the associated photo subset is

$$S(f_j) = \{p \in \mathcal{P} \mid p \text{ contains fursuit identity } f_j\}.$$

Retrieval is defined as returning $S(f_u)$ if the queried identity exists. Clustering is defined as recovering the collection $\{S(f_1), \dots, S(f_m)\}$, with overlap allowed:

$S(f_i) \cap S(f_j) \neq \emptyset$

when a photo contains multiple fursuits.

This overlap condition is central. The task is not ordinary partition clustering, because a single image may belong to multiple identity-associated subsets simultaneously. A plausible implication is that methods assuming mutually exclusive image labels are structurally misaligned with the evaluation setting.

2. Detect-then-embed-then-cluster architecture

Fursee converts a hard multi-identity, multi-object image problem into a sequence of easier subproblems: detection, embedding, and clustering (Wu, 22 Jun 2026). The first stage uses YOLO26l as a localization backbone. The input is a high-resolution $1280 \times 1280$ convention image. The detector is trained on a custom dataset with two labels: fursuit-head bounding boxes and human-face negatives. During inference, all predicted human-face boxes are filtered out, and only fursuit-head crops are retained. Each detected head is then cropped into an independent patch and resized to $224 \times 224$ for the embedding model.

The architectural rationale given in the paper is not merely improved detection accuracy. Patch extraction isolates each fursuit head so that the embedding model can focus on a single subject rather than a cluttered scene. The paper states that raw convention photos often contain tiny heads, overlapping characters, and background distractions, and that a Vision Transformer operating on the entire frame would waste capacity on irrelevant regions.

The overall pipeline can be summarized as follows.

Stage	Component	Function
1	YOLO26l	Detect and crop fursuit-head patches
2	DINOv3 ViT-L/16 + ArcFace	Learn 512-dimensional identity embeddings
3	DBSCAN + silhouette search	Group embeddings into identity clusters

The paper emphasizes that this “localize first, recognize second” design makes the later self-supervised backbone operate on a cleaner input space. A common misconception would be to treat Fursee as a generic multimodal reasoning system; the paper instead presents it as a specialized visual retrieval-and-grouping pipeline tailored to crowded convention imagery.

3. Embedding learning, attention supervision, and hyperspherical geometry

The second stage uses a pre-trained DINOv3 ViT-L/16 model to generate 512-dimensional identity embeddings (Wu, 22 Jun 2026). The authors fine-tune only the later attention layers of DINOv3 and keep most of the backbone frozen: layers 0–17 are frozen, while attention layers 18–23 are trainable in the attention-fine-tuning stage. This partial fine-tuning is intended to preserve general visual knowledge while redirecting attention toward fursuit-head regions.

A distinctive element is the attention supervision. Because the detection dataset has both bounding boxes and segmentation masks, the authors construct a $14 \times 14$ target attention heatmap from the $\mathcal{F} = \{f_1, \dots, f_m\}$0 patch input with patch size $\mathcal{F} = \{f_1, \dots, f_m\}$1. Background cells are set to 0, cells covered by a bounding box receive 0.2, and cells covered by segmentation masks are overwritten with 1.0. The target is then L1-normalized. Training minimizes

$\mathcal{F} = \{f_1, \dots, f_m\}$2

where $\mathcal{F} = \{f_1, \dots, f_m\}$3 is the Softmax-normalized predicted attention and $\mathcal{F} = \{f_1, \dots, f_m\}$4 prevents numerical instability.

After attention tuning, the DINOv3 backbone is frozen and only the projection head plus ArcFace classification head are updated. The paper combines ArcFace loss with supervised contrastive loss:

$\mathcal{F} = \{f_1, \dots, f_m\}$5

with $\mathcal{F} = \{f_1, \dots, f_m\}$6.

The ArcFace objective is given as

$\mathcal{F} = \{f_1, \dots, f_m\}$7

where $\mathcal{F} = \{f_1, \dots, f_m\}$8 is batch size, $\mathcal{F} = \{f_1, \dots, f_m\}$9 is the feature scale, $f_j$0 is the angular margin, $f_j$1 is the angle between the normalized embedding of sample $f_j$2 and the weight vector of its true class, and $f_j$3 are the angles to other classes.

The supervised contrastive term is

$f_j$4

where $f_j$5 are normalized embeddings, $f_j$6 is temperature, and $f_j$7 are positive samples of the same identity.

The geometric interpretation in the paper is the feature hypersphere. Embeddings and class weights are normalized onto a unit sphere, and classification depends on angular separation rather than raw Euclidean distance. ArcFace adds an angular margin, forcing same-class examples into tighter angular clusters and different classes into more separated directions. The paper further notes the use of a PK batch sampler, so that each batch contains $f_j$8 identities and $f_j$9 samples per identity, and lists augmentations including rotation, flips, shear, hue/brightness/saturation perturbations, and radial/tangential lens distortion.

At inference time, the ArcFace head is discarded. Only the learned 512-dimensional embeddings are retained, and similarity is computed with cosine similarity. That same embedding space is used for both retrieval and clustering.

4. Density-based clustering and overlapping-cluster evaluation

The final stage clusters the learned embeddings with DBSCAN, a choice the paper ties directly to the open-set nature of the task: the number of identities is unknown, there may be outliers, and the identities are treated as density-based groups in feature space (Wu, 22 Jun 2026). Because the embeddings are trained with ArcFace, the authors argue that same identities form angularly compact clusters on the hypersphere, making cosine-space DBSCAN a natural fit.

Rather than manually fixing a radius, Fursee performs an automatic grid search over DBSCAN hyperparameters and selects the setting that maximizes silhouette score. The silhouette coefficient is defined as

$$S(f_j) = \{p \in \mathcal{P} \mid p \text{ contains fursuit identity } f_j\}.$$0

where $$S(f_j) = \{p \in \mathcal{P} \mid p \text{ contains fursuit identity } f_j\}.$$1 is the average cosine distance from sample $$S(f_j) = \{p \in \mathcal{P} \mid p \text{ contains fursuit identity } f_j\}.$$2 to others in its own cluster, and $$S(f_j) = \{p \in \mathcal{P} \mid p \text{ contains fursuit identity } f_j\}.$$3 is the minimum average cosine distance from sample $$S(f_j) = \{p \in \mathcal{P} \mid p \text{ contains fursuit identity } f_j\}.$$4 to another cluster. The score ranges from $$S(f_j) = \{p \in \mathcal{P} \mid p \text{ contains fursuit identity } f_j\}.$$5 to $$S(f_j) = \{p \in \mathcal{P} \mid p \text{ contains fursuit identity } f_j\}.$$6, and higher is better.

The evaluation protocol is also specialized because standard clustering metrics do not handle overlap well. The paper computes IoU between ground-truth identity clusters and predicted clusters:

$$S(f_j) = \{p \in \mathcal{P} \mid p \text{ contains fursuit identity } f_j\}.$$7

and then uses the Hungarian algorithm to find an optimal one-to-one matching between predicted and ground-truth clusters.

To penalize over- or under-segmentation, the paper defines a structure coefficient

$$S(f_j) = \{p \in \mathcal{P} \mid p \text{ contains fursuit identity } f_j\}.$$8

where $$S(f_j) = \{p \in \mathcal{P} \mid p \text{ contains fursuit identity } f_j\}.$$9 is the number of ground-truth clusters and $S(f_u)$0 is the number of predicted clusters. Identity weights are defined by occurrence frequency:

$S(f_u)$1

Per-class precision, recall, and F1 are

$S(f_u)$2

These are aggregated into weighted raw metrics:

$S(f_u)$3

and the final score is

$S(f_u)$4

For retrieval, the metric is hit rate. Given a query image, the model retrieves the top-$S(f_u)$5 samples and compares them to a ground-truth set of size $S(f_u)$6:

$S(f_u)$7

This protocol makes explicit that Fursee is evaluated not only on whether it groups correctly, but also on whether it recovers the correct number of groups.

5. Empirical results, baselines, and ablation findings

The paper compares Fursee against three mainstream multimodal models used in zero-shot fashion: GPT5.5, Claude Opus 4.8, and Qwen3.7-Plus (Wu, 22 Jun 2026). On retrieval, the reported hit rates are 70.00% for GPT5.5, 85.00% for Claude Opus 4.8, 46.67% for Qwen3.7-Plus, and 93.33% for Fursee. On clustering, the reported metrics are as follows.

Method	Retrieval hit rate	Clustering $S(f_u)$8
GPT5.5	70.00%	0.6064
Claude Opus 4.8	85.00%	0.3956
Qwen3.7-Plus	46.67%	0.7043
Fursee	93.33%	0.8755

The paper also reports the full clustering metrics. GPT5.5 obtains $S(f_u)$9, $\{S(f_1), \dots, S(f_m)\}$0, and $\{S(f_1), \dots, S(f_m)\}$1. Claude Opus 4.8 obtains $\{S(f_1), \dots, S(f_m)\}$2, $\{S(f_1), \dots, S(f_m)\}$3, and $\{S(f_1), \dots, S(f_m)\}$4. Qwen3.7-Plus obtains $\{S(f_1), \dots, S(f_m)\}$5, $\{S(f_1), \dots, S(f_m)\}$6, and $\{S(f_1), \dots, S(f_m)\}$7. Fursee obtains $\{S(f_1), \dots, S(f_m)\}$8, $\{S(f_1), \dots, S(f_m)\}$9, and $S(f_i) \cap S(f_j) \neq \emptyset$0, with $S(f_i) \cap S(f_j) \neq \emptyset$1.

The paper’s interpretation is that generic language-and-vision reasoning is not the main bottleneck. The challenge is consistent localization and matching of subtle identity cues across many photos. This suggests that domain-specific detection and embedding learning are more consequential here than generic multimodal scene understanding.

An ablation removes YOLO and feeds full-resolution images directly into DINOv3. For retrieval, DINO only achieves 86.67%, whereas YOLO + DINO achieves 93.33%. For clustering, DINO only obtains $S(f_i) \cap S(f_j) \neq \emptyset$2, $S(f_i) \cap S(f_j) \neq \emptyset$3, $S(f_i) \cap S(f_j) \neq \emptyset$4, $S(f_i) \cap S(f_j) \neq \emptyset$5, and $S(f_i) \cap S(f_j) \neq \emptyset$6, while YOLO + DINO obtains $S(f_i) \cap S(f_j) \neq \emptyset$7, $S(f_i) \cap S(f_j) \neq \emptyset$8, $S(f_i) \cap S(f_j) \neq \emptyset$9, $1280 \times 1280$0, and $1280 \times 1280$1. The paper notes that DINO-only has slightly higher raw precision but lower recall and lower final F1. Its explanation is that, without explicit cropping, the ViT struggles with small and densely packed heads; it may group only the easiest cases confidently while missing many true matches.

A common misconception addressed by this ablation is that sufficiently strong self-supervised features alone should eliminate the need for detection. The reported numbers do not support that view in this domain.

6. Limitations, failure modes, and research significance

The paper states several limitations of Fursee (Wu, 22 Jun 2026). The dataset is relatively small and regionally limited, so generalization to other conventions and styles is uncertain. Performance degrades under heavy occlusion and blurry low-quality images. Another notable failure mode is accessory sensitivity: if the same fursuit appears with different accessories, the model may split it into separate clusters.

These limitations clarify the boundary of the method’s invariances. Fursee is designed to sharpen separation between visually similar identities, but the paper indicates that the resulting embedding space can still be too dependent on surface appearance. A plausible implication is that the learned representation is strong for fine-grained discrimination yet not fully invariant to within-identity costume variation.

The paper lists five main contributions: a dedicated fursuit benchmark dataset for retrieval and clustering, with overlapping identities and detection annotations; a hybrid YOLO-DINOv3 pipeline tailored to crowded convention imagery; metric-learning optimization with ArcFace plus supervised contrastive loss; a clustering method using DBSCAN with silhouette-guided hyperparameter search rather than manual radius tuning; and a custom overlapping clustering evaluation protocol using IoU, Hungarian matching, class weighting, and a structure penalty.

Within the broader space of retrieval and clustering systems, Fursee is best understood as a purpose-built solution for convention photography. YOLO isolates the relevant heads, DINOv3 produces identity embeddings, ArcFace and supervised contrastive learning impose angular discrimination, and DBSCAN groups the resulting points without predefining the number of identities. The paper’s central claim is not that generic multimodal models are unusable, but that they are not dedicatedly optimized for this fine-grained, crowded, open-set visual domain.