SHREC 2025: 3D Shape Retrieval Benchmark

• SHREC 2025 Track is a benchmark initiative presenting tasks for precise 3D shape retrieval and segmentation across triangle meshes and protein surfaces.
• It incorporates innovative algorithms that integrate geometric, photometric, physicochemical, and topological descriptors to tackle high-resolution, complex datasets.
• Robust evaluation metrics, including segmentation accuracy, mAP, and balanced accuracy, reveal both current strengths and ongoing challenges in the field.

The SHREC 2025 Track encompasses a set of benchmark tasks and analyses devoted to shape retrieval and segmentation of complex patterns embedded in 3D data, focusing on both geometric reliefs on triangle meshes and protein surface shapes with electrostatic potentials. The initiative provides realistic, large-scale datasets designed to stimulate methodological advances across computer vision and bioinformatics, emphasizing robust discrimination under challenging, high-resolution scenarios. Participating teams proposed a variety of algorithms exploiting geometric, photometric, physicochemical, and topological descriptors, with rigorous evaluation revealing both the strengths and unresolved issues in current retrieval and segmentation pipelines.

1. Benchmark Track Structure and Objectives

SHREC 2025 comprised multiple specialized tracks reflecting the growing complexity and diversity of 3D shape analysis. Two tracks are central:

• Multiple Relief Patterns: Focused on recognizing and segmenting relief patterns encoded on synthetically generated triangle meshes. The goal is precise per-face annotation and retrieval of surface-embedded textures, with applications extending to cultural heritage and industrial inspection.
• Protein Surface Shape Retrieval including Electrostatic Potential: This track extends classical shape retrieval by assessing the contribution of electrostatic potential—a critical physicochemical property—to accurate classification and matching of protein surfaces. The benchmark reflects natural data imbalance and includes surfaces from diverse experimental modalities (NMR, X-ray, cryo-EM).

Both tracks aim to assess state-of-the-art techniques, expose algorithmic limitations, and lay groundwork for methodology beyond geometric descriptors.

2. Methodologies Proposed by Track Participants

Multiple Relief Patterns

Two representative methods defined the methodological landscape:

Method Name	Key Steps	Core Descriptor/Architecture
KU‑3DSeg	Random face sampling, geometric feature extraction, LSCM UV mapping, ResNet‑50 fine-tuning, graph-based label propagation	Homogenized geometric features mapped to UV grid; ResNet‑50; adjacency graph for face label refinement
OMVMLP	Six orthographic projections, VGG19 feature extraction (partial freezing), MLP multi-label classification, retrieval with membership matrix	Global image-based 2D descriptor; VGG19; final multi-sigmoid output; learns via cross-entropy and cosine annealing

KU‑3DSeg operates at the per-face granularity but relies on random sampling due to the prohibitive size of the meshes (up to 200,000 faces), followed by adaptive geometric neighborhood analysis and label propagation designed to correct ambiguities at relief boundaries. OMVMLP leverages multi-view 2D projections and global image descriptors for retrieval, capturing the surface from six directions and synthesizing features in a Multi-Layer Perceptron.

Protein Surface Shape Retrieval

Nine teams contributed fifteen distinct pipelines utilizing a breadth of data representations:

• Point cloud approaches: Fast Point Feature Histograms (FPFH) aggregated for SVM classification (Tatsuma); RIConv++ and RISurProtNet incorporating rotation-invariant features and Transformer-based processing (Barisin, Li et al.).
• Image-based methods: Multi-view 2D projections rendered from the 3D shape, followed by convolutional neural networks such as GoogLeNet or ViT (Peng & Deng, Marco Guerra et al.).
• Topological descriptors: Alpha complex filtrations, persistence images, and statistical vectorizations.
• Physicochemical separation: 3D Zernike descriptors with surface potential bifurcation into “positive” and “negative” regions (Kagaya, Park, Kihara).
• Graph and community detection: ProtoCluster, where surface meshes form community-based subgraphs analysed by graph convolutions (Yang).
• Self-supervised architectures: 3D-PROSPER combines autoencoding for shape embedding and multitask learning with Chamfer and classification loss (He et al.).
• Learning paradigms: Siamese deep metric learning with triplet loss and LightGBM gradient boosting (Tehrani et al.).

The use of electrostatic potential as an additional descriptor constituted a core innovation, especially for class separation in the presence of ambiguous or sparse geometric cues.

3. Dataset Design and Annotation Protocols

Relief Pattern Track

Meshes are generated synthetically to support highly precise ground-truth per-face annotation. The dataset comprises a query, retrieval, and training set. Models include intricate variations in relief scale and texture, simulating realistic scenarios such as those found in manufacturing and preservation contexts.

Protein Surface Track

The protein dataset contains 11,565 surfaces across 97 functionally meaningful classes, constructed by clustering proteins from the PDB at 100% and 90% sequence identity, ensuring homologous grouping. Surface meshes are supplemented with electrostatic potential computed by a two-step protocol: charge assignment via PDB2PQR, then APBS with the TABI-PB solver for calculation on solvent-excluded surfaces generated using NanoShaper. The data is split into 9,244 training and 2,311 test cases, and contains instances from NMR, X-ray, and cryo-EM.

4. Evaluation Metrics and Performance Analysis

Relief Patterns

Evaluation emphasizes:

• Segmentation accuracy: Localized per-face predictions and qualitative segmentation (KU‑3DSeg)
• Retrieval accuracy: OMVMLP assessed via Nearest Neighbor, First/Second Tier, mean Average Precision (mAP), normalized Discounted Cumulated Gain (nDCG), e-measure, and AUC. For OMVMLP the reported AUC is 0.523, marginally surpassing random performance—attributed to omission of occluded or subtle relief regions in global view-based descriptors.

Protein Surface

Five main metrics are used for class prediction:

Metric	Definition / Formula	Significance
Accuracy	Proportion of correct predictions	Overall method correctness
Balanced Accuracy	Accuracy adjusted for class imbalance	Robustness in imbalanced scenarios
F1 Score	$F1 = \frac{2TP}{2TP + FP + FN}$	Harmonic mean of precision and recall
Precision	$TP/(TP+FP)$	Correctness of positive predictions
Recall	$TP/(TP+FN)$	Sensitivity to true positives

Reported accuracy rates ranged from 71% to 93%, and F1 scores from 66% to 92%, barring outliers from technical or training issues. Notably, augmenting geometric descriptors with electrostatic potential led to a consistent performance improvement (typically 1–5% increases in accuracy), particularly in under-represented or ambiguous classes.

5. Technical Challenges and Limitations

Several critical limitations were identified:

• Computational complexity: Processing the entirety of high-resolution meshes necessitates trade-offs (random sampling, reduced input size), which can compromise segmentation completeness (as in KU‑3DSeg).
• Ambiguous boundaries: Relief features that cross pattern borders can induce label confusion during propagation steps.
• Local vs. global representations: Relief and protein surface characteristics can span scales from fine texture to gross deformation. Global descriptors may miss localized or occluded features (see OMVMLP’s full-object orthographic view limitation).
• Generalization to unseen classes: The diversity in class occurrence and training/test splits makes adaptation to new texture or surface classes a persistent open challenge.
• Dataset imbalance: Protein classes are distributed unevenly, testing the resilience and fairness of models.

6. Future Research Directions

Emergent research avenues highlighted:

• Direct 3D methods: Development of geometric deep learning techniques that operate natively on mesh or point cloud data, bypassing 2D projection bottlenecks.
• Foundation model adaptation: Exploring extensions of visual foundation models (CLIP, DINOv2, iBOT) to 3D domains, possibly through multimodal fusion or integrating specialized 3D backbones.
• Smarter projection selection: For view-based methods, optimizing perspective selection and aggregating intermediate network activations to better represent occluded or subtle detail.
• Hybrid feature models: Integrating global shape, local geometric, and physicochemical cues for robust pattern discrimination.
• Ablation and representation fusion: Systematic studies to quantify contributions of descriptors and design more effective fusion strategies.
• Handling data imbalance: Investigation of architectures and training strategies resilient to class sparsity and imbalance.

These directions are motivated by both the limitations of current pipelines and the observed performance gains when combining heterogeneous features.

7. Resources and Benchmark Accessibility

All detailed datasets, ground truth annotations, and evaluation code are publicly available:

• Relief Pattern Track: https://sites.google.com/unifi.it/shrec25-relief-pattern
• Protein Surface Track: Embedded surface meshes, electrostatic potential maps, and processing protocols provided per the official track report.

Both benchmarks are designed to be foundational for future methodological advances and robust validation in high-resolution, multi-descriptor 3D shape analysis.