---
title: 'ReShapeBench: Multi-Domain Shape Benchmark'
url: https://www.emergentmind.com/topics/reshapebench
type: topic
---

# ReShapeBench: Multi-Domain Shape Benchmark

ReShapeBench is a recurrent benchmark label applied to several technically distinct research artifacts concerned with shape as the primary object of analysis. In the arXiv literature represented here, the name refers to at least four different constructs: a clinical statistical shape modeling validation framework for anatomy [2009.02878], a conformal-bending test suite for continuous optimization [2004.10042], a benchmark for shape-aware image editing [2508.08134], and a non-rigid 3D human-shape retrieval benchmark [2003.08763]. The common thread is not a shared dataset or protocol, but a shared emphasis on how shape variation is represented, transformed, localized, or retrieved under controlled evaluation.

## 1. Nomenclature and domain-specific uses

The name ReShapeBench is therefore polysemous. Each usage defines its own task, data modality, and evaluation protocol.

| Usage | Domain | Core object of evaluation |
|---|---|---|
| Clinical ReShapeBench | Statistical shape modeling | Anatomical correspondences, morphometrics, lesion screening |
| Conformal-bending ReShapeBench | Continuous optimization | Optimizer robustness on bent 2-D landscapes |
| Image-editing ReShapeBench | Generative image editing | Large-scale shape transformation with background preservation |
| Human-shape retrieval ReShapeBench | 3D shape retrieval | Retrieval of non-rigid human meshes |

A common misconception is to treat ReShapeBench as a single benchmark family with interoperable scores. The cited sources do not support that interpretation. Each benchmark is self-contained, and reported metrics are only meaningful within the corresponding task definition and data regime [2009.02878].

## 2. Clinical statistical shape modeling benchmark

In the clinical SSM setting, ReShapeBench denotes a framework for systematically assessing off-the-shelf statistical shape modeling tools in clinical applications. The benchmark operates on 3D binary segmentations of the LAA, scapula, humerus, and femur. Its preprocessing pipeline consists of hole-filling, isotropic resampling, antialiasing, rigid alignment using center-of-mass plus ANTs, signed-distance-transform conversion, and cropping to a common bounding box. ShapeWorks and Deformetrica use sampled particles on the surfaces, whereas SPHARM-PDM uses meshed surfaces with spherical parameterization [2009.02878].

The benchmark compares three modeling paradigms: ShapeWorks as a groupwise particle-based method, Deformetrica as a groupwise diffeomorphic atlas-based method, and SPHARM-PDM as a pairwise spherical harmonics method. Groupwise correspondence approaches optimize all shapes simultaneously to enforce consistent point-to-point mappings across the cohort. The reported consequence is more compact, reproducible statistical models that respect population variability, whereas pairwise methods register each shape independently to a template and may exhibit axis-swapping or correspondence drift [2009.02878].

Quantitative evaluation is defined through PCA-based model diagnostics over retained modes $K$. Compactness is
$$
C(K)=\frac{\sum_{i=1}^K \lambda_i}{\sum_{i=1}^{dM} \lambda_i},
$$
generalization error is
$$
G(K)=\frac1N\sum_{n=1}^N \left\|\mathbf z_n-\hat{\mathbf z}_n(K)\right\|^2,
\qquad
\hat{\mathbf z}_n(K)=\sum_{i=1}^K(\mathbf u_i^T\mathbf z_n)\mathbf u_i,
$$
and specificity is
$$
S(K)=\frac1J\sum_{j=1}^J\min_n \left\|\mathbf z_j^{(\mathrm{synth})}-\mathbf z_n\right\|^2,
\qquad
\mathbf z_j^{(\mathrm{synth})}\sim\mathcal N(\boldsymbol\mu,\mathrm{diag}(\lambda_1,\dots,\lambda_K)).
$$
Training uses 70% of each dataset, with $N\approx 80$–100 shapes; leave-one-out is used for $G(K)$ on the training set; and $J=10{,}000$ synthetic draws are used for $S(K)$ [2009.02878].

Qualitative evaluation complements these metrics. Modes of variation are visualized as $\boldsymbol\mu\pm3\sqrt{\lambda_i}\mathbf u_i$ to assess clinically meaningful deformations such as LAA ostium size, scapular glenoid dilation, humeral lesion depth, and femoral bump. Clustering is performed by applying K-means to the $dM$-dimensional correspondence coordinates. On the LAA dataset, the elbow method found four clusters matching the “chicken-wing,” “wind-sock,” “cactus,” and “cauliflower” types; ShapeWorks and Deformetrica cluster centers aligned closely with expert-defined means, while SPHARM-PDM did not [2009.02878].

The landmark and measurement validation framework begins by annotating $L$ ground-truth landmarks on the mean shape $\boldsymbol\mu$. For each test subject, correspondences are obtained via the pretrained model: ShapeWorks uses TPS warp from the mean, Deformetrica uses atlas deformation, and SPHARM-PDM uses fixed mapping. Mean landmarks are then transferred to subject space using thin-plate spline or Procrustes. The landmark error is
$$
E_{LM}=\frac1L\sum_{i=1}^L\|\mathbf p_i^{\rm pred}-\mathbf p_i^{\rm gt}\|.
$$
Paired two-sided t-tests and equivalence tests at $\alpha=0.05$ are used to assess whether $E_{LM}$ falls below clinically acceptable bounds, with power $\ge 85\%$ [2009.02878].

The lesion screening method projects a pathological shape $\tilde{\mathbf x}$ onto a control PCA subspace while allowing sparse localized offsets along surface normals:
$$
\min_{\boldsymbol\alpha,\Delta\mathbf x}\;\sum_{i=1}^M
\Bigl\|\tilde{\mathbf x}_i - [(\boldsymbol\mu+\mathbf U\boldsymbol\alpha)_i + \Delta x_i\,\boldsymbol\eta_i]\Bigr\|^2
+ \lambda\sum_{i=1}^M|\Delta x_i|.
$$
Offsets with $\Delta x_i\neq0$ localize abnormality. Subject-level abnormality is then quantified by
$$
D_M(\boldsymbol\alpha)=\sqrt{(\boldsymbol\alpha-\mathbf0)^T\Gamma^{-1}(\boldsymbol\alpha-\mathbf0)},
$$
with decision rule $D_M^2>\chi^2_{K,\,0.975}$, and ROC analysis on training offsets calibrates $\lambda$ and the $D_M$ threshold to obtain a desired sensitivity/specificity trade-off [2009.02878].

Reported results at $K=10$ modes show mean compactness, generalization error, and specificity across the four anatomies of $0.82\pm0.05$, $0.09\pm0.02\,\mathrm{mm}^2$, and $0.12\pm0.03\,\mathrm{mm}^2$ for ShapeWorks; $0.75\pm0.08$, $0.11\pm0.03\,\mathrm{mm}^2$, and $0.10\pm0.04\,\mathrm{mm}^2$ for Deformetrica; and $0.60\pm0.10$, $0.14\pm0.05\,\mathrm{mm}^2$, and $0.20\pm0.06\,\mathrm{mm}^2$ for SPHARM-PDM. Landmark or measurement errors are likewise lower for the groupwise methods: for LAA ostium max-diameter, ShapeWorks $0.8\pm0.4$ mm, Deformetrica $1.0\pm0.5$ mm, SPHARM-PDM $2.6\pm1.0$ mm; for scapula glenoid radius, $1.0\pm0.6$ mm, $1.2\pm0.8$ mm, $2.8\pm1.3$ mm; for humerus head radius, $0.6\pm0.3$ mm, $0.7\pm0.4$ mm, $1.5\pm0.9$ mm. Test pathology classification accuracy is reported as $83.2\pm7.8\%$, $83.8\pm8.3\%$, and $55.8\pm8.5\%$ for femur cam-lesion, and $96.0\pm3.3\%$, $90.7\pm12.2\%$, and $95.3\pm2.7\%$ for humerus Hill-Sachs. The stated overall conclusion is that ShapeWorks and Deformetrica outperformed SPHARM-PDM on every quantitative and validation criterion [2009.02878].

The stated limitations are also benchmark-specific: Deformetrica depends on the initial atlas, SPHARM-PDM is constrained to genus-0 surfaces and may misalign axes, and dense ground-truth correspondences remain unavailable. Future work is described as extending ReShapeBench to additional tools such as SlicerSALT, more anatomies, deep-learning-based correspondence estimators, and a fully automated clinical pipeline [2009.02878].

## 3. Conformal-bending test suite for optimization

A different use of ReShapeBench appears in the optimization literature as a family of conformal-bent benchmark functions derived from a “graphic bending” transformation. Here the purpose is not anatomical modeling but the deformation of a benchmark landscape so that the function’s “shape” changes rather than only its orientation. The construction is defined for a twice-continuously differentiable base function $f:\mathbb R^D\to\mathbb R$, with the exposition focused on $D=2$ and on the Cigar function
$$
f_{\rm cigar}(x)=10^{-6}x_1^2+\sum_{i=2}^D x_i^2
$$
over $x\in[-L/2,L/2]^2$ [2004.10042].

The conformal-bending transformation $T$ has three stages. First, a forward box transform maps $x$ to
$$
s_{\rm f}=\bigl(2\xi/L,\,-2\psi/L\bigr),\qquad
o_{\rm f}=(-\xi,\;\psi),\qquad
x'=s_{\rm f}\circ x+o_{\rm f}.
$$
Second, conformal inversion is performed in the complex plane:
$$
z=x'_1+i\,x'_2,\qquad
w=\frac1z=\frac{x'_1}{(x'_1)^2+(x'_2)^2}
-i\,\frac{x'_2}{(x'_1)^2+(x'_2)^2},
$$
followed by
$$
x''=\bigl(\Re w,\,-\Im w\bigr).
$$
Third, an inverse box transform is applied:
$$
s_{\rm i}=\bigl(2\upsilon/L,\,-2\varpi/L\bigr),\qquad
o_{\rm i}=(-\upsilon,\;\varpi),\qquad
x'''=(x''-o_{\rm i})\oslash s_{\rm i}.
$$
The final bent function is
$$
g(x)=f(x'''),
$$
with the constraint that $\|x'\|\neq0$ so that inversion is defined [2004.10042].

This transformation is presented as analytic on $\mathbb C\setminus\{0\}$ with nonzero derivative, locally bijective, and $C^\infty$ except at $x'=0$; accordingly, $g=f\circ T$ remains $C^2$ on its domain. The benchmark text states that any single-modality valley of $f$ becomes a winding ring in $g$ without creating new local minima, because conformal maps preserve topological genus and angle structure. It also states that although Hessian eigenvalue ratios are unchanged in magnitude, the main valley direction becomes curved, which defeats conventional linear-search strategies [2004.10042].

The evaluation procedure adds only $O(1)$ cost per function evaluation and can be wrapped around an existing 2-D optimizer. Parameter sensitivity is reported by varying $\{\xi,\psi,\upsilon,\varpi\}$ one at a time while keeping the others at $1$, and recording the average number of evaluations to reach $f^*+10^{-6}$ over 100 runs with CMA-ES and PSO. Increasing $\xi$ stretches the ring along one axis and can raise CMA-ES ERT by up to a factor of $10^2$; increasing $\psi$ flattens the ring and allows CMA-ES to recover much of its speed; increasing $\upsilon$ shifts the optimum into very thin parts of the ring and produces near-100% failure for both optimizers beyond $\upsilon\approx2$; increasing $\varpi$ shrinks the ring and improves ERT for both CMA-ES and PSO [2004.10042].

On the default 2-D conformal-bent Cigar benchmark with $\xi=\psi=\upsilon=\varpi=1$ and $L=10$, multi-restart CMA-ES and canonical PSO are evaluated with stopping condition $f\le f^*+10^{-6}$ or $10^5$ evaluations. The rotated-vs-bent comparison reports approximately $2\times10^3$ rotated ERT and $3\times10^4$ bent ERT for CMA-ES, with success decreasing from $100\%$ to $82\%$; for PSO, approximately $5\times10^3$ rotated ERT and $1\times10^4$ bent ERT are reported, with success decreasing from $100\%$ to $91\%$. The stated interpretation is that the conformal-bent problem slows both algorithms by roughly one to two orders of magnitude, with larger variance and higher failure rate for CMA-ES [2004.10042].

Practical recommendations are correspondingly explicit: $\xi\approx1$–$2$ and $\psi\approx1$ produce a clear winding valley of moderate width, $\upsilon$ should stay below $1.5$ to avoid almost singular regions, and $\varpi>1$ shrinks the search region and eases the problem. The benchmark is intended to be composable with classical shifts and rotations to generate richer landscapes [2004.10042].

## 4. Shape-aware image editing benchmark

In text-guided image editing, ReShapeBench is introduced as a dedicated benchmark for shape-aware editing, with emphasis on large-scale transformations rather than subtle color or style changes. The dataset contains 120 newly curated images split into two primary subsets, together with a held-out evaluation set of 50 images drawn from the 120 plus selected examples from PIE-Bench. The single-object subset contains 70 images with one well-defined foreground subject per image, and the multi-object subset contains 50 images with 2–5 interacting objects. All images are resized or padded to $512\times512$ pixels [2508.08134].

Prompt design is a central part of the benchmark. Each image is paired with a source and target prompt in a four-sentence format: a general summary of the scene, a foreground object description, a background description, and an overall atmosphere or context. In the edit prompt, only the shape-relevant attributes change. Draft prompts are generated by Qwen-2.5-VL and then manually edited to ensure consistency, clarity, and focus on large-scale shape change. Manual validation enforces completeness of shape-relevant details, consistency of background descriptions between source and target, and explicit emphasis on contour or structural change [2508.08134].

The benchmark taxonomizes transformations into four canonical groups: scaling, rotation and viewpoint shifts, perspective warping, and structural replacement. Approximate distribution across the 120 images is reported as structural replacement $\sim35\%$, scaling $\sim25\%$, rotation or viewpoint shifts $\sim20\%$, and perspective warping $\sim20\%$. Scene complexity is 70 single-object images, or 58%, and 50 multiple-object images, or 42%. Difficulty tiers, defined subjectively on the basis of silhouette divergence in pixel space, are mild for IoU $>0.7$ at 30%, moderate for $0.4<\mathrm{IoU}\le0.7$ at 45%, and extreme for $\mathrm{IoU}\le0.4$ at 25% [2508.08134].

Evaluation is organized around three axes: image fidelity, background preservation, and shape or text alignment. Background preservation includes PSNR,
$$
\text{PSNR}=10\log_{10}\!\biggl(\frac{L^2}{\mathrm{MSE}}\biggr),\qquad
\mathrm{MSE}=\frac1N\sum_{i=1}^N (I_i-\hat I_i)^2,
$$
and LPIPS, with optional background IoU when a ground-truth mask is available:
$$
\mathrm{IoU}=\frac{|M_{\text{preserved}\cap M_{\text{gt}}|}{|M_{\text{preserved}\cup M_{\text{gt}}|}.
$$
The protocol masks out a centered box around the subject to isolate the background region before computing these similarities [2508.08134].

Shape accuracy is measured by IoU between predicted and reference shape masks,
$$
\mathrm{IoU}=\frac{|S_{\text{edit}\cap S_{\text{ref}}|}{|S_{\text{edit}\cup S_{\text{ref}}|},
$$
with off-the-shelf segmentation allowed when reference masks are unavailable. Overall fidelity may be summarized with FID,
$$
\mathrm{FID}(\mathcal N(\mu_r,C_r),\mathcal N(\mu_g,C_g))
=\lVert \mu_r-\mu_g\rVert^2+\mathrm{Tr}\bigl(C_r+C_g-2(C_rC_g)^{1/2}\bigr),
$$
and text-image alignment with CLIP cosine similarity,
$$
\mathrm{CLIP\_Sim}
=\frac{\langle E_{\rm img}(I),E_{\rm txt}(P_{\rm tgt})\rangle}
{\|E_{\rm img}(I)\|\,\|E_{\rm txt}(P_{\rm tgt})\|}.
$$
The paper reports PSNR, LPIPS, CLIP-Sim, and Aesthetic Score, while the benchmark description notes that users often augment these with IoU and FID [2508.08134].

Licensing is specified as CC-BY-NC-SA 4.0, with noncommercial, research-only use and redistribution under the same license. The benchmark is explicitly associated with “Follow-Your-Shape: Shape-Aware Image Editing via Trajectory-Guided Region Control” [2508.08134].

## 5. Non-rigid 3D human-shape retrieval benchmark

In shape retrieval, ReShapeBench denotes the non-rigid human-shape retrieval benchmark introduced in Pickup et al. (2016) and extended with additional training data in [2003.08763]. The benchmark comprises three collections of 3D human meshes: a “Real” set derived from CAESAR scans, a fully “Synthetic” set created in DAZ Studio, and the FAUST public-scan set. Each has disjoint Training and Test splits. The Real set contains 100 training meshes and 400 test meshes; the Synthetic set contains 45 training meshes and 300 test meshes; and FAUST contains 100 training meshes and 200 test meshes. Training and Test are strictly non-overlapping in both subject identity and pose, and no cross-validation beyond the fixed split is prescribed [2003.08763].

Preprocessing and geometric standardization differ by subset. In the Real set, each scan is registered to a common SCAPE template of approximately 15,000 vertices, remeshed to prevent trivial nearest-vertex correspondence, and scaled to a canonical height. Synthetic meshes are remeshed to approximately 60,000 vertices to introduce discretization variation. FAUST consists of high-resolution stereo scans of approximately 172,000 vertices, with real surface noise and topological defects; a watertight version is distributed by automatic hole-filling in MeshLab. Many methods optionally simplify meshes to $O(10^3$–$10^4)$ vertices before descriptor extraction, and the FAUST scans are pre-processed to remove non-manifold edges and fill holes [2003.08763].

More than 25 methods are grouped into several descriptor families: simple global invariants, pose-canonical forms, spectral and manifold-based descriptors, local and mid-level features, learned higher-level features, topological matching, and APT or MAPT variants. Representative formulas include surface area,
$$
A=\sum_{i=1}^N \tfrac12\|(b_i-a_i)\times(c_i-a_i)\|,
$$
compactness,
$$
C=\frac{V^2}{A^3},
\qquad
V=\frac{1}{6}\sum_{i=1}^N a_i\cdot\bigl((b_i-a_i)\times(c_i-a_i)\bigr),
$$
and the biharmonic-distance construction
$$
d^2_\mathrm{bih}(x,y)
=\sum_{i=1}^m \frac{(\phi_i(x)-\phi_i(y))^2}{\lambda_i^2}.
$$
The benchmark also documents Euclidean distance, Jeffrey divergence, Earth Mover’s Distance, manifold-ranking graph-Laplacian similarity, and geodesic path-based deformable-invariant distances as common similarity measures [2003.08763].

Evaluation follows the standard retrieval protocol in which each Test mesh is used as a query and all Test meshes are ranked by distance or similarity. Reported metrics include precision and recall curves, nearest-neighbour accuracy, First-Tier and Second-Tier recall, E-measure, DCG, AUC, AP, and mAP. The benchmark gives
$$
\mathrm{DCG}_k=\sum_{i=1}^k \frac{2^{\mathrm{rel}(i)}-1}{\log_2(i+1)},
$$
$$
\mathrm{AP}=\sum_{n=1}^N (R_n-R_{n-1})P_n,
\qquad
\mathrm{mAP}=\frac1Q\sum_{q=1}^Q \mathrm{AP}_q.
$$
No re-ranking on Test is allowed, and all trainable parameters or learned components must be fixed using the Training data alone [2003.08763].

Results vary by subset. On the Real set, Giachetti et al. APT (trained) reports approximately NN $0.91$, 1-T $0.67$, and 2-T $0.85$, while Litman et al. sparse-dictionary SI-HKS reports approximately NN $0.78$, 1-T $0.66$, and 2-T $0.86$. On the Synthetic set, Li et al. Spectral Geometry reports approximately NN $0.99$ and 1-T $0.83$, while Giachetti et al. APT (trained) reports approximately NN $0.97$ and 1-T $0.81$. On FAUST, Giachetti et al. APT reports approximately NN $0.96$ and 1-T $0.87$, and Tatsuma and Aono MR-BoF-APFH reports approximately NN $0.90$ and 1-T $0.82$ [2003.08763].

Several findings are notable because they run against simple expectations. Precision-recall curves indicate that trained, spectral, and APT-type descriptors consistently lead, but even naïve global invariants such as surface area outperform many complex baselines on Synthetic data, implying that body size and scale are highly discriminative there. The benchmark also reports that models with the same pose but different subject identity account for 10–25% of nearest-neighbour mistakes among the top methods, indicating residual scope for stronger pose invariance [2003.08763].

## 6. Comparative interpretation and recurring issues

Across these usages, ReShapeBench consistently denotes a benchmark in which shape, rather than only appearance or scalar performance, is the controlled variable of interest. In the clinical framework, the central issue is dense correspondence consistency across populations; in conformal bending, it is controlled curvature of an optimization valley; in image editing, it is the fidelity of structural change under background preservation; and in human-shape retrieval, it is invariance to pose, mesh discretization, and scan noise. This suggests a family resemblance at the level of research question, but not at the level of dataset or metric interoperability [2009.02878].

The benchmark-specific limitations are also structurally different. Clinical ReShapeBench lacks dense ground-truth correspondences and includes method-specific constraints such as atlas dependence and genus-0 assumptions. The conformal-bending suite is restricted to 2-D wrapping of base functions and requires avoiding the inversion singularity at $x'=0$. The image-editing benchmark depends on prompt quality, manual validation, and sometimes approximate masks from off-the-shelf segmentation. The retrieval benchmark must control train-test leakage in subject identity and pose, and its own results show that trivial scale cues can be unexpectedly strong [2004.10042].

A plausible implication is that any citation of “ReShapeBench” should be accompanied by an explicit task description or arXiv identifier. Without that disambiguation, claims about performance, realism, compactness, IoU, or mAP are liable to be interpreted against the wrong benchmark family.

Source: https://www.emergentmind.com/topics/reshapebench