Objectness Similarity (OSIM) Metric

• Objectness Similarity (OSIM) is an evaluation metric for 3D scenes that focuses on object-level fidelity by comparing detector features within reference object regions.
• It computes per-object cosine similarities from pre-trained detector layers and aggregates them by class using visual saliency to mimic human attention.
• Empirical results show OSIM achieves higher alignment with human judgments than conventional metrics, highlighting its effectiveness in preserving object recognizability.

Searching arXiv for the cited OSIM-related papers and closely related work. Objectness Similarity (OSIM) is a training-free, detector-feature-based evaluation metric for 3D scenes that explicitly focuses on objects as the primary units of perceptual fidelity (Uchida et al., 11 Sep 2025). It was introduced to evaluate whether rendered or reconstructed scenes preserve object recognizability at the level of individual detected entities, rather than only at the level of pixels, global image structure, or whole-image embeddings. In its canonical formulation, OSIM compares intermediate features from a pre-trained object detector inside reference object regions, aggregates these local similarities by object class across matched views, and weights the resulting class scores by visual saliency. The acronym is not unique in the literature: "oSIM" in object-based visual attention denotes a different histogram-intersection metric over panoptic object bins (Vozniak et al., 19 Jan 2026), while person-search work uses an objectness term inside "OR similarity" rather than the 3D scene metric itself (Yao et al., 2020).

1. Motivation and problem setting

OSIM was proposed for the evaluation of 3D reconstruction and 3D generation systems in settings where human judgments depend strongly on whether scene objects remain recognizable and faithful (Uchida et al., 11 Sep 2025). The motivating observation is that conventional metrics often assess overall image quality rather than object-level fidelity. The paper gives a characteristic failure mode: pixel- or global-feature metrics can rate “background-correct but object-blurry” images higher than “object-sharp but background-blurry” ones, contrary to human preferences.

Within that framing, the limitations of common baselines are sharply differentiated. PSNR is dominated by pixel errors and poorly correlates with perception. SSIM and MS-SSIM focus on structural similarities but not high-level semantics. LPIPS and CLIP-sim capture semantics but aggregate globally and can miss object-local degradations. FID compares global feature distributions rather than object instances. Standard 3D shape metrics such as Chamfer Distance, F-score, and Volume IoU require consistent geometry and can be sensitive to outliers; they do not directly express whether rendered 2D objects are recognizable (Uchida et al., 11 Sep 2025).

OSIM addresses this by operationalizing “objectness” as per-object feature fidelity measured in the representation space of a pre-trained detector. The underlying hypothesis is that 3D scene evaluation should reflect whether each individual object has sufficient feature fidelity to be recognized as its class. A further design choice is saliency weighting, intended to emphasize visually important objects and thereby mirror human attention (Uchida et al., 11 Sep 2025). This suggests that OSIM is not merely an object crop metric; it is an object-centric perceptual score with an explicit prioritization mechanism.

2. Mathematical construction

The OSIM formulation in the 3D-scene paper proceeds through four stages: feature extraction from a detector, per-object similarity, per-class aggregation, and saliency-weighted scene aggregation (Uchida et al., 11 Sep 2025).

Let a pre-trained object detection model be $M$. For matched reference and test images rendered from the same camera pose $i$, extract intermediate feature maps

$F_{\mathrm{ref}}^i, F_{\mathrm{test}}^i \in \mathbb{R}^{W \times H \times D}.$

Reference detections provide 2D bounding boxes for objects $j$, with image-space coordinates $(x_1^{i,j}, y_1^{i,j})$ and $(x_2^{i,j}, y_2^{i,j})$, mapped to feature-space coordinates $(x_1'^{i,j}, y_1'^{i,j})$ and $(x_2'^{i,j}, y_2'^{i,j})$.

The local per-object similarity $r_{i,j}$ is defined as the average cosine similarity over the detector feature vectors inside the mapped box region: $r_{i,j} = \frac{1}{(x_2'^{i,j}-x_1'^{i,j}+1)(y_2'^{i,j}-y_1'^{i,j}+1)} \sum_{x=x_1'^{i,j}}^{x_2'^{i,j}} \sum_{y=y_1'^{i,j}}^{y_2'^{i,j}} \cos\!\big(f_{\mathrm{ref}}^i(x,y), f_{\mathrm{test}}^i(x,y)\big).$ The paper states that cosine similarity is adopted “to ensure the value is normalized between 0 and 1.” It also notes that cosine similarity in practice lies in $i$0, that the manuscript does not show an explicit $i$1 term, and that one may assume detector features keep cosine in $i$2, or use $i$3 if needed (Uchida et al., 11 Sep 2025).

For a class $i$4, let $i$5 be the set of all detected instances across all views whose class label equals $i$6. The per-class similarity is the average

$i$7

Saliency is computed from each reference image using Graph-Based Visual Saliency (GBVS). For each object $i$8 in view $i$9, the average saliency inside the 2D box gives $F_{\mathrm{ref}}^i, F_{\mathrm{test}}^i \in \mathbb{R}^{W \times H \times D}.$0, and the per-class saliency is

$F_{\mathrm{ref}}^i, F_{\mathrm{test}}^i \in \mathbb{R}^{W \times H \times D}.$1

The final scene-level score is the saliency-weighted class average

$F_{\mathrm{ref}}^i, F_{\mathrm{test}}^i \in \mathbb{R}^{W \times H \times D}.$2

OSIM is designed to lie in $F_{\mathrm{ref}}^i, F_{\mathrm{test}}^i \in \mathbb{R}^{W \times H \times D}.$3, and reaches $F_{\mathrm{ref}}^i, F_{\mathrm{test}}^i \in \mathbb{R}^{W \times H \times D}.$4 when all per-object similarities equal $F_{\mathrm{ref}}^i, F_{\mathrm{test}}^i \in \mathbb{R}^{W \times H \times D}.$5 (Uchida et al., 11 Sep 2025).

A central technical property of this construction is that it does not solve an explicit instance-matching problem across views. Instead, it relies on reference detections, computes regionwise similarity in corresponding matched views, and aggregates by object class. This is a deliberate simplification rather than a missing implementation detail (Uchida et al., 11 Sep 2025).

3. Computational pipeline and implementation

The metric is training-free and uses off-the-shelf components (Uchida et al., 11 Sep 2025). In the reported implementation, the detector is YOLOX-x, with 99.1M parameters and mAP 51.5. Reference detections are kept above confidence $F_{\mathrm{ref}}^i, F_{\mathrm{test}}^i \in \mathbb{R}^{W \times H \times D}.$6, and intermediate features are extracted from the backbone layer “dark5,” which was chosen after ablation. Visual saliency is obtained with GBVS.

Component	Setting in the paper	Function
Detector	YOLOX-x, confidence threshold 0.35	reference boxes and intermediate features
Feature layer	dark5	per-object feature comparison
Saliency model	GBVS	saliency weights for class aggregation

The operational pipeline is straightforward. For each matched reference/test view pair, the detector runs on the reference image to produce boxes and class labels, while the same detector extracts dark5 feature maps from both the reference and test images. Each reference detection is mapped to the feature grid, and $F_{\mathrm{ref}}^i, F_{\mathrm{test}}^i \in \mathbb{R}^{W \times H \times D}.$7 is computed by averaging cosine similarity within that region. In parallel, GBVS is applied to the reference image, and the average saliency inside each detection box yields $F_{\mathrm{ref}}^i, F_{\mathrm{test}}^i \in \mathbb{R}^{W \times H \times D}.$8. After collecting all such tuples across views, the method aggregates by class to obtain $F_{\mathrm{ref}}^i, F_{\mathrm{test}}^i \in \mathbb{R}^{W \times H \times D}.$9 and $j$0, then computes the saliency-weighted average OSIM (Uchida et al., 11 Sep 2025).

Several implementation choices are explicit. OSIM uses 2D detections per view; there is no 3D bounding-box inference or 3D association. Test renders are taken from the same camera poses as the references. Runtime is dominated by a single detector forward pass per image, which provides both detections and features, plus a GBVS saliency pass per reference image. The per-object similarity itself is a vectorized per-pixel cosine computation inside box regions, and the total cost scales linearly with the number of views and detected boxes (Uchida et al., 11 Sep 2025).

The reported practical guidance is similarly concrete: use YOLOX-x, dark5, GBVS, matched poses including elevation as well as azimuth for 3D objects, and report not only overall OSIM but also per-class $j$1 values for analysis. OSIM is intended to be computed alongside PSNR, SSIM, LPIPS, CLIP-sim, and geometry metrics such as CD, F-score, and Volume IoU, rather than as a universal replacement for them (Uchida et al., 11 Sep 2025).

4. Empirical behavior and evaluation results

The principal empirical claim for OSIM is stronger alignment with human judgments than conventional baselines under the paper’s standardized protocol (Uchida et al., 11 Sep 2025). The user study involved 23 participants. For reconstruction, 7 Mip-NeRF360 scenes and 12 models produced 84 samples and a total of 230 evaluations. For generation, 30 GSO objects and 6 models produced 180 samples and a total of 460 evaluations.

The headline results are Spearman correlations with mean opinion scores (MOS). For reconstruction, the whole-image baselines achieved PSNR $j$2, SSIM $j$3, LPIPS $j$4, and CLIP-sim $j$5. When restricted to bbox patches, the corresponding values were PSNR $j$6, SSIM $j$7, LPIPS $j$8, and CLIP-sim $j$9, while OSIM reached $(x_1^{i,j}, y_1^{i,j})$0, the highest reported correlation in that setting. For generation, OSIM reached $(x_1^{i,j}, y_1^{i,j})$1, compared with image-based baselines PSNR $(x_1^{i,j}, y_1^{i,j})$2, SSIM $(x_1^{i,j}, y_1^{i,j})$3, LPIPS $(x_1^{i,j}, y_1^{i,j})$4, CLIP-sim $(x_1^{i,j}, y_1^{i,j})$5, and FID $(x_1^{i,j}, y_1^{i,j})$6, and with 3D shape metrics CD $(x_1^{i,j}, y_1^{i,j})$7, Volume IoU $(x_1^{i,j}, y_1^{i,j})$8, and F-score $(x_1^{i,j}, y_1^{i,j})$9 (Uchida et al., 11 Sep 2025).

These results are complemented by several analyses. Average Pearson correlations between OSIM and conventional metrics were reported to be lower than among conventional metrics themselves, which the paper interprets as evidence that OSIM supplies a complementary, object-centric perspective. In a degradation analysis where Gaussian blur was applied across objects, OSIM decreased steadily as more objects were blurred, with particularly strong drops when salient objects were degraded. CLIP-sim sometimes stayed unchanged or increased in the same manipulation, illustrating OSIM’s sensitivity to object-local degradations that mattered to observers (Uchida et al., 11 Sep 2025).

Ablations on detector layer choice showed a marked effect. The reported Spearman correlations with MOS were, for reconstruction/generation respectively: dark3 $(x_2^{i,j}, y_2^{i,j})$0, dark4 $(x_2^{i,j}, y_2^{i,j})$1, dark5 $(x_2^{i,j}, y_2^{i,j})$2, p3 $(x_2^{i,j}, y_2^{i,j})$3, p4 $(x_2^{i,j}, y_2^{i,j})$4, and p5 $(x_2^{i,j}, y_2^{i,j})$5. High-level backbone features from dark5 were therefore recommended (Uchida et al., 11 Sep 2025).

The metric was also used to re-evaluate recent reconstruction methods on Mip-NeRF360. Reported OSIM values included Zip-NeRF $(x_2^{i,j}, y_2^{i,j})$6, Mip-NeRF360 $(x_2^{i,j}, y_2^{i,j})$7, GOF $(x_2^{i,j}, y_2^{i,j})$8, 3DGS $(x_2^{i,j}, y_2^{i,j})$9, Mip-Splatting $(x_1'^{i,j}, y_1'^{i,j})$0, Scaffold-GS $(x_1'^{i,j}, y_1'^{i,j})$1, NerfStudio $(x_1'^{i,j}, y_1'^{i,j})$2, and INGP $(x_1'^{i,j}, y_1'^{i,j})$3. The paper notes that MOS mirrored high OSIM for the top methods, citing for example Mip-NeRF360 MOS $(x_1'^{i,j}, y_1'^{i,j})$4 and Zip-NeRF MOS $(x_1'^{i,j}, y_1'^{i,j})$5 (Uchida et al., 11 Sep 2025). This suggests that OSIM can alter the interpretation of progress by highlighting object-level trade-offs that are less visible in global metrics.

5. Relation to other metrics and homonymous uses

Within 3D-scene evaluation, OSIM is positioned against both global image metrics and geometry-based metrics. Unlike LPIPS or CLIP-sim, it restricts comparison to object regions automatically found by a detector, builds an object- and class-level summary, and weights objects by saliency (Uchida et al., 11 Sep 2025). Unlike geometry metrics, it does not require explicit 3D correspondence quality to determine whether objects are recognizable in rendered 2D views. A plausible implication is that OSIM is best understood as an object-centric perceptual metric rather than as a surrogate for photometric accuracy or geometric consistency.

The name, however, overlaps with other object-level similarity formulations in different subfields.

Name	Domain	Core definition
OSIM	3D scene evaluation	saliency-weighted aggregation of detector-feature similarities
oSIM	visual attention prediction	histogram-intersection over object-level attention distributions
OR similarity	person search	visual similarity multiplied by objectness and repulsion

In "ObjectVisA-120: Object-based Visual Attention Prediction in Interactive Street-crossing Environments" (Vozniak et al., 19 Jan 2026), the metric is called object-based similarity, abbreviated oSIM rather than OSIM. It operates on saliency maps and panoptic segmentation. Given L1-normalized predicted and ground-truth saliency maps, attention mass is aggregated inside panoptic object masks to form categorical distributions over objects, and the score is

$(x_1'^{i,j}, y_1'^{i,j})$6

This oSIM measures whether a model assigns attention to the same objects as humans, is invariant to within-object spatial shifts, and largely de-emphasizes object area (Vozniak et al., 19 Jan 2026). Although conceptually object-centric, it is mathematically and operationally distinct from the detector-feature-based OSIM for 3D scenes.

In "Joint Person Objectness and Repulsion for Person Search" (Yao et al., 2020), the relevant construct is OR similarity,

$(x_1'^{i,j}, y_1'^{i,j})$7

where the objectness term is

$(x_1'^{i,j}, y_1'^{i,j})$8

That work explicitly states that if “OSIM” denotes weighting visual similarity by objectness, then OR similarity subsumes OSIM as its objectness component and extends it with a repulsion term (Yao et al., 2020). Here, “objectness similarity” refers to inference-time reweighting in person search rather than to scene-level evaluation. The shared terminology can therefore be misleading unless the domain and formal definition are made explicit.

6. Limitations, edge cases, and future directions

The most important limitation of OSIM is dependence on the detector (Uchida et al., 11 Sep 2025). Only classes within the detector’s label space can be scored, and missed detections—especially for very small or occluded objects—reduce coverage. Multiple instances of the same class in a scene remain challenging because the method does not distinguish instance identity across views. The paper reports that the detector was “fairly robust to overlapping objects,” but also states that multiple instances of the same class in one scene remain difficult because there is no instance tracking.

A second limitation is domain bias. Detector features inherit the biases of the detector’s training data, and performance may degrade in domains far from that distribution. A third is intentional background insensitivity: OSIM does not penalize background noise strongly, because it is designed to focus on object fidelity. This is a feature of the metric’s design rather than a hidden defect, but it means that OSIM should be paired with complementary global metrics for comprehensive assessment (Uchida et al., 11 Sep 2025).

Edge cases are also explicitly addressed. If a scene has no detected objects, which the paper describes as rare because it filtered to scenes with detectable objects, the recommended behavior is to return “not applicable” or fall back to unweighted per-image cosine on the full frame. For classes unseen by the detector, manual regions of interest may be used to compute $(x_1'^{i,j}, y_1'^{i,j})$9 and include them as pseudo-classes (Uchida et al., 11 Sep 2025). These are practical accommodations rather than part of the core definition.

The future directions named in the paper are extension to 2D IQA datasets and to 4D dynamic scenes or video, use of OSIM as a diagnostic tool to drive object-wise refinement loops, and possible exploration of more advanced instance association across views if needed (Uchida et al., 11 Sep 2025). This suggests a broader research trajectory in which object-centric evaluation is treated not only as a reporting metric but also as a mechanism for analysis and possibly optimization.