---
title: 'LocScore: Localization-Aware Retrieval Metric'
url: https://www.emergentmind.com/topics/locscore
type: topic
---

# LocScore: Localization-Aware Retrieval Metric

LocScore is a localization-aware metric introduced for spatially grounded instance-level image retrieval systems, specifically designed for patch-wise retrieval scenarios where it is essential to assess both retrieval accuracy and the spatial correctness of the matched region. LocScore jointly considers the ranked retrieval position of a true positive and the spatial overlap between the predicted patch and the ground-truth bounding box, thereby providing a diagnostic tool for evaluating the ability of retrieval systems not merely to locate positive images but to accurately localize the target object within them [2512.12610].

## 1. Motivation and Conceptual Foundation

LocScore was developed to address the limitations of traditional retrieval metrics such as average precision (AP) and intersection-over-union (IoU) when applied independently. In patch-wise retrieval, AP quantifies the rank at which ground-truth positives appear but does not measure spatial alignment, while IoU captures the overlap between the predicted and ground-truth regions but is agnostic to retrieval ordering. LocScore integrates these two aspects, weighting each true-positive retrieval by both its precision (proportion of positives retrieved up to that rank) and its IoU with the ground-truth box. High LocScore values indicate both early retrieval and accurate spatial alignment, while degradations in either ranking or localization are penalized.

## 2. Mathematical Definition

LocScore is rigorously defined as follows. For query $n$:

- $I_n$: Number of ground-truth positives for query $n$.
- $B^{n,i}_{\mathrm{gt}}\in\mathbb{R}^4$: Ground-truth bounding box for the $i$-th positive.
- $B^{n,i}_{\mathrm{pred}}\in\mathbb{R}^4$: Predicted bounding box for the highest-scoring patch of the $i$-th positive.
- $r^{n,i}\in\mathbb{N}$: Rank of the retrieved positive image.
- $h^{n,i}$: Number of ground-truth positives among the top $r^{n,i}$ retrieved slots.

The spatial overlap is quantified as:
\[
\mathrm{IoU}(B^{n,i}_{\mathrm{gt}}, B^{n,i}_{\mathrm{pred}}) = \frac{\mathrm{area}(B^{n,i}_{\mathrm{gt}} \cap B^{n,i}_{\mathrm{pred}})}{\mathrm{area}(B^{n,i}_{\mathrm{gt}} \cup B^{n,i}_{\mathrm{pred}})}
\]

**Per-query continuous LocScore:**
\[
\mathrm{LocScore}^{(n)} = \frac{1}{I_n} \sum_{i=1}^{I_n} \left( \frac{h^{n,i}}{r^{n,i}} \times \mathrm{IoU}(B^{n,i}_{\mathrm{gt}}, B^{n,i}_{\mathrm{pred}}) \right)
\]

**Dataset-wide LocScore:**
\[
\mathrm{LocScore} = \frac{1}{N} \sum_{n=1}^{N} \mathrm{LocScore}^{(n)}
\]

**Thresholded LocScore at IoU $\geq\delta$:**
\[
\mathrm{LocScore}^{(n)}(\delta) = \frac{1}{I_n} \sum_{i=1}^{I_n} \frac{h^{n,i}}{r^{n,i}} \times \mathbb{I}\left[ \mathrm{IoU}(B^{n,i}_{\mathrm{gt}}, B^{n,i}_{\mathrm{pred}}) \geq \delta \right]
\]

**Mean-thresholded LocScore ($\mathrm{mLocScore}$) over thresholds $\mathcal{T}=\{0.3,0.4,0.5\}$:**
\[
\mathrm{mLocScore} = \frac{1}{|\mathcal{T}|} \sum_{\delta\in\mathcal{T}} \mathrm{LocScore}(\delta)
\]

## 3. Computation Workflow

For each query:

1. Retrieve a ranked list of candidate images, each associated with a single top-scoring patch and bounding box.
2. Identify all ground-truth positive images.
3. For each positive image:
   - Record its retrieval rank $r^{n,i}$.
   - Compute spatial overlap $\mathrm{IoU}(B^{n,i}_{\mathrm{gt}}, B^{n,i}_{\mathrm{pred}})$.
   - Compute the precision term $h^{n,i} / r^{n,i}$, where $h^{n,i}$ counts the number of positives retrieved up to rank $r^{n,i}$.
4. Aggregate using the formula for $\mathrm{LocScore}^{(n)}$; average across all queries for $\mathrm{LocScore}$.
5. For thresholded variants, replace the IoU factor by an indicator of IoU exceeding $\delta$.

Boxes are represented by $(x_{\min}, y_{\min}, x_{\max}, y_{\max})$, and IoU is computed in $O(1)$. Only the single highest-scoring patch per image is required. Vectorized implementation can accelerate $h^{n,i}$ computation by sorting ranks and applying cumulative sums.

## 4. Properties, Interpretative Range, and Metric Behavior

- LocScore ranges in $[0,1]$ due to $h^{n,i}/r^{n,i}\leq1$ and $\mathrm{IoU}\leq1$.
- The sum $\sum_i h^{n,i}/r^{n,i}$ corresponds exactly to AP for query $n$; thus, $\mathrm{LocScore}\leq\mathrm{AP}$ since $\mathrm{IoU}\leq1$ always.
- Patches with partial localization (e.g. $\mathrm{IoU}=0.5$) contribute proportionally less.
- Sensitivity to misalignment is immediate: small spatial shifts that lower IoU reduce LocScore, even at optimal retrieval rank.
- For thresholded $\delta$, $\mathrm{LocScore}(\delta)$ is non-increasing in $\delta$ by construction.

## 5. Empirical Performance and Comparative Assessment

LocScore enables finer discrimination between retrieval systems. Empirical results on the DINOv2 backbone demonstrate that global methods reach mAP $\approx57.7\%$ but LocScore $\approx15.1\%$, whereas patch-wise local DINOv2 achieves LocScore $\approx22.2\%$. This highlights that global feature matching often correlates with background co-occurrence rather than actual object alignment. Case studies with perfect AP$=1.0$ but differing LocScores further emphasize that AP alone cannot capture spatial correctness. Thresholded LocScore analyses show that sliding-window methods yield superior localization accuracy under stricter IoU compared to coarse grid patch selection [2512.12610].

## 6. Efficient Implementation Techniques

- Compute IoU only for retrieved ground-truth positive images.
- Pre-sort retrieved lists, mask positives, and apply cumulative summation for $h^{n,i}$ calculation.
- For mLocScore across multiple thresholds, compute relevant binary masks in one batch and produce the aggregate.
- For large-scale evaluation, only compute one patch per retrieved image, not all potential alignments.

## 7. Worked Example and Practical Interpretation

Consider a single query with $I_n=2$ positives:

- Rank 1: Image A (ground-truth), $\mathrm{IoU}(A) = 1.0$
- Rank 2: Image X (false positive)
- Rank 3: Image B (ground-truth), $\mathrm{IoU}(B) = 0.5$

For Image A: $r^{n,1}=1$, $h^{n,1}=1$, contribution $=1.0$.
For Image B: $r^{n,2}=3$, $h^{n,2}=2$, contribution $=(2/3)\times0.5\approx0.333$.

Continuous LocScore: 
\[
\mathrm{LocScore}^{(n)} = \tfrac{1}{2}(1.0 + 0.333) = 0.6667
\]

Thresholded LocScore at $\delta=0.6$: only Image A passes, so 
\[
\mathrm{LocScore}^{(n)}(0.6) = \tfrac{1}{2}(1 + 0) = 0.5
\]

This procedure operationalizes LocScore as both a benchmark and a diagnostic, making it widely applicable for spatially sensitive retrieval system development.

---

LocScore represents a metricically rigorous and empirically validated approach for quantifying both retrieval rank and localization fidelity within patch-wise retrieval frameworks, and offers interpretable, spatially structured insight beyond what is possible using standard precision metrics alone [2512.12610].

Source: https://www.emergentmind.com/topics/locscore