---
title: Instance Affinity (IA) Score
url: https://www.emergentmind.com/topics/instance-affinity-ia-score
type: topic
---

# Instance Affinity (IA) Score

An Instance Affinity (IA) score is a learned or computed quantity that quantifies the likelihood that two entities—pixels, voxels, columns (“pillars”), embeddings, or region proposals—belong to the same instance within a scene. IA scores are central to a wide array of instance segmentation, proposal-free grouping, and domain adaptation architectures across 2D, 3D, and domain adaptation settings. The precise definition, prediction, and usage of IA scores vary by context, but all share the common principle of facilitating fine-grained, sample- or region-level association for reliable instance-level modeling.

## 1. Core Definitions and Mathematical Formulations

Several mathematical formalizations for the Instance Affinity score exist, depending on the setting:

- **Similarity-based (domain adaptation):** For unsupervised domain adaptation, the IA score $\phi(f_i, f_j)$ between feature embeddings is often defined as an unnormalized similarity:
  $$
  \phi(f_i, f_j) = \frac{1}{1 + \|f_i - f_j\|^2}
  $$
  where $f_i, f_j$ are deep feature vectors and $\phi$ encodes how close two samples (source or target) are in feature space [2104.01286].

- **Pixel/voxel/pillar affinity (segmentation):** In grid-based segmentation, for two spatial locations $u,v$ (pixels, voxels, or LiDAR pillars), the IA score is typically a learned probability:
  $$
  \text{IA}(u, v) = \sigma\left(\text{score}(u, v)\right) \in (0, 1)
  $$
  where $\sigma$ is the sigmoid or softmax function applied to a neural network output [1902.04478, 1909.01616, 2204.08744, 1811.10870].

- **Instance-level recall (metric evaluation):** In medical image segmentation, the IA score is used for evaluation, defined as the fraction of ground-truth instances matched (up to IoU threshold $\tau$) by at least one prediction:
  $$
  \mathrm{IA} = \frac{1}{N_{GT}} \sum_{i=1}^{N_{GT}} \mathbb{I}\left(\max_j \mathrm{IoU}(G_i, P_j) \ge \tau\right)
  $$
  where $G_i$ are ground-truth instances, $P_j$ are predictions, and $\mathbb{I}$ is the indicator function [2511.22911].

This table summarizes main IA paradigms:

| Context         | Definition (Summary)                                   | Usage                                    |
|-----------------|-------------------------------------------------------|------------------------------------------|
| Domain Adaptation [2104.01286] | Feature-space similarity (via $\phi(f_i, f_j)$) | Drives contrastive domain alignment      |
| Proposal-free Segmentation [1902.04478, 1909.01616, 1811.10870] | Pixel/voxel/pillar pairwise probability   | Graph structuring, clustering            |
| Instance Segmentation Metric [2511.22911] | Fraction of GT instances recalled (IoU-based) | Evaluation of detection/segmentation     |

## 2. Architectures and Methodologies Leveraging IA Scores

### Domain Adaptation via Instance Affinity
In unsupervised domain adaptation, ILA-DA [2104.01286] deploys an IA score as the central linkage between source and target domains. For each instance, the affinity is measured in deep feature space, enabling:

- Pseudo-labeling of target samples via k-nearest neighbors, according to $\phi(f_j^t, f_i^s)$.
- Construction of an affinity matrix $A$ encoding “same” (+1) or “different” (–1) label relation for source–target pairs.
- Filtering low-confidence associations by a ratio test on affinity-weighted neighborhoods.
- Optimization through a multi-sample contrastive loss that encourages intra-class feature cohesion and inter-class separation.

### Proposal-free Instance Segmentation and Affinity Clustering
Several proposal-free segmentation methods rely on per-pixel, per-voxel, or per-pillar IA predictions:

- **SSAP [1909.01616]:** Utilizes an affinity pyramid at multiple U-Net decoder scales; each pixel predicts a probability ($\in$ (0,1)) that it belongs to the same instance as its $r \times r$ neighbors. These affinities are symmetrized and used in a three-stage, pyramid-graph minimum-cost multicut to yield instance clusters.
- **3D Sparse Convolution (MASC) [1902.04478]:** Predicts voxel-level affinities for six nearest neighbors across multiple resolutions; these are averaged across nodes and scales to define graph-based node affinities for iterative clustering, with instance merging governed by affinity thresholding.
- **Pixel Affinity Graph Merge [1811.10870]:** Each pixel estimates affinity to 56 spatially distributed neighbors; instance masks are produced by greedy threshold-based graph merging, modulated by semantic similarity.

### LiDAR Panoptic Segmentation
Proposal-free Lidar panoptic segmentation [2204.08744] derives pillar-level IA as the softmax probability that a pillar continues a prior instance vs. starts a new one. Clustering occurs through a local, memory-constrained propagation along scanlines, stiching “thing” class pillars into instance masks.

## 3. Supervision and Loss Functions for IA Score Learning

Supervision of IA predictions is framed as binary (or multi-class) classification, with losses applied at either the pixel pair, voxel pair, or region pair level:

- **Binary Cross-Entropy:** Each affinity prediction $a_{u,v}$ is trained with respect to a binary label $y_{u,v}$ (same/different instance), via
  $$
  L_\text{aff} = -\sum_{(u,v)} [y_{u,v} \log a_{u,v} + (1-y_{u,v}) \log (1-a_{u,v})].
  $$

- **Squared Loss (SSAP):** L2 loss is used per-pixel per-level:
  $$
  L_a^k = \frac{1}{r^2} \sum_{v \in W_r(u)} (a^k_{u,v} - y^k_{u,v})^2
  $$
  with imbalance handling via sample dropping and up-weighting mixed-label windows.

- **Multi-Sample Contrastive Loss (ILA-DA):** ILA-DA’s MSC loss encourages high affinity between features of the same class and low affinity otherwise, using positive/negative sampling as guided by the affinity matrix [2104.01286].

- **Lovász Loss:** Used in LiDAR segmentation [2204.08744], the Lovász-softmax surrogate directly optimizes Jaccard-like metrics for IA scores.

## 4. Clustering, Partitioning, and Use of IA in Inference

The usage of IA scores during inference typically follows a clustering or partitioning phase:

- **Graph Partitioning:** Affinities become edge weights in a graph; graph partitioning methods (minimum-cost multicut [SSAP], greedy merging [1811.10870, 1902.04478]) then segment the space into discrete instances, with hierarchy applied for efficiency and granularity [1909.01616].
  
- **Sequential Propagation:** In pillar-based LiDAR segmentation [2204.08744], sequential traversal and a local memory structure propagate instance IDs, using thresholded IA probabilities to decide instance continuation or new instance initiation.
  
- **Semantic Modulation:** Affinities can be refined with semantic similarity measures (e.g., via Jensen–Shannon divergence [1909.01616], or inner-product over semantic class probabilities [1811.10870]) to suppress unlikely associations and improve cluster purity.

- **Instance-level Metrics:** For evaluation, the IA score is computed as the instance-level recall (fraction of ground-truth instances with a prediction of IoU ≥ $\tau$), directly reflecting model effectiveness at finding all relevant objects [2511.22911].

## 5. Interplay with Other Instance Segmentation Metrics

Instance Affinity is both a learned association signal and a metric. Comparison with other evaluation measures highlights its specificity:

- **IA score (segmentation metric) [2511.22911]:** Directly measures instance-level recall under IoU thresholding, unaffected by over-prediction (splitting) or merged predictions except as limited by best match; insensitive to shape refinement once IoU passes the threshold.
- **Dice / IoU (standard):** Sensitive to precise mask shape and boundary accuracy, but can obscure error modes such as merged/fragmented instances, which IA score detects.
- **Average Precision (AP):** Incorporates both detection and precision, but not always directly interpretable as the capacity to find all relevant instances in clinical or structural contexts.

These distinctions drive the adoption of IA as a primary metric in medical image instance segmentation challenges, especially where the correct separation of adjacent or overlapping structures is essential.

## 6. Empirical Effects and Results

Integration of IA scores into training and inference yields quantifiable improvements:

- **Domain Adaptation:** Adding the MSC loss driven by IA to DANN increases digit adaptation accuracy from 89.3% to 93.8%; for Office-31, from 93.1% to 94.9%; Birds-31, from 82.2% to 86.0% [2104.01286].
  
- **Segmentation Benchmarks:** Instance-level architectures based on IA prediction outperform proposal-based baselines. SSAP achieves a 9% relative AP gain using its affinity pyramid and cascaded partitioning [1909.01616]. MASC achieves AP@0.5IoU of 0.447 versus 0.382 for 3D-SIS [1902.04478].

- **Medical Imaging:** Top semi-supervised methods at STS 2024 increased IA score on OPG from ~44% (nnU-Net) to ~88.5% by incorporating detection-first pipelines, SAM-based refinements, and robust pseudo-labeling [2511.22911].

- **Ablations:** Removal or restriction of IA mechanisms leads to dramatic declines in instance grouping performance and slower inference [1909.01616, 1811.10870].

## 7. Limitations, Considerations, and Evaluation

Key strengths of IA-based approaches include robust handling of instance separation (e.g., in presence of mergers and splits) and architectural simplicity (direct, proposal-free affinity learning). However, binary or hard-thresholded IA metrics are insensitive to incremental mask improvements once past the IoU cutoff and may not reflect precise shape quality [2511.22911]. Moreover, final instance quality often depends on the integration of affinity, semantic, and global constraints; bottlenecks can shift from affinity prediction to semantic understanding as backbone architectures mature [2204.08744].

A plausible implication is that future methods may integrate soft/hierarchical affinity modeling with continuous instance quality measures and class-aware association signals, to further enhance both separability and discriminative segmentation quality.

Source: https://www.emergentmind.com/topics/instance-affinity-ia-score