Boundary Accuracy Evaluation Metrics

• Boundary Accuracy Evaluation Metrics are quantitative measures that assess the fidelity of predicted boundaries to reference boundaries using spatial and temporal distances.
• They incorporate methods like Hausdorff distance, Boundary F1-score, and Boundary IoU to capture misalignments, considering noise, fuzziness, and multi-reference agreements.
• Their application in fields such as medical imaging and time series analysis improves segmentation evaluation by addressing inter-annotator variability and precise boundary localization.

Boundary accuracy evaluation metrics quantify the fidelity of predicted boundaries with respect to reference boundaries across a spectrum of domains including medical image segmentation, generic image segmentation, boundary detection, time series event detection, and sequence segmentation. These metrics are critical in applications where precise boundary localization, rather than mere region overlap or classification correctness, governs downstream scientific or operational decisions. Boundary accuracy evaluation incorporates both classical mathematical distances (such as the Hausdorff family), set/contour-based alignment measures (e.g., boundary F1, Boundary IoU), temporally- or spatially-fuzzy association schemes, and risk-based or psychometric reliability assessments.

1. Formal Definitions of Boundary Accuracy Metrics

Hausdorff Distance and Robust Variants

In medical image segmentation and related fields, the classic Hausdorff distance is defined over the sets of boundary points from the ground truth ($A$) and the prediction ($B$), using Euclidean distance $d(a,b)=\lVert a-b\rVert$:

• Directed Hausdorff: $h(A,B) = \sup_{a\in A} \inf_{b\in B} d(a,b)$
• Symmetric Hausdorff: $H(A,B) = \max\{h(A,B), h(B,A)\}$

To address pathological sensitivity to outliers, the 95th-percentile Hausdorff distance (HD95) is widely adopted:

• $D_{A\to B} = \{\inf_{b\in B} d(a, b)\mid a\in A\}$, $D_{B\to A} = \{\inf_{a\in A} d(b, a)\mid b\in B\}$
• $$\mathrm{HD}_{95}(A, B) = \max\{P_{95}(D_{A\to B}), P_{95}(D_{B\to A})\}$$ with $P_{95}$ the 95th percentile

Average symmetric distances, such as Average Hausdorff Distance (AHD) or Average Symmetric Surface Distance (ASSD), are also utilized:

• $$\bar d(A,B) = \frac{1}{|A|}\sum_{a\in A} \inf_{b\in B} d(a, b)$$
• $$\mathrm{AHD}(A,B) = \max\{\bar d(A,B), \bar d(B,A)\}$$

These metrics directly quantify the maximal or mean boundary misalignment in the image domain, with physical units governed by voxel spacing (Müller et al., 2022).

Discrete and Fuzzy Boundary Match Measures

Boundary F1-score (BF$_1$), as in semantic segmentation, defines boundary precision and recall using indicator tests that a boundary point in one set is within a tolerance $\theta$ of the other:

• Boundary-precision: $$P^c = \frac{1}{|B^c_{pd}|} \sum_{x \in B^c_{pd}} [d(x, B^c_{gt}) < \theta]$$
• Boundary-recall: $$R^c = \frac{1}{|B^c_{gt}|} \sum_{x \in B^c_{gt}} [d(x, B^c_{pd}) < \theta]$$
• $BF^c_{1} = \frac{2 P^c R^c}{P^c + R^c}$

A differentiable surrogate loss is constructed by softmax-pooling and band dilation, suitable for neural network training (Bokhovkin et al., 2019).

Boundary IoU (Intersection over Union) is defined as the overlap between $d$-pixel-thick regions around each mask's boundary:

• $$d_G = \{ x \in \Omega \mid dist(x, \partial G) \leq d \} \cap G$$ ($G$: ground truth mask)
• $$d_P = \{ x \in \Omega \mid dist(x, \partial P) \leq d \} \cap P$$ (Prediction)
• $$\mathrm{BoundaryIoU}(G, P) = \frac{|d_G \cap d_P|}{|d_G \cup d_P|}$$ (Cheng et al., 2021)

This provides a symmetric, continuous, and scale-balanced grading of boundary alignment.

2. Boundary Evaluation in Noisy, Ambiguous, or Multi-Reference Settings

Segmentation Similarity and Agreement

Segmentation Similarity $S$ replaces simple pairwise count-based metrics with a normalized edit distance between token-level boundary vectors, allowing for weight-adjusted substitutions (boundary mismatches) and parameterized transpositions for near misses:

• $$S(s_1, s_2) = [t \cdot mass(i) - t - d(s_1, s_2; T)]/[t \cdot mass(i) - t]$$, where $d$ embeds both full-miss and near-miss penalties (Fournier et al., 2012).

$S$ can be inserted into inter-annotator agreement indices such as Scott's $\pi$, Cohen’s $\kappa$, and Fleiss's $\pi^\ast,\kappa^\ast$, capturing boundary-level reliability across multiple coders.

WiSeBE (Window-based Sentence Boundary Evaluation) incorporates multi-reference "fuzzy" matching and global annotator agreement:

• For a transcript $\{t_1,\ldots,t_n\}$ and $m$ references, the general reference vector is $R_G = [d_1,\ldots,d_n]$, $d_j = \sum_{i=1}^m R_i(j)$.
• Clustered windows of size $L$ capture local annotation consensus; system boundaries are true positives if within any window.
• Agreement Ratio $R_G_{AR} = R_{G_{PB}} / R_{G_{HA}}$, with $R_{G_{PB}}$ the count of boundaries agreed upon by $\ge 2$ annotators, $R_{G_{HA}}$ the maximum possible count.
• $\mathrm{WiSeBE} = F1_{R_W} \times R_G_{AR}$; this product scales fuzzy-match $F_1$ by annotation reliability (González-Gallardo et al., 2018).

Risk-Based and Psychometric Measures

Boundary detection benchmarks may be distorted by inconsistent or perceptually weak labels. A “risk” metric $R(\mathcal{S},\mathcal{A})$ estimates, via psychophysical 2AFC experiments and EM inference, the probability that a candidate (algorithmic) boundary is perceptually stronger than a benchmark (human-labeled) one:

• $$R(\mathcal S, \mathcal A) = P(x_i < x_j \mid s_i \in \mathcal S, s_j \in \mathcal A \setminus \mathcal S)$$, where $x_i$ is the inferred perceptual strength.

Systematically thresholding by inferred boundary strength reduces false penalization of true, non-reference boundaries and quantifies the trade-off between benchmark risk and coverage (Hou et al., 2013).

3. Specialized Metrics for Temporal and Event Boundary Alignment

SoftED metrics address boundary evaluation in time series event detection by introducing a temporal tolerance window and graded association:

• For each event $e_j$ at $t_{e_j}$ and detection $d_i$ at $t_{d_i}$, the membership $\mu_{e_j}(t_{d_i})$ is a symmetric triangular kernel:

$$\mu_{e_j}(t_{d_i}) = \max\left(\min\left(\frac{t_{d_i} - (t_{e_j}-\tau)}{\tau}, \frac{(t_{e_j}+\tau)-t_{d_i}}{\tau}\right), 0\right)$$

• One-to-one assignment ensures each detection/event pair is matched at most once, maximizing association weight.
• Soft precision and recall are computed as $$\mathrm{Precision}_s = TP_s/(TP_s+FP_s),\ \mathrm{Recall}_s = TP_s/m$$, where $TP_s$ sums the matched membership scores (Salles et al., 2023).

This methodology corrects for the zero credit assigned by crisp, timestamp-only metrics in presence of near-boundary detections and enables application-driven tuning of $\tau$.

4. Comparative Analysis of Boundary Metrics

Boundary metrics can be systematically contrasted with region-overlap or standard classifier metrics in terms of error sensitivity, scale behavior, and fairness under class imbalance:

Metric	Principle	Boundary Sensitivity	Outlier Robustness	Symmetry
Hausdorff	Maximal deviation	Global	Low	Yes
HD95/AHD/ASSD	Percentile/mean	Global/average	Improved	Yes (AHD)
Boundary F1-score	Local match (tolerant)	High	Yes (via $\theta$)	Yes
Boundary IoU	Overlap in boundary bands	High	Yes	Yes
Segmentation S	Edit distance with transpositions	Near-miss tolerant	Configurable	Yes
SoftED	Fuzzy temporal/dist window	Graded	Yes	Yes
WiSeBE	Multi-ref windowed $F_1\times$agreement	Fuzzy/local	Yes, window/L	Yes
Balanced Accuracy	Class reweighted	N/A (region labels)	Moderate	Yes

Boundary metrics are uniquely sensitive to small spatial or temporal displacements and enable fairer quantification of alignment in complex, ambiguous, or multi-rater tasks (Müller et al., 2022, Cheng et al., 2021, Fournier et al., 2012).

5. Implementation, Interpretation, and Pitfalls

Accurate computation of boundary accuracy metrics demands:

• Extraction of true 2D/3D boundary sets, typically using morphological gradients or explicit contour tracing.
• Distance transforms on the native image grid with voxel-rescaling (physical units).
• Careful per-class reporting in multi-class segmentation; background boundaries may otherwise dominate averages (Müller et al., 2022).
• For differentiable surrogates, e.g., boundary loss for neural nets, max-pooling layers explicitly encode $\theta$-wide tolerance bands, and metrics are combined with region-based losses for stable learning (Bokhovkin et al., 2019).

Numerical interpretation mandates explicit reporting of spatial scale (e.g., millimeters in medical imaging). Thresholds for “acceptable” boundary misalignment are always domain- and task-specific (e.g., 1–3 mm for brain ROIs).

Common computational and interpretive errors include using asymmetric/directed forms, neglecting boundary extraction details, or averaging across unbalanced class sets without normalization.

6. Advantages, Domain-Specific Choices, and Recommendations

Boundary accuracy metrics provide:

• High sensitivity to spatial or temporal misalignments, critical for tasks requiring precise localization (e.g., surgical margin planning, fine object delineation, event triage) (Müller et al., 2022, Cheng et al., 2021, Salles et al., 2023).
• Robustness to inter-annotator ambiguity, especially when combined with fuzzy or multi-reference matching (e.g., WiSeBE, S), or with psychometric modeling in risk-based assessment.
• Explicit trade-off control between precision of match and tolerance to annotation/alignment uncertainty.

Recommended practices include always pairing boundary metrics with region-overlap or prevalence-independent metrics, choosing threshold or window sizes based on domain error tolerance, and transparent reporting of implementation parameters and code.

Empirical evidence demonstrates that boundary-focused evaluation alters method selection and reveals qualitative improvements not visible with region-only metrics, especially in high-resolution or fine-grained boundary domains (Cheng et al., 2021, Bokhovkin et al., 2019, Salles et al., 2023).

7. Limitations and Open Challenges

Despite their advantages, boundary accuracy metrics are subject to:

• Sensitivity to noisy or artifact boundaries (classic Hausdorff) and reliance on accurate, unambiguous extraction of the true surface/interface.
• Dependence on parameter choices (tolerance widths, window sizes) which are sometimes data- or domain-dependent.
• Potential to be confounded by interior-only errors (Boundary IoU) or over-penalization in extremely noisy or ambiguous cases unless risk-based or multi-reference schemes are incorporated (Cheng et al., 2021, Hou et al., 2013, González-Gallardo et al., 2018).
• Quadratic complexity (edit-distance–based methods) for long sequences, although tractable for most real applications (Fournier et al., 2012).

Future research will likely focus on further harmonizing fuzzy, multi-reference, and psychometric reliability methodologies across task domains, integrating “soft boundary” evaluation into performance reporting standards, and standardizing metric computation for reproducibility and comparability (Müller et al., 2022, Cheng et al., 2021).