---
title: BloodHound Equivalency Test
url: https://www.emergentmind.com/topics/bloodhound-equivalency-test
type: topic
---

# BloodHound Equivalency Test

The BloodHound Equivalency Test refers to a rigorous statistical methodology for assessing whether two measurement methods are equivalent—up to a pre-specified performance margin—based on paired repeated measures data. Developed in the context of diagnostic device comparison studies, such as oximetry, this test is grounded in a generalized pivotal quantity approach that jointly evaluates both mean and variance components via a root mean square (RMS) criterion. The methodology addresses limitations of large-sample normal approximations, especially in small or moderate sample size settings, and provides procedures for hypothesis testing and confidence interval estimation for practical equivalence [1908.07979].

## 1. Root Mean Square Criterion and Model Framework

In diagnostic device studies, the equivalency of two methods is often evaluated by controlling the absolute difference in measurements, summarized as paired differences $Y_{ij} = \text{(Method A)} - \text{(Method B)}$ for subject $i$ and replicate $j$. The statistical model underlying these differences is a one-factor random-effects ANOVA:

\[
Y_{ij} = \mu + u_i + \epsilon_{ij}, \quad u_i \sim N(0, \sigma_b^2),\;\epsilon_{ij}\sim N(0,\sigma_w^2)
\]

Here, $\mu$ denotes the mean difference, $\sigma_b^2$ the between-subject variance, and $\sigma_w^2$ the within-subject variance. The primary performance metric is the root mean–square (RMS) difference between methods:

\[
\rho = \sqrt{E[Y_{ij}^2]} = \sqrt{\mu^2 + \sigma_b^2 + \sigma_w^2}
\]

This composite parameter $\rho$ integrates both systematic bias and total variability, matching regulatory requirements (e.g., FDA) that specify equivalence in terms of a pre-specified upper bound $\Delta_0$ on $\rho$.

## 2. Hypothesis Formulation for Equivalence

Equivalency testing targets the composite RMS metric, using hypotheses of the form:

\[
H_0: \rho \geq \Delta_0 \quad \text{vs.} \quad H_a: \rho < \Delta_0
\]

The threshold $\Delta_0$ must be specified a priori based on clinical or regulatory criteria. For pulse oximetry, $\Delta_0=3\%$ is a typical margin based on FDA guidance.

## 3. Generalized Pivotal Quantity Construction

To formulate a statistically rigorous test and confidence interval, the BloodHound approach leverages generalized pivotal quantities (GPQs):

- Summarize the data with:  
    - Per-subject means $\bar{y}_i = \frac{1}{m_i}\sum_{j=1}^{m_i} y_{ij}$  
    - Sum of squared errors $sse = \sum_{i=1}^n (m_i-1) s_i^2$, $s_i^2 = \frac{1}{m_i-1}\sum_{j=1}^{m_i}(y_{ij} - \bar{y}_i)^2$
- Let $N = \sum_i m_i$.
- Use Cochran’s theorem to relate sums of squares to scaled chi-square distributions:

\[
SSE \sim \sigma_w^2 \chi^2_{N-n},\qquad SSR \sim \sigma_b^2 \chi^2_{n-1}
\]

- Define GPQs for each component:
    - $Q_w = sse / (SSE / \sigma_w^2)$, with $Q_w \sim \sigma_w^2$
    - $Q_b = h(\bar{y}, Q_w, SSR)$, an explicit function solving for $\sigma_b^2$
    - $Q_\mu = \tilde{y} - Z/\sqrt{\sum W_i}$, $Z \sim N(0,1)$, where $W_i = 1/(\sigma_b^2+\sigma_w^2/m_i)$ and $\tilde{Y}$ is the inverse-variance weighted mean.

The generalized pivotal quantity for $\rho$ is formulated as:

\[
Q = Q_\mu + Q_b + Q_w
\]

Testing $Q \geq \Delta_0$ is algebraically equivalent to testing $\sqrt{Q_\mu^2 + Q_b^2 + Q_w^2} \geq \Delta_0$, but practical implementation proceeds with the sum $Q$.

## 4. Algorithmic Procedure via Monte Carlo Simulation

The practical implementation involves Monte Carlo sampling:

1. Set number of simulations $B$ (e.g., $B = 10^4$).
2. For $k = 1, \ldots, B$:
    - Simulate $u_k \sim \chi^2_{N-n}$ and $v_k \sim \chi^2_{n-1}$.
    - Compute $Q_{w,k} = sse / u_k$, $Q_{b,k} = h(\bar{y}, Q_{w,k}, v_k)$.
    - Sample $Z_k \sim N(0,1)$ to obtain $Q_{\mu,k} = \tilde{y} - Z_k/\sqrt{\sum W_i}$.
    - Form $Q_k = Q_{\mu,k} + Q_{b,k} + Q_{w,k}$.
3. Compute the generalized $p$-value:

\[
\hat{p} = \frac{1}{B} \sum_{k=1}^B \mathbf{1}\{ Q_k \geq \Delta_0 \}
\]

4. The two-sided $100(1-\alpha)\%$ confidence interval for $\rho$ is

\[
\left[ Q_{(\lfloor B \alpha/2 \rfloor)},\ Q_{(\lceil B(1-\alpha/2)\rceil)} \right]
\]

Sorting $Q_1, \ldots, Q_B$.

The analytic integration of $Z$ (Section 2.3) in place of repeated normal sampling can further enhance numerical accuracy.

## 5. Performance Margin Selection and Sensitivity Considerations

Selecting the equivalency threshold $\Delta_0$ requires consultation with clinical guidelines, device specifications, or subject-matter experts. For pulse oximetry, the FDA frequently uses $\Delta_0 = 3\%$ in saturation units. Sensitivity analyses over a plausible range (e.g., $2$–$5\%$) for $\Delta_0$ are recommended to contextualize conclusions, especially when margins are based on pragmatic or evolving standards.

## 6. Performance Characteristics and Method Comparison

Extensive simulation studies reveal that the generalized pivotal test (GT) maintains well-controlled type I error near nominal levels across balanced and unbalanced study designs, outperforming large-sample normal approximations. The score-based $Z$-test is conservative, while the Wald-style $Z$-test is anti-conservative and not recommended. The GT also provides substantially higher power, particularly in small sample or stringent alpha scenarios; for example, for $n=16$, $m_i \in [5,20]$, and $\alpha=0.05$, GT achieves power of 82.1% compared to 72.1% for the $Z$-score test. In more stringent settings ($\alpha=0.01$), the difference is even more pronounced [1908.07979].

## 7. Software Implementation and Practical Guidelines

The BloodHound-style equivalency test is implemented in the R package `RAMgt`, available on GitHub and CRAN. The test requires only summary statistics—replicate counts, within-subject means, and sum of squared errors—obviating the need for full linear mixed model fits when subject-level summaries are available. Recommended study sizes are in the range $10 \leq n \leq 30$ with moderate replicates per subject. Monte Carlo sample sizes ($B$) can be scaled for desired precision, and batch reuse of random draws is enabled for multiple thresholds or significance levels. Pre-study sample size calculation, grounded in prior estimates of $\sigma_w$ and $\sigma_b$, is advised to ensure adequate power.

| Step                  | Input         | Output                       |
|-----------------------|--------------|------------------------------|
| Data summary          | $\bar{y},\ sse$ | $(Q_w,\ Q_\mu,\ Q_b)$     |
| Monte Carlo algorithm | $B$           | $p$-value, confidence interval |
| R package use         | `ng, mus, sse` | `p.value,\ ci.lower,\ ci.upper` |

## 8. Practical Considerations and Study Design Recommendations

The BloodHound equivalency test is particularly effective for studies with small to medium sample sizes where large-sample approximations fail. When complete subject-level data are unavailable, summary statistics suffice for inference. Computation is efficient in R for standard study sizes and simulation batch sizes. Pre-study power and sample size analyses are essential to calibrate operating characteristics to regulatory or clinical demands. The method's type I error control and statistical power make it the preferred approach for paired repeated measures equivalency testing in diagnostic device evaluation scenarios [1908.07979].

Source: https://www.emergentmind.com/topics/bloodhound-equivalency-test