sSID: Sign-Augmented Structural Intervention Distance

• sSID is a metric that extends the Structural Intervention Distance by penalizing both structural errors and sign discrepancies in causal effect estimation.
• It introduces a weighted variant, wsSID, which quantifies the magnitude of polarity deviations to handle partial cancellations in biological networks.
• The algorithm employs matrix algebra and path enumeration to efficiently evaluate both reachability and directional accuracy, supporting improved model selection for interventions.

The sign-augmented Structural Intervention Distance (sSID) extends the Structural Intervention Distance (SID) framework for causal graph comparison by incorporating the directionality (sign) of causal effects. Developed in the context of biological networks where reference annotations are qualitative or sign-valued, sSID penalizes both incorrectly inferred interventional targets and disagreement in the net sign of “total-effect” causal relationships. A further weighted version, wsSID, quantifies the magnitude of polarity deviation and addresses partial path cancellations. sSID is particularly relevant for evaluating causal graphs inferred from biological data, where direction and sign of gene regulation or intervention effects are often critical for downstream applications such as drug target identification or clinical outcome prediction (Sato et al., 16 Nov 2025).

1. Formal Definition and Mathematical Framework

Let $G = (V, E)$ be a directed acyclic graph (DAG) with weighted adjacency matrix $B$ (entries $B_{ij} \in \{0\} \cup \mathbb{R}$), encoding direct causal effects. The total-effect matrix is defined as

$T(G) = \sum_{k=1}^{p-1} B^k,$

so that $T(G)_{ij}$ sums the weights of all directed paths from $i$ to $j$. For sign annotation,

$$S_{ij}(G) = \mathrm{sign}\, T(G)_{ij} \in \{-1, 0, \mathrm{NA}\},$$

with NA denoting “undefined” due to missing sign annotation.

For graphs $G$ (reference) and $G'$ (estimate), the pairwise SID indicator $SID_{ij}(G, G') = 1$ if the adjustment set implied by $G'$ for $\mathrm{do}(X_i) \rightarrow X_j$ is incorrect relative to $G$ (i.e., the $do$-distributions differ), and $0$ otherwise (Peters et al., 2013). The sign-error indicator is

$$E^{\mathrm{sign}}_{ij}(G,G') = 1 \iff (SID_{ij} = 0) \wedge S_{ij}(G) \in \{\pm 1\} \wedge S_{ij}(G') \neq S_{ij}(G),$$

$0$ otherwise.

The sign-augmented SID is then defined as

$$\mathrm{sSID}_\lambda(G, G') = \sum_{i \neq j} \Big[(1 - \lambda)\, SID_{ij}(G, G') + \lambda\, E^{\mathrm{sign}}_{ij}(G, G') \Big],$$

with $\lambda \in [0,1]$; default $\lambda = 0.5$.

The weighted variant (wsSID) is based on the average polarity

$$A_{ij}(G) = \frac{1}{|\mathcal{P}_{ij}|} \sum_{\pi \in \mathcal{P}_{ij}} \mathrm{sign}(\pi)$$

over all fully annotated directed paths $\mathcal{P}_{ij}$ from $i$ to $j$, and the polarity difference

$\Delta_{ij}(G,G') = |A_{ij}(G) - A_{ij}(G')|.$

Thus,

$$\mathrm{wsSID}_\lambda(G,G') = \sum_{i \neq j} \Big[(1 - \lambda) SID_{ij}(G,G') + \lambda \Delta_{ij}(G,G') \Big].$$

2. Algorithmic Computation

Efficient computation of sSID and wsSID employs both matrix algebra and path enumeration:

1. Construct adjacency matrices $B_\mathrm{ref}$ and $B_\mathrm{est}$ for the reference and estimated graphs, with $B_\mathrm{ref} \in \{-1, 0, +1, \mathrm{NA}\}$.
2. Compute total-effect matrices: $$T_{\mathrm{ref}} = \sum_{k=1}^{p-1} (B_\mathrm{ref})^k$$, or via $(I - B)^{-1} - I$ for acyclic graphs.
3. Derive sign matrices $S = \mathrm{sign}(T)$.
4. Enumerate all fully annotated directed paths to obtain average polarities $A_{ij}$ for all pairs.
5. For each pair $(i, j)$: evaluate $SID_{ij}$, $E^{\mathrm{sign}}_{ij}$, and $\Delta_{ij}$.
6. Aggregate as per the sSID or wsSID definition.

Complexity depends on sparsity and graph size. Computation of $T = (I - B)^{-1} - I$ is $O(p^3)$ for dense graphs, with dynamic programming enabling practical performance in sparse graphs (Sato et al., 16 Nov 2025).

3. Theoretical Properties

Key mathematical and operational properties include:

• Non-negativity: $\mathrm{sSID}_\lambda \geq 0$ always, with equality iff both structure and effect sign are matched for all pairs.
• Monotonicity: Adding structural or sign errors strictly increases sSID.
• Asymmetry: Generally, $\mathrm{sSID}(G, G') \neq \mathrm{sSID}(G', G)$ due to the directed comparison inherent in SID.
• Sensitivity: Sign errors propagate along reachable descendant pairs, allowing sSID to penalize incorrect sign orientation even if structural reachability is preserved.
• Robustness: Sign errors are only penalized when reachability is correct (no mistaken intervention in adjustment sets), avoiding duplicated penalties.
• Limitations: The metric presumes linear or monotonic relationships where sign annotations are meaningful; not directly applicable to nonparametric or cyclic structures (Sato et al., 16 Nov 2025).

4. Illustrative Examples and Interpretation

Concrete scenarios clarify sSID’s discriminative power:

Example	SID Outcome	sSID Outcome	wsSID Outcome
Simple chain (A→B→C), sign flip on A→B	0	0.5 (sign error A→C)	1.0 ($\Delta_{\mathrm{AC}} = 2$)
Parallel paths, partial sign cancellation	0	0	1.0 (partial polarity loss)

In structures with parallel or multi-step paths, wsSID quantifies graded disagreement ($\Delta_{ij}$) even when the discrete sign majority (and thus sSID) is unchanged. Thus, wsSID resolves partial cancellations, while sSID detects binary sign inversions.

5. Comparison to Other Metrics: SID and SHD

Metric	Penalizes	Sensitivity	Distinguishes Functional Errors?
SID	Wrong adjustment sets (structure only)	Insensitive to effect sign	No
sSID	SID + sign flips	Sensitive to direction of effect	Yes
wsSID	sSID + average polarity deviation	Graded, path-wise effect magnitude	Yes (quantitative)

SID counts only mismatches in interventional reachability. sSID adds penalties for sign disagreement in total effects, capturing “functionally wrong” inferences undetectable by structural metrics alone. wsSID further quantifies magnitude of disagreement for partially cancelling errors (Sato et al., 16 Nov 2025).

6. Empirical Performance and Applications

Empirical analysis in simulations and cancer transcriptomic datasets demonstrates that sSID and wsSID select different estimator algorithms relative to SID. For example, in gene expression networks inferred for LUAD and PRAD TCGA datasets, networks optimizing sSID achieved higher classification precision-recall for tumor stage using predicted gene expression features (AUPRC ≈ 0.764 for sSID-optimal vs. ≈ 0.749 for SID-optimal) (Sato et al., 16 Nov 2025).

Simulations show sSID and wsSID grow linearly with the frequency of sign-flip errors, while SID and SHD do not detect such errors. wsSID additionally discerns partial polarity loss due to parallel paths, demonstrating value in biological contexts where signed causal direction is fundamental.

7. Practical Considerations and Implementation

Efficient sSID/wsSID usage requires:

• Data requirements: Reference networks annotated with {+1, −1, NA}; estimated networks with quantitative weights.
• Computational cost: Preferential for sparse DAGs; dynamic programming methods recommended for large $p$ or complex path structure.
• Choice of $\lambda$: Default is $\lambda = 0.5$, with higher values for sign-sensitive applications.
• Software: Public R/C++ implementations support sSID/wsSID computation, with integration to common structure learning toolkits.

The incorporation of sign-awareness mediates more biologically and functionally meaningful model evaluation, providing discrimination between “structurally right” but “functionally wrong” networks and thereby guiding model selection for downstream intervention analysis (Sato et al., 16 Nov 2025).