---
title: Sidechannel Independence Score (SIS)
url: https://www.emergentmind.com/topics/sidechannel-independence-score-sis
type: topic
---

# Sidechannel Independence Score (SIS)

Searching arXiv for the SIS paper and closely related side-channel/independence foundations.
arXiv search query: "Sidechannel Independence Score Concept Sidechannel Models"
Sidechannel Independence Score (SIS) is a metric introduced for Concept Sidechannel Models (CSMs) to quantify how much a model’s predictions rely on an uninterpretable sidechannel rather than on supervised concepts. It is defined by comparing a model’s standard prediction with the prediction obtained when the sidechannel is disabled and replaced by an input-independent prior; the score is the probability that the two predictions agree. In this formulation, high SIS indicates low practical reliance on the sidechannel and greater representation interpretability, whereas low SIS indicates that predictions change frequently when the sidechannel is removed [2510.05670].

## 1. Problem setting and conceptual motivation

SIS was introduced in the context of Concept Sidechannel Models, which extend concept bottleneck models by adding a sidechannel, residual, or sidepath that carries information from the input to the task predictor outside the concept bottleneck. The motivating problem is the well-known tension between concept sufficiency and predictive accuracy: concept bottleneck models are more interpretable because prediction must flow through concepts, but that bottleneck often reduces accuracy; CSMs recover accuracy by allowing additional non-concept information through a sidechannel, but this can make the model representation-uninterpretable if decisions are materially driven by uninterpretable latent features [2510.05670].

The metric targets what the paper calls **representation interpretability**, not **functional interpretability**. A model can have a functionally interpretable predictor, such as a linear model, logic rules, or a small tree, and still fail to be representation-interpretable if the predictor mainly consumes sidechannel variables. SIS is designed to measure whether predictions are effectively concept-grounded by testing their invariance to sidechannel removal.

A terminological caution is relevant. The acronym “SIS” is also used in the statistics literature for **Sure Independence Screening**, including KCCA-SIS for ultrahigh-dimensional feature screening. That usage refers to a screening score for marginal feature dependence, not to Sidechannel Independence Score [1604.07354].

## 2. Probabilistic framework and inference modes

The SIS paper places CSMs in a unified probabilistic meta-model with input \(X\), concepts \(C\), sidechannel variables \(Z\), and target label \(Y\). The joint distribution factorizes as
\[
p(y,c,z,x) = p(x)\cdot p(c|x)\cdot p(z|x)\cdot p(y|c,z),
\]
and inference is written as
\[
p(y|x) = \sum_{c,z} p(c|x)\cdot p(z|x)\cdot p(y|c,z).
\]
Architecturally, the model is specified by a concept predictor \(\phi_c:X\to C\), a sidechannel predictor \(\phi_z:X\to Z\), and a task predictor \(\phi_y:C,Z\to Y\) [2510.05670].

SIS depends on comparing two inference modes. In **default mode**, prediction uses the input-dependent sidechannel:
\[
p_{\theta,\phi,\psi}(y|x)=\sum_{c,z} p_\theta(c|x)\cdot p_\phi(z|x)\cdot p_\psi(y|c,z).
\]
In **bottleneck mode**, the sidechannel is disabled by replacing \(p_\phi(z|x)\) with an input-independent prior \(p_\gamma(z)\):
\[
p_{\theta,\gamma,\psi}(\bar y|x)=\sum_{c,z} p_\theta(c|x)\cdot p_\gamma(z)\cdot p_\psi(y|c,z).
\]
The prior is first proposed as the marginal
\[
p_\gamma(z)=\sum_x p(x)\cdot p_\phi(z|x)\approx \frac{1}{|\mathcal D|}\sum_{x\in\mathcal D} p_\phi(z|x).
\]

This construction is central because bottleneck mode is taken as the fully representation-interpretable reference behavior: when \(Z\) is replaced by an \(X\)-independent variable, prediction depends on \(X\) only through concepts. SIS then measures how closely default-mode behavior tracks that bottleneck reference.

## 3. Exact definition, estimation, and interpretation

The paper defines SIS by comparing thresholded predictions in default and bottleneck mode. Let \(y_x\) and \(\bar y_x\) denote the corresponding predictions. Then
\[
SIS = \mathbb{E}_{x\sim p(X)}[\mathbbm{1}[y_x=\bar y_x]].
\]
Its empirical estimator is
\[
\widehat{SIS}=\frac{1}{|\mathcal D|}\sum_{x\in\mathcal D}\mathbbm{1}[y_x=\bar y_x].
\]
Because the score is an average of Bernoulli indicators, it satisfies the Hoeffding bound
\[
p(|\widehat{SIS}-SIS|\ge \epsilon)\le 2e^{-2|\mathcal D|\epsilon^2}.
\]
The paper gives the example that if \(\widehat{SIS}=60\%\) on a test set of size \(1000\), then a 95% confidence interval is approximately \([58\%,62\%]\) [2510.05670].

The score is bounded between 0 and 1 by construction. High SIS means that predictions are mostly unchanged when sidechannel information is removed; low SIS means that the sidechannel materially changes decisions. The paper explicitly characterizes SIS as measuring “the frequency with which the model’s prediction changes when the sidechannel is removed.”

This makes SIS an **agreement-under-ablation** metric rather than a direct measure of probabilistic conditional independence. The paper explicitly notes that SIS does not directly measure a statement such as \(Y \perp Z \mid C\), nor is it defined as a divergence between predictive distributions. A plausible implication is that SIS should be read as an operational proxy for sidechannel reliance at the decision level: it detects whether the sidechannel changes the model’s chosen output, not whether it changes internal predictive distributions in a finer-grained sense.

## 4. SIS regularization and practical computation

The paper uses SIS not only as an evaluation metric but also as a training principle. The proposed **SIS regularization** penalizes divergence between default-mode and bottleneck-mode predictive distributions:
\[
\argmax_{\phi,\psi,\gamma,\theta}\left[
\sum_{(x,c,y)\in\mathcal D}
\left(
\log p_{\phi,\psi}(y|c,x)
+\alpha\cdot \log p_\theta(c|x)
-\beta\cdot \mathrm{DIV}\bigl(p_{\phi,\psi}(y|c,x)\|p_{\gamma,\psi}(\bar y|c,x)\bigr)
\right)
\right].
\]
Here \(\alpha\) weights concept supervision and \(\beta\) controls the strength of SIS regularization. The paper lists total variation distance and symmetric Kullback–Leibler divergence as suitable choices for \(\mathrm{DIV}\) [2510.05670].

The regularizer is a differentiable surrogate rather than the indicator-based SIS itself. In practice, training requires two prediction paths for each input: default mode with \(p_\phi(z|x)\), and bottleneck mode with \(p_\gamma(z)\). For deterministic or delta-style models, both are straightforward forward evaluations; for stochastic concept or sidechannel variables, probabilistic inference may require summation or approximation depending on the architecture.

The paper discusses two bottleneck-prior constructions. One is the empirical marginal prior over \(p_\phi(z|x)\). The other, used in the experiments, is a **learnable prior** \(p(z)\), chosen because it reduces computational cost, simplifies optimization, and in some models improves bottleneck-mode expressivity. For deterministic embedding sidechannels, the appendix notes a practical relaxation in which the marginalized prior is approximated by averaging embeddings.

## 5. Predictor expressivity, sidechannel reliance, and the accuracy–interpretability trade-off

A central theoretical contribution of the SIS paper is the claim that sidechannel reliance depends jointly on concept sufficiency and the expressivity of the bottleneck-mode predictor. The paper distinguishes several architecture families. CRM uses a neural network over concepts and is fully expressive in bottleneck mode but not functionally interpretable. LRM, DCR, and CBM-AUC are functionally interpretable but relatively inexpressive. CMR is intermediate, with expressivity depending on the number of learned rules [2510.05670].

This distinction is used to explain when high accuracy and high SIS can coexist. If concepts are sufficient for the task and the bottleneck predictor is expressive enough, then a CSM can in principle achieve high task accuracy without relying on the sidechannel. The paper states that expressive CSMs can then “effectively function as concept bottleneck models with the same interpretability.” Conversely, if the concept-to-task map is expressive but the bottleneck predictor is too weak, sidechannel usage can become necessary even when the concepts themselves are sufficient.

The XOR example is the clearest formal illustration. Let
\[
y := c_1 \oplus c_2.
\]
With a linear predictor over \((c,z)\), bottleneck mode cannot represent XOR from concepts alone. In default mode, however, the sidechannel can encode nonlinear combinations such as
\[
c_1 \land \neg c_2, \qquad \neg c_1 \land c_2,
\]
allowing the linear layer to recover the task. The paper uses this to argue that some CSMs rely on the sidechannel not only to supply information absent from the concepts, but also to compensate for an inexpressive concept-to-task predictor.

This analysis gives SIS a structural interpretation. Low SIS is not always evidence of gratuitous shortcut learning; it can also reflect an architectural bottleneck. High SIS is easiest to attain when concepts are sufficient and bottleneck-mode expressivity is adequate. This suggests that SIS should be interpreted jointly with model class and task structure rather than as an architecture-independent ranking of interpretability.

## 6. Empirical findings, limitations, and relation to broader independence notions

The empirical study covers two datasets and several CSM families. On **CelebA**, the concept set is insufficient for the target tasks. On **MNIST-Addition**, the concepts are sufficient, but the task requires expressive concept composition. Evaluated models include CRM, CEM, DCR, CMR, and LRM. The paper reports that state-of-the-art CSMs trained only for predictive accuracy tend to have low SIS, including on MNIST-Addition where the sidechannel is not theoretically necessary. SIS regularization moves models along a Pareto frontier toward higher SIS, usually with some accuracy loss; the trade-off is sharper on CelebA than on MNIST-Addition [2510.05670].

The reported qualitative consequences are broader than the score itself. SIS regularization improves **representation interpretability**, **intervenability**, and the quality of learned interpretable task predictors. In the LRM analysis on CelebA, the paper notes that without SIS regularization, large linear weights often attach to sidechannel neurons such as \(z11\), whereas with SIS regularization the dominant weights shift toward meaningful concepts such as *Gray Hair*. The paper also compares SIS regularization with concept-usage heuristics inspired by dropout and detach-style training; dropout improves SIS somewhat, detach only slightly, and both yield smaller SIS gains than the direct regularizer.

The paper also identifies several limitations of SIS. It is a prediction-level metric based on thresholded outputs, so substantial distributional differences that do not flip the decision can go uncounted. It depends on the choice of bottleneck prior \(p(z)\). It measures representation interpretability rather than functional interpretability. High SIS does not guarantee that the concepts themselves are well aligned or leakage-free.

In broader side-channel research, “independence” is often formalized differently. Information-theoretic traffic-analysis work uses mutual information \(I(X;Y)\) as the central leakage quantity and treats \(I(X;Y)=0\) as exact independence [2602.14055]. Noise-injection defenses for physical side channels likewise optimize mutual information or channel capacity between secrets and observations rather than prediction agreement [2504.20556]. Conditional Sibson \(\alpha\)-information provides another conditional leakage formalism, with \(I_\alpha(X;Y|Z)=0\) corresponding to conditional non-leakage [2105.07167]. Formal masking verification in the probing model expresses security as conditional independence between hidden secret parts and observed probes [2605.23316]. By contrast, SIS in CSMs is an ablation-based metric over model decisions rather than a universal information functional. This suggests that the term “independence” in SIS is operational and architectural: it measures invariance to sidechannel removal, not general statistical independence in the sense used in information theory or formal noninterference.

Source: https://www.emergentmind.com/topics/sidechannel-independence-score-sis