Sidechannel Independence Score (SIS)

• The paper introduces SIS to assess representation interpretability by measuring the agreement between default and bottleneck-mode predictions.
• SIS is defined as the probability that a model’s prediction remains unchanged when the sidechannel is replaced with an input-independent prior.
• SIS regularization is proposed to balance accuracy and interpretability, guiding models to rely more on supervised concepts than on uninterpretable sidechannels.

Searching arXiv for the SIS paper and closely related side-channel/independence foundations. arXiv search query: "Sidechannel Independence Score Concept Sidechannel Models" Sidechannel Independence Score (SIS) is a metric introduced for Concept Sidechannel Models (CSMs) to quantify how much a model’s predictions rely on an uninterpretable sidechannel rather than on supervised concepts. It is defined by comparing a model’s standard prediction with the prediction obtained when the sidechannel is disabled and replaced by an input-independent prior; the score is the probability that the two predictions agree. In this formulation, high SIS indicates low practical reliance on the sidechannel and greater representation interpretability, whereas low SIS indicates that predictions change frequently when the sidechannel is removed (Debot et al., 7 Oct 2025).

1. Problem setting and conceptual motivation

SIS was introduced in the context of Concept Sidechannel Models, which extend concept bottleneck models by adding a sidechannel, residual, or sidepath that carries information from the input to the task predictor outside the concept bottleneck. The motivating problem is the well-known tension between concept sufficiency and predictive accuracy: concept bottleneck models are more interpretable because prediction must flow through concepts, but that bottleneck often reduces accuracy; CSMs recover accuracy by allowing additional non-concept information through a sidechannel, but this can make the model representation-uninterpretable if decisions are materially driven by uninterpretable latent features (Debot et al., 7 Oct 2025).

The metric targets what the paper calls representation interpretability, not functional interpretability. A model can have a functionally interpretable predictor, such as a linear model, logic rules, or a small tree, and still fail to be representation-interpretable if the predictor mainly consumes sidechannel variables. SIS is designed to measure whether predictions are effectively concept-grounded by testing their invariance to sidechannel removal.

A terminological caution is relevant. The acronym “SIS” is also used in the statistics literature for Sure Independence Screening, including KCCA-SIS for ultrahigh-dimensional feature screening. That usage refers to a screening score for marginal feature dependence, not to Sidechannel Independence Score (Liu et al., 2016).

2. Probabilistic framework and inference modes

The SIS paper places CSMs in a unified probabilistic meta-model with input $X$, concepts $C$, sidechannel variables $Z$, and target label $Y$. The joint distribution factorizes as

$$p(y,c,z,x) = p(x)\cdot p(c|x)\cdot p(z|x)\cdot p(y|c,z),$$

and inference is written as

$$p(y|x) = \sum_{c,z} p(c|x)\cdot p(z|x)\cdot p(y|c,z).$$

Architecturally, the model is specified by a concept predictor $\phi_c:X\to C$, a sidechannel predictor $\phi_z:X\to Z$, and a task predictor $\phi_y:C,Z\to Y$ (Debot et al., 7 Oct 2025).

SIS depends on comparing two inference modes. In default mode, prediction uses the input-dependent sidechannel: $$p_{\theta,\phi,\psi}(y|x)=\sum_{c,z} p_\theta(c|x)\cdot p_\phi(z|x)\cdot p_\psi(y|c,z).$$ In bottleneck mode, the sidechannel is disabled by replacing $C$0 with an input-independent prior $C$1: $C$2 The prior is first proposed as the marginal

$C$3

This construction is central because bottleneck mode is taken as the fully representation-interpretable reference behavior: when $C$4 is replaced by an $C$5-independent variable, prediction depends on $C$6 only through concepts. SIS then measures how closely default-mode behavior tracks that bottleneck reference.

3. Exact definition, estimation, and interpretation

The paper defines SIS by comparing thresholded predictions in default and bottleneck mode. Let $C$7 and $C$8 denote the corresponding predictions. Then

$C$9

Its empirical estimator is

$Z$0

Because the score is an average of Bernoulli indicators, it satisfies the Hoeffding bound

$Z$1

The paper gives the example that if $Z$2 on a test set of size $Z$3, then a 95% confidence interval is approximately $Z$4 (Debot et al., 7 Oct 2025).

The score is bounded between 0 and 1 by construction. High SIS means that predictions are mostly unchanged when sidechannel information is removed; low SIS means that the sidechannel materially changes decisions. The paper explicitly characterizes SIS as measuring “the frequency with which the model’s prediction changes when the sidechannel is removed.”

This makes SIS an agreement-under-ablation metric rather than a direct measure of probabilistic conditional independence. The paper explicitly notes that SIS does not directly measure a statement such as $Z$5, nor is it defined as a divergence between predictive distributions. A plausible implication is that SIS should be read as an operational proxy for sidechannel reliance at the decision level: it detects whether the sidechannel changes the model’s chosen output, not whether it changes internal predictive distributions in a finer-grained sense.

4. SIS regularization and practical computation

The paper uses SIS not only as an evaluation metric but also as a training principle. The proposed SIS regularization penalizes divergence between default-mode and bottleneck-mode predictive distributions: $Z$6 Here $Z$7 weights concept supervision and $Z$8 controls the strength of SIS regularization. The paper lists total variation distance and symmetric Kullback–Leibler divergence as suitable choices for $Z$9 (Debot et al., 7 Oct 2025).

The regularizer is a differentiable surrogate rather than the indicator-based SIS itself. In practice, training requires two prediction paths for each input: default mode with $Y$0, and bottleneck mode with $Y$1. For deterministic or delta-style models, both are straightforward forward evaluations; for stochastic concept or sidechannel variables, probabilistic inference may require summation or approximation depending on the architecture.

The paper discusses two bottleneck-prior constructions. One is the empirical marginal prior over $Y$2. The other, used in the experiments, is a learnable prior $Y$3, chosen because it reduces computational cost, simplifies optimization, and in some models improves bottleneck-mode expressivity. For deterministic embedding sidechannels, the appendix notes a practical relaxation in which the marginalized prior is approximated by averaging embeddings.

5. Predictor expressivity, sidechannel reliance, and the accuracy–interpretability trade-off

A central theoretical contribution of the SIS paper is the claim that sidechannel reliance depends jointly on concept sufficiency and the expressivity of the bottleneck-mode predictor. The paper distinguishes several architecture families. CRM uses a neural network over concepts and is fully expressive in bottleneck mode but not functionally interpretable. LRM, DCR, and CBM-AUC are functionally interpretable but relatively inexpressive. CMR is intermediate, with expressivity depending on the number of learned rules (Debot et al., 7 Oct 2025).

This distinction is used to explain when high accuracy and high SIS can coexist. If concepts are sufficient for the task and the bottleneck predictor is expressive enough, then a CSM can in principle achieve high task accuracy without relying on the sidechannel. The paper states that expressive CSMs can then “effectively function as concept bottleneck models with the same interpretability.” Conversely, if the concept-to-task map is expressive but the bottleneck predictor is too weak, sidechannel usage can become necessary even when the concepts themselves are sufficient.

The XOR example is the clearest formal illustration. Let

$Y$4

With a linear predictor over $Y$5, bottleneck mode cannot represent XOR from concepts alone. In default mode, however, the sidechannel can encode nonlinear combinations such as

$Y$6

allowing the linear layer to recover the task. The paper uses this to argue that some CSMs rely on the sidechannel not only to supply information absent from the concepts, but also to compensate for an inexpressive concept-to-task predictor.

This analysis gives SIS a structural interpretation. Low SIS is not always evidence of gratuitous shortcut learning; it can also reflect an architectural bottleneck. High SIS is easiest to attain when concepts are sufficient and bottleneck-mode expressivity is adequate. This suggests that SIS should be interpreted jointly with model class and task structure rather than as an architecture-independent ranking of interpretability.

6. Empirical findings, limitations, and relation to broader independence notions

The empirical study covers two datasets and several CSM families. On CelebA, the concept set is insufficient for the target tasks. On MNIST-Addition, the concepts are sufficient, but the task requires expressive concept composition. Evaluated models include CRM, CEM, DCR, CMR, and LRM. The paper reports that state-of-the-art CSMs trained only for predictive accuracy tend to have low SIS, including on MNIST-Addition where the sidechannel is not theoretically necessary. SIS regularization moves models along a Pareto frontier toward higher SIS, usually with some accuracy loss; the trade-off is sharper on CelebA than on MNIST-Addition (Debot et al., 7 Oct 2025).

The reported qualitative consequences are broader than the score itself. SIS regularization improves representation interpretability, intervenability, and the quality of learned interpretable task predictors. In the LRM analysis on CelebA, the paper notes that without SIS regularization, large linear weights often attach to sidechannel neurons such as $Y$7, whereas with SIS regularization the dominant weights shift toward meaningful concepts such as Gray Hair. The paper also compares SIS regularization with concept-usage heuristics inspired by dropout and detach-style training; dropout improves SIS somewhat, detach only slightly, and both yield smaller SIS gains than the direct regularizer.

The paper also identifies several limitations of SIS. It is a prediction-level metric based on thresholded outputs, so substantial distributional differences that do not flip the decision can go uncounted. It depends on the choice of bottleneck prior $Y$8. It measures representation interpretability rather than functional interpretability. High SIS does not guarantee that the concepts themselves are well aligned or leakage-free.

In broader side-channel research, “independence” is often formalized differently. Information-theoretic traffic-analysis work uses mutual information $Y$9 as the central leakage quantity and treats $$p(y,c,z,x) = p(x)\cdot p(c|x)\cdot p(z|x)\cdot p(y|c,z),$$0 as exact independence (Liu et al., 15 Feb 2026). Noise-injection defenses for physical side channels likewise optimize mutual information or channel capacity between secrets and observations rather than prediction agreement (Woo et al., 29 Apr 2025). Conditional Sibson $$p(y,c,z,x) = p(x)\cdot p(c|x)\cdot p(z|x)\cdot p(y|c,z),$$1-information provides another conditional leakage formalism, with $$p(y,c,z,x) = p(x)\cdot p(c|x)\cdot p(z|x)\cdot p(y|c,z),$$2 corresponding to conditional non-leakage (Liu et al., 2021). Formal masking verification in the probing model expresses security as conditional independence between hidden secret parts and observed probes (Kura et al., 22 May 2026). By contrast, SIS in CSMs is an ablation-based metric over model decisions rather than a universal information functional. This suggests that the term “independence” in SIS is operational and architectural: it measures invariance to sidechannel removal, not general statistical independence in the sense used in information theory or formal noninterference.