Inherent Interpretability Score (IIS)

• Inherent Interpretability Score (IIS) is a dual-purpose metric that evaluates interpretability via predictive accuracy, stability, simplicity, and loss of semantic information.
• For rule-based models, IIS combines predictivity, q-stability, and simplicity through a weighted sum to compare learned rule sets across different algorithms.
• For pre-trained visual representations, IIS is computed as the area under the accuracy-retention curve, capturing how sparse concept projections preserve classification performance.

Inherent Interpretability Score (IIS) denotes two distinct quantitative constructions in the recent interpretability literature. In the rule- and tree-based setting, IIS is a weighted sum of three explicitly defined terms—predictivity, stability, and simplicity—intended to compare interpretable learning algorithms (Margot et al., 2020). In pre-trained vision, IIS is the area under an accuracy-retention curve obtained when a representation is projected into a human-understandable concept space and progressively sparsified, so that the score quantifies the fraction of task-relevant semantic information preserved by concept-based interpretations (Shen et al., 28 Oct 2025). The shared acronym therefore does not identify a single universal metric; rather, it names two paper-specific measures of interpretability with different objects of evaluation, mathematical structures, and assumptions.

1. Two paper-specific definitions under a shared acronym

The two IIS formulations differ at the level of what is being scored. Margot and Luta define IIS for fitted predictors produced by rule-based or tree-based algorithms, with interpretability operationalized through predictive performance, robustness of the discovered rules, and model parsimony (Margot et al., 2020). Shen et al. define IIS for pre-trained representations, with interpretability operationalized through the extent to which classification power can be recovered from sparse human-understandable concepts (Shen et al., 28 Oct 2025).

IIS usage	Object scored	Construction
Margot & Luta	Rule-based and tree-based algorithms	Weighted sum of predictivity, $q$-stability, and simplicity
Shen et al.	Pre-trained vision representations	Area under the $ARR(s)$ curve over sparsity $s \in [0,1]$

This terminological overlap matters methodologically. The first IIS is comparative across an analyst-chosen library of algorithms, because its simplicity term is normalized relative to that library. The second IIS is comparative across representations and concept libraries, because its value depends on how much original classification accuracy can be retained after projection into a specified concept space. A plausible implication is that any citation to “IIS” requires immediate disambiguation by paper, model class, and evaluation protocol.

2. IIS for rule- and tree-based algorithms

In the formulation introduced in "A New Method to Compare the Interpretability of Rule-based Algorithms" (Margot et al., 2020), the starting point is an i.i.d. sample $(X_i,Y_i)$, $i=1,\dots,n$, a contrast function $\gamma(g;(X,Y)) \ge 0$, and the empirical risk

$$L_n(g) \;=\; \frac{1}{n}\sum_{i=1}^n \gamma\bigl(g;(X_i,Y_i)\bigr).$$

Examples given for $\gamma$ are squared error for regression and $0$–$1$ loss for classification.

The first component is the predictivity score. Given a fitted model $ARR(s)$0 and a baseline predictor $ARR(s)$1, it is defined as

$ARR(s)$2

The data specify that $ARR(s)$3 means “worse than baseline,” while $ARR(s)$4 means “approaching perfect accuracy” (Margot et al., 2020). Example baselines include the empirical mean for regression and the majority class rule for binary classification.

The second component is $ARR(s)$5-stability, based on the Dice–Sorensen index. Two independent models are fit on two disjoint halves of the training data, producing rule sets $ARR(s)$6 and $ARR(s)$7. Continuous conditions are discretized by empirical $ARR(s)$8-quantiles, and the resulting rule sets $ARR(s)$9 and $s \in [0,1]$0 are compared through

$s \in [0,1]$1

with the convention $s \in [0,1]$2. A value of $s \in [0,1]$3 means identical rule sets after discretization, whereas $s \in [0,1]$4 means no overlap.

The third component is simplicity. If $s \in [0,1]$5 denotes the rule set, or the set of root-to-leaf paths in a tree, the interpretability index is

$s \in [0,1]$6

where $s \in [0,1]$7 is the number of atomic tests in rule $s \in [0,1]$8. Over a collection $s \in [0,1]$9, the simplicity score for algorithm $(X_i,Y_i)$0 is

$(X_i,Y_i)$1

The simplest algorithm in the comparison set has $(X_i,Y_i)$2.

These three terms are combined as

$(X_i,Y_i)$3

subject to $(X_i,Y_i)$4. The paper allows equal weights $(X_i,Y_i)$5, or weights inversely proportional to the empirical variance of each component across a suite of datasets, and leaves the final choice to the analyst (Margot et al., 2020).

3. Computation, interpretation, and limitations in the rule-based formulation

The practical workflow separates the three components. For $(X_i,Y_i)$6, the data are split into train and test, the model is fit on train, and both $(X_i,Y_i)$7 and $(X_i,Y_i)$8 are evaluated on test. For $(X_i,Y_i)$9, the training set is randomly partitioned into two equal halves, two independent models are fit, the rules are discretized, and the Dice index is computed. For $i=1,\dots,n$0, the final rule set is extracted on the full training set, or a decision tree is converted into its set of root-to-leaf rules and their lengths are summed (Margot et al., 2020).

The treatment of trees is explicit: any decision tree with $i=1,\dots,n$1 leaves can be turned into exactly $i=1,\dots,n$2 rules by reading each path from root to leaf, and the length of each rule is the number of splits on that path. Rule-based learners such as RIPPER, BRL, RuleFit, and SIRUS already output a set of If-Then rules and can be used directly. The discretization parameter $i=1,\dots,n$3 should be moderate, with examples $i=1,\dots,n$4 or $i=1,\dots,n$5; too small $i=1,\dots,n$6 blurs stability through excessive collisions, while too large $i=1,\dots,n$7 makes exact matching almost impossible (Margot et al., 2020).

The paper gives a toy regression example with four observations $i=1,\dots,n$8, a baseline predictor $i=1,\dots,n$9, and a two-rule learner returning “if $\gamma(g;(X,Y)) \ge 0$0 then $\gamma(g;(X,Y)) \ge 0$1” and “if $\gamma(g;(X,Y)) \ge 0$2 then $\gamma(g;(X,Y)) \ge 0$3.” The empirical MSE is $\gamma(g;(X,Y)) \ge 0$4, the baseline MSE is $\gamma(g;(X,Y)) \ge 0$5, so $\gamma(g;(X,Y)) \ge 0$6. If the discretized rules coincide across two halves, $\gamma(g;(X,Y)) \ge 0$7. Each rule has length $\gamma(g;(X,Y)) \ge 0$8, so $\gamma(g;(X,Y)) \ge 0$9, and if this is the simplest algorithm in the collection then $$L_n(g) \;=\; \frac{1}{n}\sum_{i=1}^n \gamma\bigl(g;(X_i,Y_i)\bigr).$$0. With equal weights, the resulting IIS is approximately $$L_n(g) \;=\; \frac{1}{n}\sum_{i=1}^n \gamma\bigl(g;(X_i,Y_i)\bigr).$$1 (Margot et al., 2020).

Two real-data summaries illustrate how the score behaves under trade-offs. On Ozone regression, CART with at most $$L_n(g) \;=\; \frac{1}{n}\sum_{i=1}^n \gamma\bigl(g;(X_i,Y_i)\bigr).$$2 leaf nodes yields $$L_n(g) \;=\; \frac{1}{n}\sum_{i=1}^n \gamma\bigl(g;(X_i,Y_i)\bigr).$$3 rules, average predictivity $$L_n(g) \;=\; \frac{1}{n}\sum_{i=1}^n \gamma\bigl(g;(X_i,Y_i)\bigr).$$4, stability $$L_n(g) \;=\; \frac{1}{n}\sum_{i=1}^n \gamma\bigl(g;(X_i,Y_i)\bigr).$$5, simplicity $$L_n(g) \;=\; \frac{1}{n}\sum_{i=1}^n \gamma\bigl(g;(X_i,Y_i)\bigr).$$6, and thus $$L_n(g) \;=\; \frac{1}{n}\sum_{i=1}^n \gamma\bigl(g;(X_i,Y_i)\bigr).$$7. On the Speaker classification dataset, CART, RIPPER, and PART have similar accuracy with $$L_n(g) \;=\; \frac{1}{n}\sum_{i=1}^n \gamma\bigl(g;(X_i,Y_i)\bigr).$$8, RIPPER and PART are fully stable with $$L_n(g) \;=\; \frac{1}{n}\sum_{i=1}^n \gamma\bigl(g;(X_i,Y_i)\bigr).$$9, CART is slightly less stable at approximately $\gamma$0, and RIPPER has $\gamma$1, leading to the highest IIS at approximately $\gamma$2 (Margot et al., 2020).

Interpretive guidance is also given. All three component scores lie in $\gamma$3, so $\gamma$4. The paper proposes the rough guide $\gamma$5 for “very interpretable,” $\gamma$6 for mixed interpretability with trade-offs, and $\gamma$7 for poor interpretability. At the same time, the stated limitations are substantial: simplicity counts only atomic tests and not semantic naturalness; stability relies on discretization and exact rule matching, so semantically identical rules with slightly different thresholds can be counted as different; simplicity is inherently relative to a chosen library of algorithms; fairness and ethical constraints are not addressed; and the framework is mainly applicable to rule- or tree-based learners unless other model classes are first converted into rule sets (Margot et al., 2020).

4. IIS for pre-trained visual representations

In "Enhancing Pre-trained Representation Classifiability can Boost its Interpretability" (Shen et al., 28 Oct 2025), IIS is defined in a concept-bottleneck setting for downstream classification. Given a dataset $\gamma$8, a pre-trained feature extractor $\gamma$9, and a library of $0$0 human-understandable concepts $0$1 with $0$2, the representation is first projected into concept space: $0$3 where $0$4 is the $0$5 concept matrix. The $0$6-th coordinate $0$7 measures the linear contribution of concept $0$8 to the representation of $0$9.

These concept contributions are then sparsified to retain only the top $1$0 concepts, with $1$1 chosen so that exactly $1$2 coordinates are driven to zero. A linear head $1$3 is attached for classification. Let $1$4 be a linear head on the original representation $1$5. The key quantity at sparsity $1$6 is the Accuracy Retention Rate,

$1$7

where $1$8 is top-1 accuracy. IIS is then defined as the area under the $1$9 curve: $ARR(s)$00 In practice, this integral is approximated by evaluating $ARR(s)$01 on a discrete sparsity grid and computing a Riemann sum (Shen et al., 28 Oct 2025).

The conceptual distinction in this framework is between interpretable and uninterpretable semantics. The representation $ARR(s)$02 is treated as a mixture of semantics aligned with human-understandable concepts and semantics not captured by the concept library. When the representation is projected into concept space and sparsified, only the interpretable fraction survives; the rest induces information loss, visible as a drop in classification accuracy. By integrating accuracy retention over all sparsity levels, IIS summarizes that loss in a single scalar: larger IIS means that even very sparse concept explanations recover most of the original classification power (Shen et al., 28 Oct 2025).

This formulation therefore measures interpretability through recoverability. It does not ask whether a classifier is short or stable in the syntactic sense used for rules; it asks whether the predictive content of a representation can be faithfully expressed in a sparse, human-understandable concept basis.

5. Concept libraries, computation, and theoretical assumptions in the representation formulation

The practical computation of IIS begins with the choice of a concept library $ARR(s)$03 of size $ARR(s)$04. Four types are listed: Prototype, where each concept is a single image patch or segment; Cluster, where patches are clustered, for example by k-means, into $ARR(s)$05 concepts; End2End, where concept vectors are learned jointly by maximizing classification accuracy; and Textual, where words or phrases are drawn by prompting a LLM and vectors are fitted via Equation (2) in the paper (Shen et al., 28 Oct 2025).

Concept vectors are then extracted. For visual concepts based on patches, the definition is

$ARR(s)$06

For textual concepts, the paper uses

$ARR(s)$07

where $ARR(s)$08 is the soft label from a vision-LLM. At each sparsity level $ARR(s)$09, the features are projected to $ARR(s)$10, the parameters $ARR(s)$11 of $ARR(s)$12 are fit by minimizing cross-entropy while keeping $ARR(s)$13 and $ARR(s)$14 fixed, and the resulting accuracy is used to compute $ARR(s)$15. The paper gives an example sparsity grid $ARR(s)$16 and approximates IIS by a trapezoidal sum (Shen et al., 28 Oct 2025).

The theoretical rationale depends on three stated assumptions. First, the concept library should be complete with respect to the human-relevant semantics one cares to interpret. Second, concept contributions should combine linearly, and a low-dimensional linear head should suffice to measure accuracy retention. Third, top-1 or top-5 accuracy should faithfully capture downstream task performance. Under these assumptions, Shen et al. argue that IIS becomes a consistent, model-agnostic measure of how semantically transparent a representation is (Shen et al., 28 Oct 2025).

Empirically, the paper reports a strong positive correlation between IIS and zero-shot classification accuracy across a wide range of pre-trained models, including ResNet-18/34/50/101/152, ViT-B/L, Swin-T/S/B, and ConvNeXt-T/S/B/L, on ImageNet1K. The examples listed include ResNet-18 with accuracy approximately $ARR(s)$17 and IIS approximately $ARR(s)$18, ResNet-50 with approximately $ARR(s)$19 and $ARR(s)$20, ViT-B-16 with approximately $ARR(s)$21 and $ARR(s)$22, and ViT-L-16 with approximately $ARR(s)$23 and $ARR(s)$24. The same pattern is reported on CUB-200, CIFAR-10/100, and Kinetics-400. Correlation coefficients exceed $ARR(s)$25 in many settings, under Prototype, Cluster, End2End, and Text concept libraries. During pre-training, IIS is reported to increase alongside accuracy after a transient warm-up phase when both accuracies are low (Shen et al., 28 Oct 2025).

6. Applications, comparative significance, and recurring misunderstandings

The 2025 IIS is not only diagnostic but also used as an optimization target. The paper introduces a learnable projection $ARR(s)$26, fixes sparsity $ARR(s)$27, and optimizes

$ARR(s)$28

over $ARR(s)$29 and $ARR(s)$30, where $ARR(s)$31 is cross-entropy. On ImageNet, this procedure yields the reported top-1 improvements ResNet-50 $ARR(s)$32, Swin-T $ARR(s)$33, ConvNeXt-T $ARR(s)$34, and ViT-B $ARR(s)$35 (Shen et al., 28 Oct 2025). In the same framework, interpretable predictions can be produced directly from the concept-based classifier. With approximately $ARR(s)$36 textual concepts and $ARR(s)$37, ViT-B retains $ARR(s)$38 accuracy versus an original $ARR(s)$39, giving $ARR(s)$40, while ViT-L retains $ARR(s)$41 versus $ARR(s)$42, giving $ARR(s)$43. The paper states that these interpretable predictions surpass prior interpretability-oriented models such as LaBo at $ARR(s)$44 $ARR(s)$45 (Shen et al., 28 Oct 2025).

Additional applications follow from sparsification. Because a prediction depends on a small set of key concepts, a user can zero out a concept deemed spurious and observe the corrected output without retraining. The same IIS-maximized backbones are also reported to improve mean-IoU when fine-tuned on semantic segmentation datasets such as ADE20K, indicating transfer beyond classification (Shen et al., 28 Oct 2025).

A recurrent misunderstanding is to treat “IIS” as if it referred to a single general-purpose interpretability metric. The two formulations answer different questions. The rule-based IIS quantifies the trade-off among accuracy relative to a baseline, rule-set reproducibility, and syntactic compactness. The representation IIS quantifies information loss when original features are replaced by sparse concept-based interpretations. Another recurrent misunderstanding is the assumption of an inevitable trade-off between classifiability and interpretability in pre-trained representations. Shen et al. explicitly pose that question and report a positive correlation instead of an opposition (Shen et al., 28 Oct 2025). Conversely, in the rule-based setting, no such claim is made; the score is designed to expose trade-offs through analyst-chosen weights rather than to eliminate them (Margot et al., 2020).

Taken together, the two IIS definitions illustrate two non-equivalent traditions in interpretability research. One is model-centric and syntax-sensitive, focusing on rules, paths, and exact overlap. The other is representation-centric and semantics-sensitive, focusing on concept recoverability and information loss. This suggests that the substantive meaning of an IIS value is inseparable from the formal object being scored, the chosen concept or algorithm library, and the assumptions built into the respective paper.