OptiLIME: Enhanced Tree-based LIME Explanations

• OptiLIME is a suite of methods that enhance traditional LIME by substituting linear surrogates with optimized tree-based models, improving local fidelity and consistency.
• It employs tree-based surrogates to capture nonlinear interactions, deliver multi-class coherent explanations, and offer richer interpretability with rules and counterfactuals.
• Empirical studies demonstrate that OptiLIME reduces surrogate error and improves human interpretability across image, tabular, and text domains.

OptiLIME refers to a family of methods that augment and generalize the Local Interpretable Model-Agnostic Explanations (LIME) framework by replacing its standard linear surrogate with optimized, tree-based or partitioned models, to afford higher-fidelity, more consistent, and interpretable explanations of complex black-box predictors. OptiLIME methodologies subsume several published lines of work, including Tree-LIME, LIMEtree, and LIME-SUP, each targeting improved local fidelity, capturing nonlinear and global-local phenomena, and multi-class consistency.

1. Motivation and Context

LIME is a canonical post-hoc local explainer which, given a black-box classifier or regressor $f$ and an input $x$, approximates $f$ in a neighborhood of $x$ by an interpretable surrogate $g$ (originally, a sparse linear regressor). While widely adopted, standard LIME surrogates exhibit three critical limitations:

• Poor handling of local nonlinearities and feature interactions, especially for image and tabular data (Shi et al., 2019).
• Incoherent or conflicting explanations across different classes, due to the one-vs-rest paradigm for classification (Sokol et al., 2020).
• Weak fidelity and stability in many real-world regression and multiclass settings, especially with high-dimensional data (Thombre, 2024, Ranjbar et al., 2022).

OptiLIME approaches address these constraints by:

• Employing tree-based surrogates (single-output and multi-output trees), supervised partitioning, or supervised tree ensembles for interpretable modeling.
• Enabling coherent, multi-class explanations and supporting a richer suite of explanation modalities (rules, counterfactuals, feature importances, exemplars).
• Achieving quantifiably greater local fidelity, stability, and human interpretability.

2. Tree-Based Surrogates: Methodological Foundations

OptiLIME replaces the linear $g$ of LIME with optimized regression trees or forests, generalizing to both regression and classification settings.

Let $f: \mathcal{X} \to \mathcal{Y}$ be the black-box predictor; e.g., $\mathcal{Y} = [0,1]^n$ for $n$-class probabilities. For an instance $x$, a neighborhood $Z$ is built via interpretable perturbations and proximity kernel weighting $\pi_x(z) = \exp\left(-\ell(x,z)^2/\nu^2\right)$.

The surrogate $g$ is chosen (for multi-output, $g: \mathcal{X}' \to \mathbb{R}^n$ for interpretable encoding $\mathcal{X}'$) as

$$g^* = \arg\min_{g \in \mathcal{G}} \sum_{z \in Z} \pi_x(z)\, \mathcal{L}(f(z), g(z)) + \Omega(g)$$

with $\mathcal{L}$ a squared error (for regression or class probabilities), $\Omega(g)$ a complexity regularizer (e.g., tree depth or number of leaves).

Multi-output regression trees are constructed by recursively splitting on interpretable features at thresholds (0.5 for binary encoding). Feature splits maximize reduction in total weighted impurity across all outputs:

$$i^* = \arg\max_{i} \left\{ I(N) - [I(N_L(i)) + I(N_R(i))] \right\}$$

where $I(N)$ is node impurity (weighted sum of squared deviations from node-mean response vector).

3. Multi-Class and Structured Explanation Guarantees

Classic LIME explanations are delivered per-class, yielding mutually incompatible local surrogates—this hampers insight in multi-class or structured output settings. OptiLIME's multi-output tree ($g$) framework, as exemplified by LIMEtree (Sokol et al., 2020), creates a unified surrogate that delivers:

• Class-consistent explanations: all class probabilities are modeled jointly, preserving inter-class dependencies.
• Strong local fidelity: with sufficient tree capacity ($2^d$ leaves for $d$ binary features), the surrogate can exactly represent the black-box model's outputs over the perturbed neighborhood:

$$\mathcal{L} = 0 \implies \text{all rule, counterfactual, and "what-if" explanations are structurally faithful}$$

• Structural guarantees: minimal-representation and full-data-fidelity are formalized, with theoretical proofs.

This paradigm supports extraction of feature importances, decision rules, exemplars, what-if predictions, and counterfactuals from a single, interpretable structure.

4. Empirical Performance and Quantitative Comparison

Comprehensive empirical results across image (ImageNet, CIFAR-10/100), tabular (Wine, CoverType, UCI regressions), and text (IMDb) domains demonstrate:

• Substantially lower local surrogate weighted-MSE (fidelity loss) for trees versus linear surrogates, with trees achieving the same fidelity as LIME at only 66–75% of LIME’s complexity (Sokol et al., 2020).
• For regression, tree-based surrogates outperform LIME-linear in 87% of runs across standard datasets, with lower RMSE observed: e.g., RMSE $\approx 1.6$ for tree versus $6.8$ for linear-LIME on the Yacht dataset (Thombre, 2024).
• In user studies, multi-output trees yield 25% higher question-answering accuracy than separate class-wise LIME explanations, albeit with increased cognitive extraction burden for manual "tree parsing" (Sokol et al., 2020).
• Human interpretability ratings (1–5 scale) are consistently higher for tree-based explanations using optimized trees, especially in text and tabular domains; in some datasets, examiner-prediction accuracy doubles versus linear LIME (Ranjbar et al., 2022).

A summary of fidelity and interpretability metrics is provided in the following table:

Method	Local Fidelity (Accuracy/RMSE)	Human Clarity (1–5)	Surrogate Complexity
LIME (linear)	Up to 0.97 / RMSE 6.8	3–4	High (per-class model)
OptiLIME (Tree-LIME)	0.92–1.00 / RMSE 1.6	4–5	Lower (shared tree)
LIMEtree (multi-output)	25% lower loss than LIME	Higher QA accuracy	66–75% of LIME

5. Algorithmic Variants and Extensions

Notable OptiLIME approaches include:

• LIMEtree: Multi-output regression tree optimizing

$$\sum_{z \in Z}\pi_x(z)\sum_{c \in C}[ f_c(z) - g_c(z) ]^2 + \Omega(g)$$

with coherent explanations and counterfactuals (Sokol et al., 2020).

• Tree-LIME with SHAP: Computation of exact Shapley values on the tree surrogate for feature attributions, inheriting axiomatic SHAP properties while preserving local faithfulness and efficiency enhancements over KernelSHAP (Aditya et al., 2022).
• Tree-SUP (LIME-SUP): Supervised partitioning trees for global-local fidelity, with splits determined to minimize local SSE on $f(x)$ or its derivatives, outperforming unsupervised cluster partitioners (KLIME) in fidelity and stability (Hu et al., 2018).
• Tree-LIME with autoencoders (Tree-ALIME): Uses denoising autoencoder for perturbation weighting prior to tree induction, further increasing stability and clarity of explanations in high-dimensional domains (Ranjbar et al., 2022).

In all cases, core steps include: sampling perturbations, proximity kernel weighting, tree (or partition) induction with regularization, and extraction of human-interpretable rule sets, importances, or attributions.

6. Practical Considerations: Hyperparameters and Limitations

Effective use of OptiLIME methodologies requires careful selection of:

• Number of perturbations $N$: Empirically, $500$–$2000$ supports stable fitting.
• Tree complexity: Maximum depth (3–6), minimum weighted samples per leaf, and regularization $\lambda$ control the interpretability–fidelity tradeoff.
• Kernel width $\sigma$: Governs locality; too small induces overfitting/noisy surrogates; too large sacrifices local fidelity.
• Interpretability constraints: Ensuring tree explanations remain succinct and comprehensible, sometimes via forced sparsity or post-hoc pruning.

Limitations include: potential overfitting of high-capacity trees, decreased interpretability in high-dimensional encodings (notably for images), and, in some datasets, slightly reduced local fidelity relative to linear surrogates, especially as the number of perturbations decreases (Ranjbar et al., 2022). Automated interface and visualization support can ameliorate cognitive extraction challenges identified in user studies (Sokol et al., 2020).

7. Related Methods and Theoretical Implications

OptiLIME is closely related to other surrogate-based explainers (e.g., KLIME—k-means cluster with local linear fitting), but tree-based supervised partitioning inherently yields more stable and interpretable segmentations aligned with $f$'s behavior (Hu et al., 2018). The framework is compatible with both local (per-instance) and semi-global (region-based) explanations.

A plausible implication is that, as model complexity and number of classes increase, multi-output, partition-based surrogates offer the only viable path to both faithful and actionable interpretation of black-box decisions.

• "LIMEtree: Consistent and Faithful Surrogate Explanations of Multiple Classes" (Sokol et al., 2020)
• "Local Interpretable Model Agnostic Shap Explanations for machine learning models" (Aditya et al., 2022)
• "Explaining the Predictions of Any Image Classifier via Decision Trees" (Shi et al., 2019)
• "Comparison of decision trees with Local Interpretable Model-Agnostic Explanations (LIME) technique and multi-linear regression for explaining support vector regression model in terms of root mean square error (RMSE) values" (Thombre, 2024)
• "Using Decision Tree as Local Interpretable Model in Autoencoder-based LIME" (Ranjbar et al., 2022)
• "Locally Interpretable Models and Effects based on Supervised Partitioning (LIME-SUP)" (Hu et al., 2018)