DL-LIME: Enhanced Explanations for Deep Models

• DL-LIME is an enhanced version of LIME that leverages deep neural networks to generate physically plausible, locally faithful explanations for deep learning models.
• It improves explanation fidelity in high-correlation environments by replacing independent noise sampling with conditional, DNN-driven perturbations, reducing surrogate MAE.
• The technique offers actionable insights in applications like radar resource management by ensuring that generated explanations respect real-world physical dependencies.

DL-LIME refers to an enhanced version of the Local Interpretable Model-agnostic Explanations (LIME) technique, specifically adapted to deep learning models by modifying the sampling mechanism for generating local surrogate explanations. This approach was introduced to address the unique challenges encountered in applying standard LIME to domains such as deep reinforcement learning (DRL) for radar resource management, where strong correlations exist between input features due to underlying physical dependencies and temporal dynamics (Lu et al., 26 Jun 2025). DL-LIME integrates a deep learning model into the sampling process of LIME, enabling explanation procedures that respect feature correlations and generate physically plausible perturbations.

1. Motivation and Problem Context

DRL models have achieved high performance in complex control and resource allocation domains, including radar resource management (RRM), where they are tasked with dynamically allocating radar dwell time between multiple tracking and scanning objectives. However, the decision process of DRL—relying on high-capacity neural architectures with many correlated input features—remains non-transparent, and traditional XAI approaches like LIME are limited by their independence assumption during perturbation sampling. In application to RRM, independently perturbing state features (e.g., tracking cost and target distance) creates unrealistic samples that do not respect actual domain physics, undermining the fidelity and trustworthiness of local explanations derived from standard LIME.

DL-LIME addresses this by leveraging a deep neural network to model feature correlations observed during actual agent operation, thereby producing perturbations that are consistent with the problem’s structure.

2. Methodological Framework

2.1. Modified LIME Sampling

Standard LIME:

For a given time step $t$ and state $\mathbf{s}_t$, perturbed samples are generated as

$$\mathbf{s}_t^{(k)} = \mathbf{s}_t + \mathbf{n}^{(k)},\quad n_m^{(k)} \sim \mathcal{N}(0, \sigma_m^2)$$

with independent noise for each feature $m$. This results in perturbed states that may violate real feature correlations.

DL-LIME Modification:

• Partition the feature vector into uncorrelated and correlated subsets.
• Sample uncorrelated features by adding noise as above.
• Use a deep neural network, trained on the agent’s experience, to generate correlated features given sampled uncorrelated ones. This step effectively samples from the conditional distribution $$P(\text{correlated features} ~|~ \text{uncorrelated features})$$ as learned by the DNN.

This process ensures all samples remain on the data manifold and maintain physically plausible dependencies, a crucial factor for high-stakes domains like RRM.

2.2. Surrogate Model Fitting

Rather than altering the surrogate model itself, DL-LIME fits a local linear regression (for each action dimension): $$\mathcal{L}(\mathbf{w}, b) = \sum_{k=1}^K w(\mathbf{s}_t^{(k)})\|\pi(\mathbf{s}_t^{(k)}) - (\mathbf{w}^T\mathbf{s}_t^{(k)} + b)\|^2 + c\|\mathbf{w}\|^2$$ where $w(\mathbf{s}_t^{(k)})$ is a kernel-based similarity weight to the reference point, $\pi(\cdot)$ is the DRL policy, and $(\mathbf{w}, b)$ are the surrogate parameters. The weights provide the local feature importance for that decision.

3. Application in Radar Resource Management

DL-LIME was applied within a constrained DRL system that regulates radar time allocation among tracking multiple targets and scanning. The environment state vector includes multiple correlated features per target (positions, tracking costs—which depend on positions and historical actions—and a dual variable for resource constraints).

• During explanation, only physically meaningful perturbations are produced, and the resulting local linear surrogate provides actionable feature attributions for each radar dwell allocation.
• Explanations demonstrate which historical and instantaneous quantities (e.g., previous tracking costs, other targets’ costs) drive the agent’s allocations at each step.

4. Empirical Results and Performance Analysis

Key empirical findings include:

Method	MAE	Utility ($\times 10^4$)	Runtime (s)	Peak Perf. Period
DDPG Agent	—	4.39	—	48.57%
LIME	2.27	4.01	0.42	11.20%
DL-LIME	1.95	4.49	1.70	40.23%

• DL-LIME reduces surrogate Mean Absolute Error (MAE) compared to LIME (1.95 vs. 2.27), indicating greater local fidelity.
• Task performance (utility) using the surrogate model is increased under DL-LIME, closely matching the unconstrained agent’s strategy.
• Explanatory insights generated by DL-LIME align with domain knowledge: tracking allocations are primarily driven by the previous step’s target-specific tracking cost, and to a lesser extent, other targets’ states, reflecting reasonable and trusted resource management decisions.
• Runtime is higher due to DNN sampling, but can be managed by trading off perturbation batch size.

5. Interpretability and Insights in High-Correlation Environments

DL-LIME’s approach enables reliable interpretation of a DRL agent’s decisions in complex, structured domains. Specifically:

• Dominant features are consistently revealed as the most recent relevant costs or positions for each allocation decision, with credible secondary roles for global, multi-target context.
• Inter-feature dependencies, otherwise obscured in standard LIME’s explanations, are preserved in the perturbed samples, increasing the trustworthiness of local surrogate attributions.
• Adaptivity in explanations: When overall tracking demand is low, explanations show shifting importance toward scanning actions, consistent with expected optimal policy behavior.

6. Comparison with Conventional LIME and Generalization Potential

DL-LIME demonstrates clear advantages over standard LIME in the presence of feature correlations:

• Physically plausible perturbations avoid misleading explanations caused by out-of-manifold sampling.
• Improved explanation fidelity and utility metrics across time, supporting operational adoption in safety- and mission-critical systems.
• The methodological principle—replacing LIME’s independent sampling with deep generative modeling conditioned on observed data structure—is generalizable to high-dimensional, correlated domains beyond RRM, including healthcare, finance, and autonomous systems.

A plausible implication is that similar DL-LIME approaches may enhance local explanations in any setting where standard LIME fails due to strong (possibly non-linear) feature dependencies, provided sufficient domain data exists to train realistic generative samplers.

7. Limitations and Future Directions

• Computational overhead increases due to deep model sampling, but can be mitigated by trade-offs in sample size or using more efficient generative modeling.
• Accuracy of modeled correlations by the DNN directly impacts explanation plausibility; poor generative modeling may revert DL-LIME toward the pitfalls of the original method.
• Extending to other forms of generative models (e.g., VAEs, GANs) may further enhance the capacity to capture complex correlations.
• Applicability to other XAI tasks is promising but requires careful adaptation to domain-specific structures and training regimes.

---

Aspect	Standard LIME	DL-LIME
Sampling mechanism	Independent noise	DNN-based: respects feature correlations
Explanation fidelity (MAE)	2.27	1.95
Task utility	$4.01 \times 10^4$	$4.49 \times 10^4$
Interpretability	May generate implausible	Produces trustworthy, domain-aware
Runtime (per explanation)	0.42s	1.70s

DL-LIME represents a substantive enhancement of LIME for deep models operating in correlated domains, providing robust, high-fidelity, and trustworthy explanations indispensable for critical decision-making systems (Lu et al., 26 Jun 2025).