eDySec: A Deep Learning-based Explainable Dynamic Analysis Framework for Detecting Malicious Packages in PyPI Ecosystem

Abstract: The security of open-source software repositories is increasingly threatened by next-gen software supply chain attacks. These attacks include multiphase malware execution, remote access activation, and dynamic payload generation. Traditional Machine Learning (ML) detectors struggle to detect these attacks due to the high-dimensional and sparse nature of dynamic behavioral data, including system calls, network traffic, directory access patterns, and dependency logs. As a result, these data characteristics degrade the performance, stability, and explainability of ML models. These challenges have made Deep Learning (DL) a promising alternative, given its success across various domains and its potential for modeling complex patterns. This paper presents eDySec, a DL-based efficient, stable, and explainable framework for dynamic behavioral analysis to detect malicious packages. Using the QUT-DV25 dataset, which captures both install-time and post-installation behaviors of packages, we evaluate DL models and investigate feature sets to identify the most discriminative attributes for enabling efficient malicious package detection. Additionally, model stability analysis and explainable AI techniques are incorporated into the detection pipeline to enable stable, and transparent interpretations of model decisions. Experimental results demonstrate that eDySec significantly outperforms the state-of-the-art frameworks. Specifically, it halves feature dimensionality while lowering false positives by 82% and false negatives by 79%. It also improves accuracy by 3%, achieves near-perfect stability, and maintains an inference latency of 170ms per package. Further analysis reveals that feature and model selection play a critical role, as certain combinations degrade performance. Ultimately, this study advances the understanding of the strengths and limitations of dynamic analysis against next-gen attacks.

• The paper introduces eDySec, a framework leveraging dynamic behavioral analysis and DL to detect malicious PyPI packages.
• It utilizes the QUT-DV25 dataset and FLAML-based feature reduction, achieving F1 scores up to 0.99 and significant error rate reductions.
• The framework integrates SHAP and LIME for explainability, ensuring robust, transparent, and deployable security measures.

Deep Learning-based Explainable Dynamic Analysis for Detecting Malicious Packages in PyPI: The eDySec Framework

Introduction

Modern software supply chains are increasingly targeted by advanced and stealthy attacks, exploiting open-source repository ecosystems such as PyPI. Traditional ML-based methods relying on metadata or static code analysis are fundamentally limited against next-gen supply chain attacks, which leverage multiphase and dynamic execution, often revealing malicious behavior only during install-time or post-installation. The eDySec framework addresses this challenge by introducing an efficient, stable, and explainable deep learning (DL)-based architecture that utilizes dynamic behavioral analysis leveraging QUT-DV25, a dynamic trace dataset capturing real-world PyPI ecosystem behaviors.

Figure 1: Overall system architecture of the proposed eDySec framework for detecting malicious packages.

QUT-DV25 Dataset and Dynamic Trace Foundations

eDySec's experimental backbone is the QUT-DV25 dataset, the only large-scale, real-world behavioral dataset capturing both benign and malicious packages during install and post-install phases. It comprises granular trace categories, including system calls, file operations, directory accesses, network activities, and derived execution patterns. This dataset provides an almost perfectly balanced class distribution, essential for unobscured evaluation of classifier discrimination.

Figure 2: Overview of the QUT-DV25 dataset: (a) statistics of the feature sets across traces; (b) class distribution of benign and malicious packages.

Comprehensive data exploration, including t-SNE projections, demonstrates that dynamic traces from QUT-DV25 yield markedly superior class separation compared to static feature spaces, empirically reinforcing the hypothesis that run-time evidence is crucial for effective malicious package detection.

Figure 3: t-SNE visualization of 200 randomly selected samples (100 benign and 100 malicious) using (a) metadata, (b) static, and (c) dynamic features, where the QUT-DV25 dynamic features show clearer class separation.

Feature Selection and Model Regimes

eDySec incorporates multi-pronged feature selection including classical statistical tests, probabilistic optimization (PSO, WOA), and AutoML-centric mechanisms (FLAML). Notably, FLAML achieves state-of-the-art performance while offering over 50% dimensionality reduction, highlighting the overparametrization and redundancy present in raw behavioral data.

Figure 4: Performance comparison of feature selection methods on the QUT-DV25 dataset using the MLP model.

Combinations of traces—especially when uniformly reduced via FLAML—outperform single-trace or naively selected feature sets across all evaluated model architectures. The optimal base comprises 17 dynamic features (out of 36), yielding maximum discriminative strength.

Deep Learning Model Comparison and Performance Analysis

A spectrum of DL models is compared, including MLP, CNNs, LeNet, MDCNN, as well as sequential (LSTM, RNN) and attention-based architectures (Transformer, BERT, DistilGPT2). Empirically, lightweight classical architectures (MLP, LeNet, MDCNN, simple NN) consistently dominate, achieving F1-scores of 0.99 and reducing both FPR and FNR by up to 82% and 79%, respectively, over strong ML baselines (e.g., Random Forest, SVM) (2604.26219).

Evaluation demonstrates that increased network complexity (i.e., deeper or attention-based) confers no consistent further benefit, attributed to the already highly informative, well-structured dynamic input space.

Figure 5: Performance of FLAML-based MLP model on the QUT-DV25 Combined traces dataset: (a) accuracy and (b) loss.

Figure 6: Performance of FLAML-based MLP model on the QUT-DV25 Pattern traces dataset: (a) accuracy and (b) loss.

Robustness: Model Stability and Generalization

A critical dimension in eDySec is the rigorous quantification of model stability under repeated training/testing splits. Classical DL models demonstrate mean F1-scores in the 0.988 range with 0.004 standard deviation and near-perfect stability metrics (0.996). Even under cross-trace variation, error rates and F1-scores demonstrate invariance, confirming the robustness and practical deployability of the framework.

Figure 7: Comparison of (a) FPR and FNR across trace types using the MLP model, and (b) model stability across all trace types measured by F1-score.

Explainable AI Integration for Security Transparency

To address the black-box nature of high-capacity DL, eDySec integrates SHAP and LIME for both global and instance-level explanation. SHAP analysis consistently highlights process activity, I/O operations, state transitions, and derived execution patterns as dominant factors—corresponding to security-relevant, semantically interpretable features strongly aligned with software supply chain attack vectors.

Figure 8: Global SHAP summary of the most influential features for malicious package detection. The distribution of SHAP values shows how each feature contributes to benign (left) or malicious (right) predictions across all samples. Color indicates feature magnitude (blue: low, red: high).

Instance-level (waterfall) SHAP explanations confirm class-discriminative attribution, as do LIME visualizations, ensuring that both global and local rationales for predictions are consistent, robust, and security-meaningful.

Figure 9: SHAP waterfall explanations for representative samples. (a) Benign sample; (b) malicious sample, showing how individual features contribute to the final prediction.

Figure 10: Local explanations for representative samples. (a) Benign sample and (b) malicious sample explanations.

Comparative Evaluation and Computational Aspects

Compared to the prior art (DySec, PyRadar, etc.), eDySec attains superior performance on all key metrics—including AUC, F1, FPR, FNR—while cutting computational overhead and feature space size. MLP provides an optimal balance between accuracy, speed, and interpretability, with 170 ms inference latency per package, making it suitable for CI/CD integration or repository-level deployment.

Implications and Future Trajectories

Practical implication: The findings advocate for the operational deployment of dynamic behavioral analysis and lightweight classical DL models in real-time package vetting, substantially raising the bar for software supply chain security in the PyPI ecosystem.

Theoretical implication: The results highlight that, in high-quality dynamic scenarios, model complexity is a secondary lever; proper feature engineering/selection and robust training yield greater benefits than deep or attention-based architectures.

Research outlook: eDySec paves the way for cross-ecosystem generalization, adversarial robustness studies, dynamic trace augmentation, and integration with LLM-assisted hybrid detection pipelines. Kernel-level trace collection combined with explainable DL forms a foundational paradigm for future supply chain neurosecurity frameworks.

Conclusion

eDySec establishes a new benchmark for dynamic malicious package detection in open-source Python ecosystems. Its rigorous integration of efficient feature selection, robust classical DL, and interpretable decision pipelines sets a foundation for practical, trustworthy software supply chain defense against stealthy and advanced attack vectors (2604.26219).