SMSI: System Model Security Inference: Automated Threat Modeling for Cyber-Physical Systems

Abstract: Threat modeling for cyber-physical systems (CPS) remains a largely manual exercise. This project presents SMSI (System Model Security Inference), a hybrid neuro-symbolic pipeline that starts from a SysML architecture model and produces a prioritized list of NIST 800-53 security controls. The prototype has three main stages: a deterministic parser mapping system components to vulnerabilities via the NVD; a family of retrieval and classification models linking vulnerabilities to MITRE ATT&CK techniques; and a control recommender. We explore three approaches for CVE-to-ATT&CK mapping: a supervised classifier using fine-tuned SecureBERT+, retrieval-based dense encoders, and a zero-shot LLM approach using Gemma-4 26B. We validate the pipeline on a healthcare IoT gateway with nine software components. For the ATT&CK-to-NIST stage, pretrained SecureBERT achieves the highest control retrieval scores, demonstrating that dense embeddings provide a strong basis for automated control recommendation.

• The paper introduces SMSI, an end-to-end neuro-symbolic pipeline that automates threat modeling in cyber-physical systems.
• It employs a multi-model design that parses SysML models, maps CPEs to CVEs, and generates prioritized NIST 800-53 control recommendations.
• Empirical results show that zero-shot LLMs enhance rare-class recall over dense models, significantly improving threat triage and traceability.

SMSI: Automated Threat Modeling for Cyber-Physical Systems

Overview and Motivation

Threat modeling in cyber-physical systems (CPS) remains a labor-intensive, non-scalable task due to the need to map system architectures, component vulnerabilities, adversary behaviors, and recommended safeguards. The sharp growth in the number of disclosed vulnerabilities—over 44,000 CVEs in 2025 alone—renders manual approaches infeasible. SMSI (System Model Security Inference) addresses this by providing an end-to-end neuro-symbolic pipeline capable of ingesting SysML models and generating prioritized, traceable NIST 800-53 control recommendations. The pipeline operationalizes the full stack from architecture to actionable controls, enabling rapid, repeatable, and auditable threat modeling.

Pipeline Structure and Core Methodology

Architecture: Multi-Model Hybrid Design

SMSI comprises five sequential stages:

1. SysML System Parsing: Deterministic extraction of deployed components and CPEs from SysML XML.
2. CPE-to-CVE Resolution: Mapping extracted CPEs to CVEs from the NVD, preserving component-CVE traceability.
3. CVE-to-ATT{CK} Technique Mapping: Three distinct strategies are deployed:
   • Supervised Multi-Label Classification: SecureBERT+ fine-tuned on a comprehensive synthetic dataset constructed via CWE⟶CAPEC⟶ATT{CK} transitive paths and KEV labels.
   • Dense Retrieval Models: TF-IDF and multiple pretrained/fine-tuned dense encoders (SecureBERT, ATTACK-BERT, MiniLM) with KEV supervision.
   • Zero-Shot LLM: Gemma-4 26B via structured prompts, tested in both plain and database-hinted configurations.
4. ATT{CK}-to-NIST 800-53 Control Recommendation: Hybrid approach based on the CTID crosswalk (deterministic), augmented by TF-IDF and embedding-based similarity, all prioritized by CVSS severity.
5. Report Generation: Produces a traceable mapping from each SysML component to recommended controls via explicit intermediary links.

This architecture results in full traceability from system block to control, supports multi-modal inference, and allows risk-weighted triage.

Empirical Results and Model Comparisons

CVE-to-ATT{CK} Mapping

• Retrieval Models: Unsupervised MiniLM achieved the top retrieval metrics (MRR 0.252, Hits@10 0.578), outperforming both SecureBERT and ATTACK-BERT, regardless of supervision. SecureBERT underperformed, even below TF-IDF.

  Figure 1: CVE-to-ATT{CK} retrieval: all methods vs KEV ground truth (n=45 test CVEs). MiniLM dominates all metrics.

• Signal Independence: Pearson correlations revealed that TF-IDF and dense models encode largely orthogonal signals, with fine-tuned and unsupervised models displaying varying degrees of alignment.

  Figure 2: Pearson correlation between all CVE-to-ATT{CK} model scores. TF--IDF and dense models capture largely independent signals.

• Supervised Multi-Label Classification: SecureBERT+ classifier achieved Micro F1 of 0.757 but only Macro F1 of 0.383. Low Macro F1 corroborates severe class imbalance, as seen in the per-class F1 distribution.

  Figure 3: SecureBERT+ aggregate test metrics at optimal threshold 0.45 (105 parent techniques). Hamming loss is 0.032 (table).

  

  Figure 4: SecureBERT+ per-class F1: best 10 vs worst 10 parent techniques. The long tail of zero-scoring classes drives the low Macro F1.

• Zero-Shot LLM: The Gemma-4 26B model achieved an unprecedented Hit Rate@1 of 51.8% on the full KEV CVE set—over four times higher than the advanced embedding models. Coverage for rare classes (Macro F1 on SMET of 0.405) was also substantially better than the supervised classifier.

  Figure 5: Hit Rate comparison: Gemma-4 26B vs ATTACK-BERT embedding baseline on KEV (419 CVEs) and SMET (302 CVEs).

Comparative Tradeoffs: SecureBERT+ dominates on precision and Micro F1, whereas the LLM is stronger in recall and rare class recovery, as summarized in their direct confrontation.

Figure 6: Supervised SecureBERT+ fine-tuning vs zero-shot Gemma-4 26B. The supervised model leads on precision and F1; the LLM is more competitive on recall and rare-class coverage.

ATT{CK}-to-NIST 800-53 Control Recommendation

• Embedding Models: Pretrained SecureBERT, rather than any fine-tuned or generic encoder, achieved the highest retrieval metrics (MRR 0.582, Hits@10 0.798) when matched against the official CTID crosswalk.

  Figure 7: ATT{CK}-to-controls retrieval: all methods vs CTID crosswalk ground truth (n=94). Pretrained SecureBERT dominates.

• Prioritization Distribution: The hybrid CVSS-weighted score produced a heavy-tailed priority distribution, aligning with triage needs—most recommendations are low-priority but a small fraction are flagged as critical.

  Figure 8: Priority distribution for technique-control pairs (priority = hybrid_mapping_soi times max_cvss); clipped at the 99th percentile.

End-to-End Case Study

The synthetic MedGateway case validates the full stack: 9 components, 199 CVEs, complete traceability from model entry through ATT{CK} mapping to actionable NIST recommendations. Severe vulnerabilities such as Log4Shell are rapidly surfaced and matched to their canonical mitigation controls.

Implications and Future Directions

Theoretical Implications

SMSI’s findings confirm that neuro-symbolic fusion outperforms either symbolic or neural techniques in isolation for threat modeling:

• Dense retrievers can capture signal ignored by classical lexical models, but not all security-specific pretraining transfers well to cross-domain retrieval.
• Zero-shot LLMs can dramatically enhance rare-class recall, though their F1 lags in abundance-heavy settings.
• Pretrained domain encoders excel at mapping between dense technical narratives (e.g., ATT{CK} and NIST control texts), but not when the domain language diverges (e.g., vulnerability reports).

Practical Impact

• Automation of Laborious Analysis: SMSI reduces multi-day manual workflows to automated, fully transparent reports suitable for analyst validation. The approach supports real-time updates and auditability.
• Triage and Risk Prioritization: Integration of CVSS weighting ensures that only high-impact threats are escalated, optimizing analyst workload.
• Ensembling Potential: By deploying multiple paradigms (supervised, dense retrieval, and LLM), the pipeline can be tuned for deployment-specific objectives (e.g., precision, recall, triage robustness).

Future Research

• Enhanced Contextual Weighting: Incorporating deployment context (network exposure, data sensitivity) within the risk model to further refine control prioritization.
• Dynamic Knowledge Graph Reasoning: Deploying the CWE/CAPEC graph for real-time inference, enabling deeper neuro-symbolic reasoning.
• Comprehensive Ensembling: Calibrating score fusion among classifiers, retrievers, and LLMs to dynamically reweight according to incident response context.
• Practitioner Benchmarks: Conducting field studies with professional security analysts to quantify reductions in manual effort and validate analytical relevance.

Conclusion

SMSI establishes that fully automated, traceable threat modeling is feasible for CPS environments when integrating symbolic and neural models. The pipeline’s modularity permits rapid evolution as models and datasets improve. No single modeling class is sufficient—dense retrieval, supervised learning, and generative LLMs must all be leveraged to balance coverage, accuracy, and traceability. SMSI thus provides a compelling reference framework for future operationalization of AI-driven cybersecurity analysis.