Threat Intelligence Effectiveness Index

• Threat Intelligence Effectiveness Index (TIEI) is a composite metric that measures the effectiveness of cyber threat intelligence by combining scores on intelligence quality, enrichment, integration, and operational impact.
• It employs a non-linear weighted geometric mean to capture weakest-link behavior, ensuring that deficiencies in any component significantly lower the overall index.
• The index guides decision-making by linking operational CTI performance to measurable defensive outcomes and ROI in cybersecurity.

Searching arXiv for the cited TIEI and related CTI evaluation papers to ground the article in current literature. arXiv search results for "Threat Intelligence Effectiveness Index TIEI cyber threat intelligence":

• (Strada, 23 Jul 2025) — Quantifying the ROI of Cyber Threat Intelligence: A Data-Driven Approach
• (Freitas et al., 2024) — Web Scale Graph Mining for Cyber Threat Intelligence
• (Alevizos et al., 2024) — Threat-Informed Cyber Resilience Index: A Probabilistic Quantitative Approach to Measure Defence Effectiveness Against Cyber Attacks
• (Al-Ibrahim et al., 2017) — Beyond Free Riding: Quality of Indicators for Assessing Participation in Information Sharing for Threat Intelligence
• (Mitra et al., 2024) — LOCALINTEL: Generating Organizational Threat Intelligence from Global and Local Cyber Knowledge
• (Nguyen et al., 6 May 2025) — Towards Effective Identification of Attack Techniques in Cyber Threat Intelligence Reports using LLMs
• (Liu et al., 28 Feb 2025) — CyLens: Towards Reinventing Cyber Threat Intelligence in the Paradigm of Agentic LLMs
• (Al-Yasiri et al., 4 Jun 2025) — A Threat Intelligence Event Extraction Conceptual Model for Cyber Threat Intelligence Feeds
• (Nakano et al., 3 Jun 2026) — TIBlender: Early-Warning Threat Intelligence from Cross-Platform Social Media Evidence Threat Intelligence Effectiveness Index (TIEI) denotes a family of composite measures for quantifying how effectively cyber threat intelligence is generated, enriched, integrated into security operations, and translated into measurable defensive outcomes. In its explicit formalization, TIEI is a weighted geometric mean over four component scores—Intelligence Quality, Enrichment, Integration & Automation, and Operational Impact—each normalized to a 0–100 scale; related work extends the same concept toward graph-based disruption performance, quality-of-indicator assessment, ATT&CK-space extraction quality, contextualized organizational intelligence, multilingual event extraction, early-warning lead time, and threat-informed resilience uplift (Strada, 23 Jul 2025).

1. Formal definition and semantic scope

The formal definition introduced in the ROI-oriented CTI literature specifies component scores $s_k \in (0,100]$ and weights $w_k > 0$, for $k \in \{1,2,3,4\}$, with $\sum_{k=1}^{4} w_k = 1$, where

$$\mathbf{s} = (s_1, s_2, s_3, s_4) = (Q_{\text{score}}, E_{\text{score}}, I_{\text{score}}, O_{\text{score}}).$$

The index is then

$$\mathrm{TIEI} = 100 \prod_{k=1}^{4} \left( \frac{s_k}{100} \right)^{w_k}, \qquad \mathrm{TIEI} \in (0,100].$$

A floor of $s_k = 1$ is imposed to avoid $\log 0$ and geometric-mean collapse unless a hard zero is intentionally used to denote total absence of capability (Strada, 23 Jul 2025).

This construction is explicitly non-linear. Its purpose is not merely to summarize CTI activity, but to encode weakest-link behavior: strong intelligence quality cannot compensate for broken integration, and extensive automation cannot compensate for poor enrichment or minimal operational effect. This suggests that TIEI is best understood as a systems-level effectiveness index rather than a single-model accuracy score.

Across adjacent work, the same term or closely related formulations are used more broadly. In graph-mining CTI, effectiveness is decomposed into predictive accuracy, coverage, timeliness, and operational impact; in information-sharing research, it is decomposed into correctness, relevance, utility, and uniqueness; in threat-informed resilience modeling, it is framed as the marginal uplift attributable to CTI under a fixed defensive baseline (Freitas et al., 2024).

2. Core dimensions, sub-metrics, and normalization

In the four-component formulation, each component score is itself a weighted arithmetic mean of normalized sub-metrics. Intelligence Quality $Q$ captures the quality of raw intelligence before downstream use. Its sub-metrics are Accuracy, Timeliness, Relevance, and Duplicates, with weights $0.35$, $w_k > 0$0, $w_k > 0$1, and $w_k > 0$2, respectively. The normalization rules are given explicitly: $w_k > 0$3

$w_k > 0$4

$w_k > 0$5

$w_k > 0$6

and

$w_k > 0$7

The raw semantics are also fixed: Accuracy is “% of artifacts validated as correct (no false positives),” Timeliness is “Median hours from external sighting to internal availability,” Relevance is “% of artifacts mapped to organizational assets or TTPs,” and Duplicates is “% of duplicates removed prior to processing” (Strada, 23 Jul 2025).

Enrichment $w_k > 0$8 measures how thoroughly raw artifacts are contextualized. Its sub-metrics are MITRE ATT&CK Coverage, Internal Correlation, and Actionability Notes, weighted $w_k > 0$9, $k \in \{1,2,3,4\}$0, and $k \in \{1,2,3,4\}$1: $k \in \{1,2,3,4\}$2 The corresponding raw measures are “% of artifacts tagged to ATT&CK TTPs,” “% of artifacts linked to internal telemetry,” and “% of artifacts containing operational guidance” (Strada, 23 Jul 2025). This establishes enrichment as a contextualization function: intelligence is not merely collected, but linked to ATT&CK space, local telemetry, and concrete analyst actions.

Integration & Automation $k \in \{1,2,3,4\}$3 measures how well CTI is wired into the security stack. Its sub-metrics are Control Breadth, Feed Health, Automation Utilization, and Ticket Assist, with weights $k \in \{1,2,3,4\}$4, $k \in \{1,2,3,4\}$5, $k \in \{1,2,3,4\}$6, and $k \in \{1,2,3,4\}$7: $k \in \{1,2,3,4\}$8 The raw measures are “% of critical security tools ingesting CTI feeds,” “Feed pipeline uptime,” “% of artifacts automatically applied (blocking, alerting, etc.),” and “% of tickets resolved with CTI enrichment” (Strada, 23 Jul 2025). This component formalizes a recurrent theme in the literature: intelligence has limited value if it does not reach SIEM, SOAR, EDR, IPS, WAF, email gateways, or incident workflows.

Operational Impact $k \in \{1,2,3,4\}$9 links CTI to incident handling and risk reduction. Its sub-metrics are Detection Lift on MTTD, Response Lift on MTTR, Prevented Events, and Risk Reduction via $\sum_{k=1}^{4} w_k = 1$0, weighted $\sum_{k=1}^{4} w_k = 1$1, $\sum_{k=1}^{4} w_k = 1$2, $\sum_{k=1}^{4} w_k = 1$3, and $\sum_{k=1}^{4} w_k = 1$4: $\sum_{k=1}^{4} w_k = 1$5 The risk term is defined by

$\sum_{k=1}^{4} w_k = 1$6

A plausible implication is that this component is the main bridge between technical CTI metrics and executive decision-making, because it embeds avoided loss directly inside the index rather than treating it as a separate business-case calculation (Strada, 23 Jul 2025).

3. Aggregation logic and theoretical antecedents

The choice of a weighted geometric mean is motivated explicitly by non-compensatory behavior. In the illustrative example with $\sum_{k=1}^{4} w_k = 1$7, a baseline score vector $\sum_{k=1}^{4} w_k = 1$8 yields a weighted arithmetic mean of $\sum_{k=1}^{4} w_k = 1$9 and a geometric TIEI of approximately $$\mathbf{s} = (s_1, s_2, s_3, s_4) = (Q_{\text{score}}, E_{\text{score}}, I_{\text{score}}, O_{\text{score}}).$$0. If integration collapses to $$\mathbf{s} = (s_1, s_2, s_3, s_4) = (Q_{\text{score}}, E_{\text{score}}, I_{\text{score}}, O_{\text{score}}).$$1, the weighted arithmetic mean drops to $$\mathbf{s} = (s_1, s_2, s_3, s_4) = (Q_{\text{score}}, E_{\text{score}}, I_{\text{score}}, O_{\text{score}}).$$2, a change of about $$\mathbf{s} = (s_1, s_2, s_3, s_4) = (Q_{\text{score}}, E_{\text{score}}, I_{\text{score}}, O_{\text{score}}).$$3, whereas TIEI drops to approximately $$\mathbf{s} = (s_1, s_2, s_3, s_4) = (Q_{\text{score}}, E_{\text{score}}, I_{\text{score}}, O_{\text{score}}).$$4, a change of about $$\mathbf{s} = (s_1, s_2, s_3, s_4) = (Q_{\text{score}}, E_{\text{score}}, I_{\text{score}}, O_{\text{score}}).$$5. The geometric form therefore penalizes bottlenecks much more strongly than a linear aggregate (Strada, 23 Jul 2025).

A closely related precursor appears in the quality-of-indicators literature, where contribution to threat-intelligence sharing is measured not by raw quantity but by a weighted sum of correctness, relevance, utility, and uniqueness: $$\mathbf{s} = (s_1, s_2, s_3, s_4) = (Q_{\text{score}}, E_{\text{score}}, I_{\text{score}}, O_{\text{score}}).$$6 Correctness is defined as agreement with a reference model, relevance as label-weighted importance to the community, utility as the usefulness of an indicator’s features or labeling granularity, and uniqueness as non-redundancy relative to other shared indicators. This formulation does not use the name TIEI, but it provides a direct conceptual lineage: effectiveness is quality-weighted contribution, not sheer volume (Al-Ibrahim et al., 2017).

A separate lineage comes from threat-informed cyber resilience. In the CRI framework, attack campaigns are represented as ATT&CK-based attack flows and evaluated through POMDPs whose states, actions, observations, transitions, and rewards are parameterized by CTI. An explicit TIEI adaptation is then defined as the marginal improvement attributable to CTI, for example

$$\mathbf{s} = (s_1, s_2, s_3, s_4) = (Q_{\text{score}}, E_{\text{score}}, I_{\text{score}}, O_{\text{score}}).$$7

or, on the attacker-utility side,

$$\mathbf{s} = (s_1, s_2, s_3, s_4) = (Q_{\text{score}}, E_{\text{score}}, I_{\text{score}}, O_{\text{score}}).$$8

This suggests a distinct interpretation of TIEI: not absolute program maturity, but the isolated marginal value of CTI under controlled counterfactuals (Alevizos et al., 2024).

4. Graph-based and operational blueprints

A large-scale operational blueprint is provided by “Web Scale Graph Mining for Cyber Threat Intelligence,” which motivates TIEI through four recurring dimensions: how accurately risk is assigned, how much adversary surface is covered and how fresh the intelligence is, and how much operational impact the intelligence has on real attacks (Freitas et al., 2024). TITAN’s graph is a dynamic, weighted, undirected $$\mathbf{s} = (s_1, s_2, s_3, s_4) = (Q_{\text{score}}, E_{\text{score}}, I_{\text{score}}, O_{\text{score}}).$$9-partite graph over organizations, incidents, alerts, entities, and parent entities, with per-edge initial weights, decay functions, decay rates, and maximum alive times. Reputation scores are propagated through label propagation, then calibrated with temperature scaling to improve probabilistic interpretability.

The resulting metrics are unusually close to an index-ready decomposition. On predictive quality, TITAN reports average macro-F1 $$\mathrm{TIEI} = 100 \prod_{k=1}^{4} \left( \frac{s_k}{100} \right)^{w_k}, \qquad \mathrm{TIEI} \in (0,100].$$0 across 12 regions, average PR-AUC $$\mathrm{TIEI} = 100 \prod_{k=1}^{4} \left( \frac{s_k}{100} \right)^{w_k}, \qquad \mathrm{TIEI} \in (0,100].$$1, region-level macro-F1 ranging from $$\mathrm{TIEI} = 100 \prod_{k=1}^{4} \left( \frac{s_k}{100} \right)^{w_k}, \qquad \mathrm{TIEI} \in (0,100].$$2 to $$\mathrm{TIEI} = 100 \prod_{k=1}^{4} \left( \frac{s_k}{100} \right)^{w_k}, \qquad \mathrm{TIEI} \in (0,100].$$3, PR-AUC from $$\mathrm{TIEI} = 100 \prod_{k=1}^{4} \left( \frac{s_k}{100} \right)^{w_k}, \qquad \mathrm{TIEI} \in (0,100].$$4 to $$\mathrm{TIEI} = 100 \prod_{k=1}^{4} \left( \frac{s_k}{100} \right)^{w_k}, \qquad \mathrm{TIEI} \in (0,100].$$5, and 99% precision in disruptive actions, validated by customer feedback and expert review. On breadth and scale, it identifies millions of high-risk entities per week, produces a 6× increase in non-file threat intelligence, and operates on graphs up to 5.7M nodes and 21M edges in a single region. On timeliness, it updates hourly, decays and prunes stale edges, and reduces time to disrupt by 1.9×. On operational effect, it increases incident disruption rate by 21% while maintaining 99% precision (Freitas et al., 2024).

The TITAN-inspired TIEI synthesis defines four sub-indices—$$\mathrm{TIEI} = 100 \prod_{k=1}^{4} \left( \frac{s_k}{100} \right)^{w_k}, \qquad \mathrm{TIEI} \in (0,100].$$6, $$\mathrm{TIEI} = 100 \prod_{k=1}^{4} \left( \frac{s_k}{100} \right)^{w_k}, \qquad \mathrm{TIEI} \in (0,100].$$7, $$\mathrm{TIEI} = 100 \prod_{k=1}^{4} \left( \frac{s_k}{100} \right)^{w_k}, \qquad \mathrm{TIEI} \in (0,100].$$8, and $$\mathrm{TIEI} = 100 \prod_{k=1}^{4} \left( \frac{s_k}{100} \right)^{w_k}, \qquad \mathrm{TIEI} \in (0,100].$$9—combined as

$s_k = 1$0

Under the explicitly illustrative weighting scheme $s_k = 1$1, $s_k = 1$2, $s_k = 1$3, $s_k = 1$4, the paper’s synthesized example yields $s_k = 1$5. The presentation is explicitly illustrative rather than normative, but it is significant because it shows how a production CTI system can be evaluated simultaneously on model quality, graph coverage, update cadence, and incident-disruption outcomes (Freitas et al., 2024).

5. Task-specific operationalizations across CTI workflows

In ATT&CK-technique extraction from CTI reports, effectiveness is operationalized as multi-label classification quality against human-annotated ground truth. The proposed TIEI in that setting is defined in ATT&CK space using per-technique precision, recall, and F1, then aggregated via macro-F1, criticality-weighted F1, coverage of techniques above a quality threshold, and an optional fairness penalty: $s_k = 1$6

$s_k = 1$7

$s_k = 1$8

The empirical comparisons are also informative: zero-shot Llama2 variants achieve average F1 values of approximately $s_k = 1$9–$\log 0$0, original TRAM yields median F1 just over $\log 0$1, and the full pipeline with summarization, rebalancing, and retrained SciBERT improves median F1 by about 7 percentage points, with several techniques surpassing $\log 0$2 F1 (Nguyen et al., 6 May 2025).

In organizational CTI contextualization, LocalIntel treats effectiveness as accurate, contextually relevant, and consistent fusion of global CTI with local organizational knowledge. The formal problem is

$\log 0$3

with a three-phase RAG pipeline: global knowledge retrieval, local knowledge retrieval, and contextualized completion generation. Quantitative evaluation uses RAGAS, with reported mean $\log 0$4 and standard deviation $\log 0$5. A minimal TIEI in this setting is proposed as a RAGAS-centered composite, using answer accuracy/faithfulness and consistency as the immediately measurable dimensions (Mitra et al., 2024).

In multilingual CTI feed preprocessing and event extraction, the XBC conceptual model suggests a stage-wise TIEI centered on collection coverage, preprocessing quality, multilingual robustness, event extraction performance, scalability, and real-time responsiveness. The pipeline comprises data collection from Facebook groups, hacker forums, and CERT-IN feeds; multilingual language detection and segmentation; normalization, tokenization, stopword removal, lemmatization, stemming, and NER; and an XLM-RoBERTa + BiGRU + CRF architecture for event extraction. The paper emphasizes standard metrics—accuracy, precision, recall, and F1-score—together with multilinguality, noise reduction, and computational efficiency as the core axes of effectiveness (Al-Yasiri et al., 4 Jun 2025).

CyLens generalizes TIEI toward a lifecycle-aligned CTI copilot. It decomposes threat management into attribution, contextualization, detection, correlation, prioritization, and remediation, and evaluates each using task-specific metrics including accuracy, BERTScore, precision, recall, F1, IoU, Hit@10, and RMSE. This suggests that a TIEI can be factorized by lifecycle stage rather than by platform layer alone, with cross-cutting modifiers for freshness, robustness, explainability, and organizational customization (Liu et al., 28 Feb 2025).

6. Early-warning value, ROI linkage, and limitations

Early-warning CTI introduces additional TIEI dimensions that are not fully captured by static quality measures. TIBlender defines an implicit lead-time metric

$\log 0$6

and a feed-scoped absence rate

$\log 0$7

In a real-world deployment, 18.7% of overlapping IoCs were detected by TIBlender ahead of six public feeds; for APT/Ransomware campaigns matched against OTX Pulses, 23.5% were detected earlier, with mean lead time 7.2 days. The feed-scoped absence rate ranges from 83.0% to 99.6%, indicating that most IoCs surfaced by the system are absent from each evaluated feed individually. Quality control is also explicit: the human audit reports report-level false-positive rates of 2.2% and 1.8%, unsupported-IoC issue rates of 3.6% and 4.0%, major quality issue rates of 1.6% and 1.4%, and inter-annotator agreement of 98.7%. This suggests a TIEI variant for early-warning pipelines that weights timeliness, uniqueness, platform diversity, and evidence quality much more strongly than a conventional feed-evaluation index would (Nakano et al., 3 Jun 2026).

The ROI-oriented formulation links TIEI directly to FAIR and Gordon–Loeb–style models. FAIR defines

$\log 0$8

and CTI ROI is then

$\log 0$9

A separate cost-avoidance expression is

$Q$0

Because Operational Impact includes normalized $Q$1, TIEI is explicitly intended to support the otherwise difficult conversion of “negative evidence” into a defensible ROI narrative (Strada, 23 Jul 2025).

The main interpretive cautions are also explicit. TIEI depends on expert-assigned weights and target values; sub-metrics such as MTTD, MTTR, ATT&CK coverage, and $Q$2 require mature telemetry and risk-modeling practices; attribution of prevented events and risk reduction to CTI alone is intrinsically difficult because multiple controls jointly shape security outcomes; and absolute cross-organizational comparison is unstable unless weights, normalization targets, and sector context are aligned (Strada, 23 Jul 2025). Related resilience work reinforces the same point from a different angle: isolating CTI’s contribution requires holding non-TI variables fixed, modeling TI-driven control changes explicitly, and using temporal snapshots to distinguish intelligence-driven improvements from general hygiene or budget-driven changes (Alevizos et al., 2024).

A plausible implication is that TIEI should be read less as a universal scalar truth about “how secure” an organization is, and more as a calibrated, context-specific measure of how effectively threat intelligence functions within a particular operational and economic system. In that narrower sense, the literature converges strongly: effective CTI is accurate, relevant, timely, sufficiently unique, well-enriched, well-integrated, and demonstrably connected to faster detection, faster response, better disruption, or lower expected loss.