Knowledge Dispersion Index (KDI)

• KDI is a quantitative metric that operationalizes knowledge dispersion through precise mathematical models applied to LLM responses, organizational metrics, and citation structures.
• It systematically measures diversity and reliability by analyzing response variance, intellectual capital flows, and influence mapping in scientific networks.
• Empirical analyses show KDI’s effectiveness in benchmarking model accuracy and forecasting scholarly impact, supporting robust model selection and strategic insights.

The Knowledge Dispersion Index (KDI) refers to a class of quantitative metrics designed to capture the spread, diversity, or reliability of knowledge within a system, model, or organization. Multiple KDI formulations exist in the research literature, each contingent on the underlying domain—LLM response evaluation, intellectual capital, scientific influence, field-mapping, or information-theoretic model comparison. All share the goal of operationalizing the concept of “dispersion” as it applies to knowledge, information flow, or citation structure.

1. KDI for LLM Response Dispersion

The most recent instantiation of KDI is as a black-box metric for benchmarking domain-specific knowledge in LLMs without requiring labeled datasets or access to model internals (II, 2024). Here, the KDI—termed “response dispersion”—quantifies the variability in an LLM’s answers to repeated prompts within a fixed domain. The precise definition is as follows:

• Run N independent generations of an LLM on a fixed opinion-style prompt within a domain (e.g., N=100, prompt asks for a favorite single-word topic in a category).
• Embed each single-word response $t_i$ into a vector in $\mathbb{R}^d$ using one of:
  • OpenAI’s text-embedding-3-large (d≈1536)
  • Reference-Sentence-Similarity (RSS) embeddings (d=N, based on pairwise string similarity).
• Stack responses into an $E\in\mathbb{R}^{N\times d}$ embedding matrix.
• Compute the singular value decomposition $E = U\Sigma V^\top$, with singular values $\sigma_1\ge\sigma_2\ge\dots$.
• Let $\tau=0.95$ (95% of total variance). Define $$\mathit{Var}(k)=\frac{\sum_{i=1}^{k}\sigma_i^2}{\sum_{i=1}^m\sigma_i^2}$$, $m=\min(N,d)$. The KDI is $k^* = \min\{k:\mathit{Var}(k)\ge \tau\}$.

Thus, KDI specifies the number of principal directions required to account for 95% (or another chosen threshold) of the response space variability for a topic+model pair.

Empirical findings indicate an average Spearman rank correlation of approximately $-0.59$ between KDI and gold-standard QA accuracy across models and domains (lower KDI, signifying less diverse answers, corresponds to higher QA accuracy). For pairwise model selection, KDI matches or approximates accuracy-based rankings in roughly 74–89% of cases, depending on chosen tolerance for accuracy trade-off (II, 2024).

2. KDI in Organizational Intellectual Capital

KDI was originally introduced to quantify knowledge flow and intellectual capital at the micro- (organizational) and macro- (economic sector) levels (Dhillon, 2011). The construction is based on aggregating normalized indicators:

• Micro-KDI: Weighted sum of up to 23 human- and function-oriented organizational metrics (e.g., patents pending, % R&D in basic research, IT capacity). Raw values $\mathbb{R}^d$0 are normalized $\mathbb{R}^d$1, then KDI$\mathbb{R}^d$2 with weights $\mathbb{R}^d$3 s.t. $\mathbb{R}^d$4.
• Macro-KDI: Aggregates flow metrics for industrial (inter-firm technical knowledge) and consumer (public broadcast and media dissemination) sectors. Let $\mathbb{R}^d$5 be suitable flow indices, then KDI$\mathbb{R}^d$6.
• Total KDI: $\mathbb{R}^d$7, with analyst-chosen weights $\mathbb{R}^d$8.

Theoretical extension involves mapping knowledge flow as a network graph with nodes, capacity constraints, and self-correcting equilibria (modeled analogously to max-flow/min-cut). Robustness is assessed by studying network “super-families” and propagation under perturbations (Dhillon, 2011).

3. KDI via Influence Dispersion in Citation Networks

Another formalization of KDI is tied to the topological structure of a scientific paper’s influence in citation networks, notably through the Influence Dispersion Tree (IDT) model (Mohapatra et al., 2019). The construction is as follows:

• Build the IDT for a focal paper $\mathbb{R}^d$9 with $E\in\mathbb{R}^{N\times d}$0 citing papers $E\in\mathbb{R}^{N\times d}$1, assigning edges so as to maximize tree depth (parent selection favors farthest ancestor).
• Compute the Influence Dispersion Index (IDI): $E\in\mathbb{R}^{N\times d}$2.
• The Normalized Influence Divergence (NID) is defined as $E\in\mathbb{R}^{N\times d}$3, where $E\in\mathbb{R}^{N\times d}$4 is the maximal dispersion tree given $E\in\mathbb{R}^{N\times d}$5.
• The Knowledge Dispersion Index is then $E\in\mathbb{R}^{N\times d}$6, so $E\in\mathbb{R}^{N\times d}$7 with higher values denoting balanced dispersion (depth $E\in\mathbb{R}^{N\times d}$8 breadth $E\in\mathbb{R}^{N\times d}$9).

Empirical analysis on large bibliometric corpora shows that $E = U\Sigma V^\top$0 is more predictive than simple citation counts in forecasting future impact and identifying highly influential papers (Mohapatra et al., 2019).

4. KDI as Diversity-Coherence Composite in Knowledge Mapping

KDI is also used as a composite metric integrating cognitive diversity and coherence, especially in knowledge integration and diffusion studies (Rafols, 2014). The method proceeds as:

• Diversity (Rao–Stirling): $E = U\Sigma V^\top$1, where $E = U\Sigma V^\top$2 is categorical share and $E = U\Sigma V^\top$3 the cognitive distance (often $E = U\Sigma V^\top$4 minus cosine similarity).
• Coherence: $E = U\Sigma V^\top$5, where $E = U\Sigma V^\top$6 indicates relational intensity (e.g., citation, co-occurrence).
• Composite KDI: $E = U\Sigma V^\top$7, with normalized terms and $E = U\Sigma V^\top$8.

The approach is sensitive to the categorization scheme and the definition of cognitive distance. Visualization overlays map diversity and coherence values onto “basemaps” of science or technology (Rafols, 2014).

5. KDI as Dispersion Index in Information Theory

In information theory, KDI describes the variance of the pointwise Kullback–Leibler (KL) divergence between two probability distributions $E = U\Sigma V^\top$9 and $\sigma_1\ge\sigma_2\ge\dots$0 (Buono et al., 2021). The definition is:

$\sigma_1\ge\sigma_2\ge\dots$1

This quantity measures the spread (reliability) of the instantaneous KL divergence and is used to supplement the mean divergence in model selection via a mean–variance trading rule. KDI is always nonnegative, vanishes iff $\sigma_1\ge\sigma_2\ge\dots$2 almost surely, and is not a metric. It is particularly recommended when models have similar average divergence but differ in variance of fit (Buono et al., 2021).

6. Cross-Contextual Properties and Implementation Considerations

Despite domain diversity, KDIs share systematic features:

• Normalization and Intelligibility: KDIs are either directly interpretable as (dimension-count, network) indices or are normalized to $\sigma_1\ge\sigma_2\ge\dots$3 for comparison.
• Embedding and Representation: Construction may depend on data embedding (e.g., response embeddings for LLMs, vectorized categories for knowledge mapping, trees for citation analysis).
• Robustness: KDI metrics must be stress-tested against category choice, sample size, and parametrization (e.g., choice of thresholds $\sigma_1\ge\sigma_2\ge\dots$4 or normalization bases).
• Empirical Utility: Where validated, KDIs serve as surrogates or complements to more direct but expensive measures (e.g., QA accuracy, human-judged impact, manual diversity analyses).
• Limiting Factors: Predictive power and robustness are context sensitive—e.g., KDI may underperform in domains with inherently high answer variability or ill-defined conceptual distances.

7. Summary Table: Major KDI Variants

Context	Definition/Formula	Application Domain
LLM Response Dispersion	$\sigma_1\ge\sigma_2\ge\dots$5 in SVD of response embeddings	LLM benchmarking (II, 2024)
Organizational Knowledge	Weighted sum of normalized micro/macro indicators	Intellectual capital (Dhillon, 2011)
Citation Influence	$\sigma_1\ge\sigma_2\ge\dots$6 Normalized Influence Divergence (NID)	Scholarly impact (Mohapatra et al., 2019)
Diversity-Coherence Mapping	$\sigma_1\ge\sigma_2\ge\dots$7	Science mapping (Rafols, 2014)
Information-Theoretic	$\sigma_1\ge\sigma_2\ge\dots$8 variance of $\sigma_1\ge\sigma_2\ge\dots$9	Model selection (Buono et al., 2021)

Each KDI variant is anchored in a precise mathematical formulation and an explicit workflow, as detailed in the cited works. Adherence to these domain-specific definitions is necessary for valid computation, interpretation, and comparative analysis.