Explained Information Fraction (EIF)

• Explained Information Fraction (EIF) is a normalized metric quantifying the fraction of task-relevant or predictive information present in selected model features, applicable in RAG systems, hypothesis testing, and representation learning.
• Its key properties—boundedness in [0,1], monotonicity, and data-processing stability—provide operational guarantees for explanation fidelity and feature efficiency.
• Empirical evaluations demonstrate EIF’s practical utility by reducing hallucination rates in language models and guiding resource optimization in genetic studies and model compression.

The Explained Information Fraction (EIF) is a normalized information-theoretic metric quantifying the proportion of task-relevant or predictive information that is provably present in a selected intermediate, interpretive, or masked feature set relative to the model’s own capacity. EIF appears in several domains: certifying explanation fidelity in RAG systems (Deiseroth et al., 12 Dec 2025), quantifying loss from missing data in likelihood-based hypothesis testing (Nicolae et al., 2011), and formalizing representation usefulness in interpretability contexts under the term "Interpretive Efficiency" (Katende, 6 Dec 2025). EIF ratios are bounded in [0,1], admit operational guarantees, and generalize mutual information ratios for model and channel evaluation.

1. Definitions and Formalization

In retrieval-augmented generation for LLMs, EIF quantifies the fraction of a model’s predictive information that is rigorously certified to originate from the retrieved context spans. Formally, let $A$ be the generator and $c(x)$ the ground-truth label, with context features $M(x)$. EIF is defined as

$\text{EIF} = \frac{\,I_{\mathrm{guaranteed}\;}{\,I_{\mathrm{baseline}\;} \approx \frac{\,1 - H_b\left(p_{\mathcal{D}}(M)\right)\;}{\,1 - H_b(C)\;}}$

where $H_b(\cdot)$ denotes binary entropy, $p_{\mathcal{D}}(M)$ aggregates completeness and soundness error, and $C$ is overall accuracy (Deiseroth et al., 12 Dec 2025).

In hypothesis testing with incomplete data, EIF is the ratio between observed and complete-data log-likelihood ratios, providing a direct measure of information recovery:

$$EIF = \frac{\ell(\hat\theta_{obs}\mid Y_{obs}) - \ell(\theta_0\mid Y_{obs})}{E_{Y_{co}\mid Y_{obs},\,\hat\theta_{obs}}\left[\ell(\hat\theta_{obs}\mid Y_{co})-\ell(\theta_0\mid Y_{co})\right]}$$

This ratio is bounded by [0,1], where values near unity indicate negligible information loss (Nicolae et al., 2011).

In representation learning, EIF (termed "Interpretive Efficiency") is given as

$EIF = \frac{I(Z;Y)}{I(X;Y)}$

where $Z = \varphi(X)$ is a compressed or interpretable channel, and $Y$ is the task label. This ratio encodes the fraction of mutual information retained for the task (Katende, 6 Dec 2025).

2. Information-Theoretic Properties

EIF generalizes classical mutual information ratios by normalizing explanation fidelity and efficiency with respect to achievable baseline performance. Key properties include:

• Boundedness: EIF ∈ [0,1] for all admissible explanations or compressed representations; zero signals absence of certified information, one signifies complete retention.
• Monotonicity: In Blackwell-style comparison, more informative channels yield higher EIF.
• Data-processing stability: EIF cannot increase under admissible post-maps, reflecting the data-processing inequality.
• Admissible invariance: EIF is invariant under invertible, admissible reparameterizations (e.g., affine transformations).

These properties link EIF directly to classical information-theoretic and Fisher-geometric constraints, ensuring theoretical consistency across sample sizes and modeling regimes (Katende, 6 Dec 2025).

3. Estimation and Empirical Computation

In RAG system evaluation, EIF is derived via error rates under the Merlin-Arthur protocol: completeness error ($\epsilon_c$), soundness error ($\epsilon_s$), and precision ($p_{\mathcal D}(M)$). For conditional evaluation (conditioning on model correctness), the effective conditional error

$$\tilde\epsilon_{\mathrm{eff}} = \tilde\epsilon_c + \frac{\tilde\epsilon_s}{1 - \tilde\epsilon_c + \tilde\epsilon_s}$$

yields conditional EIF:

$$EIF_\mathrm{cond} \approx 1 - H_b(\tilde\epsilon_\mathrm{eff})$$

Empirical evaluation on SQuAD 2.0, HotpotQA, and TriviaQA demonstrates typical unconditional EIF ≈ 0.1 and conditioned EIF ≈ 0.3 following M/A training. Standard fine-tuning yields near-zero EIF, reflecting the inability to guarantee context-grounding for predictions (Deiseroth et al., 12 Dec 2025).

In genetic applications, EIF guides experiment design and missing-data imputation. Large-sample EIF under the alternative ($R_I^{(1)}$) can approach unity when observed genotypes and pedigree structures are highly informative. Conversely, rare haplotypes or incomplete marker data lower EIF, identifying key loci for augmentation (Nicolae et al., 2011).

Interpretive Efficiency admits finite-sample and asymptotic estimation guarantees via empirical process theory. Rates under sub-Gaussian or Bernstein conditions are explicit, supporting uniform convergence and robust operational diagnostics (Katende, 6 Dec 2025).

4. Typical Ranges and Practical Interpretation

Empirical EIF values depend on modeling context and explanation strategy:

• RAG systems: Unconditional EIF ≈ 0.1; conditioned EIF ≈ 0.2–0.4. High EIF signifies robust grounding and low hallucination rates in LLM answers (Deiseroth et al., 12 Dec 2025).
• Hypothesis Testing: EIF ≈ 0.9 for linkage analysis and common haplotypes; ~0.7 or lower for rare variants. Values guide genotyping and sample priorities (Nicolae et al., 2011).
• Representation Learning: Identity channels yield EIF = 1; truncated PCA or random projections result in EIF ~0.3–0.4 for high-dimensional digits/classification tasks. Surprisingly, high predictive accuracy can coexist with low EIF, revealing overlooked redundancy or interpretive fragility (Katende, 6 Dec 2025).

EIF thus serves as a diagnostic for explanation faithfulness, experiment efficiency, and representation robustness—always relative to model and channel constraints.

5. Connections to Classical Information Measures

EIF admits deep connections to established information-theoretic and statistical constructs:

• Mutual Information: EIF reduces to mutual information fraction in supervised learning and interpretability analysis.
• Fisher Information: In local asymptotic normality regimes, EIF characterizes the proportion of Fisher curvature retained after compression.
• Likelihood Ratio Statistics: In missing-data inference, EIF generalizes log-likelihood ratios and Chapman–Robbins bounds via relative Kullback–Leibler decompositions.
• Bayesian analogues: For small samples, EIF is generalized via robust Bayesian measures ($BI_1^\pi, BI_2^\pi$), linking variance estimates under prior beliefs to information fraction (Nicolae et al., 2011).

These relationships affirm EIF as a unifying construct bridging explainability, estimation, and efficiency across statistical and machine learning domains.

6. Significance and Applications

EIF establishes a rigorous, operational certificate of explanation fidelity and context dependence. In RAG, it quantifies the reduction in hallucinations and the grounding of predictions in evidence, supporting autonomous interactive-proof-style supervision without manual unanswerable annotation (Deiseroth et al., 12 Dec 2025). In genetic studies, EIF supports experiment optimization and resource allocation. In model compression and interpretability, it diagnoses the loss of task-relevant information, predicting robustness and redundancy effects not captured by accuracy alone (Katende, 6 Dec 2025).

A plausible implication is that EIF’s normalization to model or channel capacity renders it robust to noisy benchmarks and imperfect baselines, providing a consistent, interpretable signal for explanation and representation quality in both theoretical and applied research.