Influence-Driven Response Rate (IRR)

• IRR is a quantitative metric that measures the fraction of system output causally attributed to explicit external influences across diverse domains.
• In retrieval-augmented language models, IRR aggregates unique influence scores from retrieved documents to quantify their non-redundant contribution to the generated response.
• In temporal and social engagement networks, IRR employs Hawkes processes and counterfactual simulations to isolate causal effects, enabling robust diagnostics and control.

The Influence-Driven Response Rate (IRR) is a quantitative metric designed to measure how strongly external, context-providing elements—such as retrieved documents, network events, or exogenous policy changes—shape the output or activity of a system. Its conceptual and mathematical formulation varies by domain and modeling assumptions, but the unifying theme is the attribution of observed responses to distinct, influence-carrying sources. IRR features prominently in diverse settings including Retrieval-Augmented Generation (RAG) with LLMs, temporal event networks modeled with Hawkes processes, and social media engagement via sequential causal inference frameworks (Gerami et al., 27 Oct 2025, N. et al., 2023, Tian et al., 25 May 2025).

1. Foundational Definitions and Motivations

The principal motivation for IRR is to isolate and quantify the effect of explicit external information or actions on the behavior of complex, often sequential or networked systems. In RAG-LLMs, IRR captures the fraction of the generated response attributable to specific retrieved contexts; in influence networks, IRR directly encodes the per-event excitation rate from one node to another; in social engagement, IRR expresses the normalized causal effect of exogenous signals on user actions.

Formally, IRR answers: "How much of a system's output can be causally attributed, per unit or per event, to a defined set of external influences, versus endogenous or background effects?" The precise mathematical definition depends on the stochastic process or causal structure in question.

2. IRR in Retrieval-Augmented LLMs

In the context of Retrieval-Augmented Generation, IRR is constructed atop the Influence Score (IS). For a query $Q$ and $k$ retrieved documents $X_1, \ldots, X_k$, and model response $Y$, one computes:

• Semantic Entropy ($H_S$): Cluster $N$ generated responses by semantic meaning, estimate cluster probabilities $p_i$, and calculate $H_S = -\sum_i p_i \log_2 p_i$ as a proxy for classical entropy.
• Mutual Information of document $i$:

$I(X_i; Y) = H_S(Y) - H_S(Y|X_i)$

• Union information of all documents:

$k$0

• Influence Score (IS) for document $k$1:

$k$2

This value reflects the unique, non-redundant information document $k$3 contributes to the response when all documents are present. A large positive $k$4 signals irreplaceable influence.

The IRR is obtained by aggregating the document-level IS values into a single normalization. Common variants include:

• $k$5
• $k$6, where $k$7 quantifies total information gain from retrieval ($k$8)
• $k$9, a document "coverage" metric

A high IRR indicates heavy reliance on retrieved evidence, while a low IRR suggests that the model output is dominated by internal pretraining (Gerami et al., 27 Oct 2025).

3. IRR in Temporal Influence Networks (Hawkes Processes)

In multidimensional Hawkes processes applied to event-driven networks, the IRR is formalized as the influence-driven response kernel $X_1, \ldots, X_k$0, which quantifies how an event at node $X_1, \ldots, X_k$1 at time $X_1, \ldots, X_k$2 influences the event rate at node $X_1, \ldots, X_k$3 at time $X_1, \ldots, X_k$4.

The discrete-time model is:

$X_1, \ldots, X_k$5

• $X_1, \ldots, X_k$6 is the baseline intensity for node $X_1, \ldots, X_k$7.
• $X_1, \ldots, X_k$8 is the rate of exponential decay (memory-loss).
• $X_1, \ldots, X_k$9 are the influence weights, with $Y$0 meaning no immediate excitation from $Y$1 to $Y$2.
• $Y$3 is the influence-driven response rate kernel.

Stacking all $Y$4 in the $Y$5 matrix $Y$6, the IRR matrix encodes the short-term causal boost in activity between every pair of nodes (N. et al., 2023).

4. Causal IRR in Counterfactual Social Engagement

Causal IRR, as defined in counterfactual treatment-outcome modeling for social media engagement, measures the engagement attributed per unit increase in external treatment.

• Treatment (Signal) variable $Y$7: Binary or continuous feature representing exposure, e.g., a spike in Google Trends.
• Outcome $Y$8: Multidimensional engagement metric (likes, shares, comments, emojis).
• Causal effect (ATE):

$Y$9

For example, comparing full vs. partial exposure policies ($H_S$0):

$H_S$1

• Influence-Driven Response Rate (IRR):

$H_S$2

where $H_S$3 is the total change in treatment intensity ($H_S$4) under a counterfactual scenario.

IRR here quantifies the marginal causal effect—engagement attributable per additional unit of external influence, robust to confounders and time lags (Tian et al., 25 May 2025).

5. Algorithmic Computation and Practical Pseudocode

Each domain develops bespoke estimation frameworks:

• RAG-LLMs: Repeated LLM sampling under different document subsets, entropy clustering for $H_S$5, and IS computation via $H_S$6 LLM calls per input. IRR is calculated by normalizing aggregated IS values.
• Hawkes Networks: The Ensemble Poisson–Gamma Filter (EnPGF) sequentially propagates and updates each node’s rate and parameters over time using only count data. IRR weights and their uncertainty are output as posterior distributions over the influence matrix $H_S$7.
• Causal Engagement Models: Sequential deep learners (RNN/Transformer/Mamba) are trained to jointly model treatment and outcome. Counterfactual timelines are generated by altering signal timing and strength, ATE is evaluated by difference in predicted engagement, and IRR is computed as ratio to total change in signal exposure.

Pseudo-algorithmic workflows from each methodology are as follows:

Setting	Key Steps
RAG-LLMs	Compute $H_S$8 on all doc subsets, derive $H_S$9, normalize to get IRR
Hawkes Networks	Update particle ensemble, regress $N$0, extract IRR matrix
Causal Engagement	Train joint model, simulate counterfactuals, compute ATE and IRR

6. Empirical Results and Interpretation

Studies have validated IRR metrics across settings:

• Poison attack detection (RAG): The IS metric flagged the malicious document as most influential in 86% of failures; top-2 IS captured 95%. Ablation studies showed that the top-IS documents recreate original responses with high fidelity, confirming the value of IRR for reliability auditing (Gerami et al., 27 Oct 2025).
• Influence network inference (Hawkes): The method inferred evolving networks of up to 10,000 nodes, with uncertainty quantification via ensemble spread (N. et al., 2023).
• Causal social media influence: IRR derived from ATE distinguished causal impact from correlated exposure, with case studies showing alignment with expert-judged user influence (Spearman $N$1) (Tian et al., 25 May 2025).

A high IRR consistently indicates that external context or events are essential to the observed responses, providing an interpretable, actionable measure for diagnosis, control, or intervention.

7. Limitations and Deployment Considerations

• Computational cost: RAG-IRR requires $N$2 LLM calls per input; Hawkes EnPGF scales with $N$3 but remains feasible with sufficient parallelization.
• Metric approximation: Semantic entropy proxies may over/underestimate true information content; IS in RAG-LLMs collapses non-unique contributions, lacking explicit redundancy/synergy decomposition.
• Sensitivity to hyperparameters: The accuracy and interpretability of IRR depend on the choice of sampling parameters ($N$4), clustering methods, and counterfactual scenario design.
• Intrinsic knowledge confounding: If the model’s baseline knowledge is already confident, IRR may underestimate retrieval or treatment influence.

Despite these challenges, IRR metrics—by isolating true causal, document-level, or event-driven contributions—enable greater transparency, robustness, and interpretability in both machine learning and networked systems (Gerami et al., 27 Oct 2025, N. et al., 2023, Tian et al., 25 May 2025).