Interference Endurance Score (IES) Overview

• IES is a quantitative metric that assesses system resilience by summarizing the worst-case interference events in reinforcement learning and wireless communications.
• In reinforcement learning, IES is computed as the tail-mean of instantaneous expected interference, guiding hyperparameter tuning and mitigating catastrophic forgetting.
• In wireless systems, IES is defined by the maximum tolerable aggregate interference power to maintain acceptable SINR, underpinning robust coexistence studies.

The Interference Endurance Score (IES) is a quantitative metric developed to assess a system’s robustness to interference, appearing independently in the reinforcement learning (RL) and wireless communications literature. In RL, IES gauges the severity of catastrophic interference during the learning process, enabling the principled comparison of architectures and hyperparameters (Liu et al., 2020). In RF coexistence analysis for mobile broadband systems, IES denotes the maximum aggregate interference power a receiver can tolerate before incurring unacceptable quality degradation, with direct application to IEEE 802.20 (MBWA) standards (Abdulla et al., 2013).

1. Formal Definitions of IES in RL and Wireless Communications

In RL, IES is constructed from a sequence of instantaneous expected interference measurements computed after each minibatch update in a Markov Decision Process. The key quantities are:

• Optimality Residual (OR):

$$\mathrm{OR}(\theta) := \sum_{s,a}d(s,a)\bigl[Q^*(s,a) - Q^{\pi_\theta}(s,a)\bigr]$$

with $d(s,a)$ typically being the empirical distribution over state-action pairs, $Q^*$ the optimal $Q$-function, and $Q^{\pi_\theta}$ the $Q$-function for the greedy policy under parameter $\theta$.

• Instantaneous Expected Interference (EI):

$$\mathrm{EI}(\theta_t,B_t) := \mathrm{OR}(\theta_{t+1}) - \mathrm{OR}(\theta_t) = \sum_{s,a}d(s,a)\bigl[Q^{\pi_t}(s,a) - Q^{\pi_{t+1}}(s,a)\bigr]$$

Positive EI indicates detrimental interference; negative EI implies net performance improvement.

• Interference Endurance Score (IES):

$$\mathrm{IES}_\alpha := \mathbb{E}[X \mid X \ge \mathrm{Percentile}_{1-\alpha}(X)]$$

where $X$ is the sequence of EI values over a selected window. This "Expected Tail Interference" (ETI) summarizes the mean of the worst $\alpha$-fraction of interference events.

In IEEE 802.20 wireless systems, the IES is defined as the maximum permissible aggregate interference power that maintains observed output signal-to-interference-plus-noise ratio (SINR) within specified system degradation $d_{max}$:

• Analytical Benchmark:

$I_{agg,max} = N_0 \bigl(10^{d_{max}/10} - 1\bigr)$

where $N_0$ is the system thermal noise power and $d_{max}$ the allowed SINR degradation.

2. Methodologies for Computation

RL Context

• Pipeline:
  • Gather per-update EI values using Monte Carlo estimation of trajectory returns after each update.
  • Utilize a buffer holding the most recent $W$ EI values.
  • Compute the empirical $(1-\alpha)$–quantile, then average all EI values exceeding this threshold for IES, and compute dispersion (IQR) for robustness analysis.
• Efficient Proxy (AEI):
  • Approximate EI with per-minibatch changes in squared TD-errors:

  $$\mathrm{AEI}_t = \mathbb{E}_{(s,a,r,s')\sim \hat d}\left[\delta_t(s,a,r,s')^2 - \delta_{t-1}(s,a,r,s')^2\right]$$

  where $\delta_t$ is the TD-error at step $t$, and $\hat d$ is an empirical sampling distribution.

• Hyperparameters: Tail-fraction $\alpha$ ($\approx 0.1$), buffer size ($\geq 1000$), window length (final 30–50% of training), sampling distribution for states.

Wireless Systems Context

• Parameterization:

  • $$I_{agg,max}\,[\mathrm{dBm}] = -138.6 - 10\log_{10}n(M, L) + 10\log_{10}R_b(B_{CH}, L, E) + 10\log_{10}(10^{d_{max}/10}-1) + 10\log_{10}[kB_{CH}(T_0 + T_0(10^{F_{sys}/10}-1))]$$
  • Variables incorporate channel bandwidth, mobility (e.g., pedestrian vs. high-speed), link direction (uplink/downlink), bit rate, system noise figure, and allowed degradation.
• Computation:
  • For any set operating point (user mobility, link direction, bandwidth, data rate, etc.), compute $I_{agg,max}$ to characterize system endurance against aggregate RF interference.

3. Empirical Behavior and Benchmark Results

Empirical investigation in RL (Liu et al., 2020) reveals:

• High IES (ETI$_{0.1}$) is strongly correlated with poor sample efficiency, unstable reward, and catastrophic forgetting. Kendall’s $\tau$ between ETI$_{0.1}$ and multiple RL performance metrics was observed in the range $-0.3$ to $-0.6$ (statistically significant).
• Target-network update frequency is a primary factor affecting IES: higher delays increase interference in some systems (Cart-pole), but may improve stability and reduce IES in others (Two-Rooms) at the cost of sample efficiency.
• Updates limited to the last layer produce sharply elevated IES (approximately 3$\times$ the interference of internal-layer updates), traceable to catastrophic forgetting in function approximation schemes.
• The AEI proxy achieves Pearson correlation $0.8$–$0.9$ with true instantaneous expected interference, allowing inexpensive tracking.

For IEEE 802.20 (Abdulla et al., 2013):

• In FDD OFDMA mode (pedestrian, $B_{CH}=2.5$ MHz), $I_{agg,max} \approx -111$ dBm; at $B_{CH}=18.75$ MHz, $I_{agg,max} \approx -98$ dBm.
• High-speed mobility (120 km/h) relaxes the endurance marginally (e.g., $I_{agg,max} \approx -110$ dBm at $2.5$ MHz).
• In 625 kHz-MC mode, $I_{agg,max}$ ranges from $-115$ dBm to $-108$ dBm depending on subcarrier count.
• The IES for a given scenario serves as the pass/fail benchmark in coexistence studies; if measured interference exceeds $I_{agg,max}$, system degradation surpasses prescribed tolerances.

4. Practical Implementation and Usage

RL

• Instrumentation: Agents record either the true EI (using MC rollouts) or AEI (TD-errors) per update, retaining a fixed-length buffer over the analysis window.
• Analysis: Compute tail IES and dispersion after training. High IES indicates nontrivial catastrophic interference.
• Hyperparameter Tuning: IES provides actionable feedback for tuning target-net delay, representation learning methods, and replay buffer sizes.

Wireless

• System-Level Evaluation: For each IEEE 802.20 device and deployment mode, $I_{agg,max}$ defines the operational interference threshold.
• Coexistence Planning: Total expected RF interference from external transmitters is computed and directly compared to IES; system operation is guaranteed only if aggregate interference is below this threshold.
• Mobility/Link Profile: Adjustments to service or system-level deployment can be engineered using IES as a benchmark.

5. Significance, Limitations, and Recommendations

IES provides, for the first time in both RL and wireless system analysis, a scalar and operationally meaningful characterization of system tolerance to interference.

• In RL, IES enables the principled diagnosis and ablation of catastrophic interference, supporting the direct comparison of architectures, update protocols, and sampling strategies. Dispersion complements IES by flagging non-stationary or volatile behavior.
• In wireless coexistence, IES consolidates link, channel, and device-specific parameters into a single actionable interference tolerance metric, facilitating robust network deployment.
• Computationally, the exact IES (ETI in RL) is expensive, motivating the use of AEI as a highly correlated, tractable proxy. In communications, the only challenge is parameter specification and system profiling.
• Recommended default settings in RL are $\alpha\approx0.1$ and window length of the last 30–50% of training; buffer size $\geq 1000$ with reservoir sampling.
• Practitioners are advised to routinely monitor both IES and dispersion to manage and mitigate catastrophic forgetting or performance volatility.

6. Comparative Summary of IES Formulations

Domain	IES Definition	Key Parameters
Reinforcement Learning	Tail mean of EI values (ETI$_\alpha$)	$\alpha$, window length $W$, sampling $d(s,a)$
Wireless (IEEE 802.20)	Max tolerable interference ($I_{agg,max}$)	$d_{max}$, $F_{sys}$, $B_{CH}$, $n(M,L)$, $R_b(B_{CH},L,E)$

The unifying concept is a single, operationally defined scalar threshold or index capturing the system’s capacity to withstand deleterious interference events, directly informing system tuning, architecture choice, and coexistence planning (Liu et al., 2020, Abdulla et al., 2013).