XRL-Bench: Unified XRL Benchmark

• XRL-Bench is a unified benchmark that standardizes the evaluation of explainable reinforcement learning methods and state attribution techniques.
• It integrates modular environments supporting both tabular and image data, alongside state-importance explainers such as TabularSHAP.
• The platform fosters reproducible research and enables practical evaluations in safety-critical applications like online gaming services.

XRL-Bench is a unified, standardized benchmark designed for the evaluation and comparison of Explainable Reinforcement Learning (XRL) techniques, with a particular focus on state-explaining methods. Developed to address the absence of systematic frameworks in XRL evaluation, XRL-Bench provides a modular platform supporting both tabular and image state data, integrating standard RL environments, state-importance explainers, and evaluation protocols. The benchmark also introduces TabularSHAP, a competitive state attribution method, and demonstrates its practical utility in real-world online gaming service scenarios. XRL-Bench is released as open-source to facilitate reproducible research and accelerate development in the XRL community (Xiong et al., 2024).

1. Background: Explainable Reinforcement Learning and State-Explaining Methods

Explainable Reinforcement Learning (XRL), a branch of Explainable AI (XAI), seeks to render the decision mechanisms of RL agents more transparent, particularly in domains where interpretability, rationality, and safety are critical. XRL methods typically elucidate either model structures, behavior policies, or, notably, the state factors influencing agent actions at decision time. State-explaining techniques comprise a substantial subset of XRL, providing localized attribution about which features or elements in the observed state most contributed to the agent’s action selection (Xiong et al., 2024).

2. Motivation for a Unified Benchmark

Prior to XRL-Bench, assessment and comparison of XRL methods lacked standardized environments, evaluation protocols, or consensus on methodological rigor, impeding progress toward robust, reproducible explainability metrics. Most notably, the field suffered from disparate experiment setups and absent baselines for state-importance explanation, limiting the reliability of cross-method comparisons. XRL-Bench addresses these limitations through a modular, open-source framework, thus filling a critical methodological gap for both practitioners and evaluators (Xiong et al., 2024).

3. Architecture and Core Modules of XRL-Bench

XRL-Bench is organized around three primary functional modules:

1. Standard RL Environments: The framework includes a suite of canonical reinforcement learning environments covering both tabular (e.g., gridworlds) and high-dimensional observation spaces (e.g., images), enabling method development and evaluation across a range of problem complexities.
2. State-Explaining Modules: This module implements explainers based on state or feature importance, with interface support for diverse attribution techniques. Notably, XRL-Bench supports explanation generation for both tabular and image-form state observations.
3. Standard Evaluators: Systematic evaluation protocols are provided to ensure comparability across different XRL approaches, using standardized metrics and evaluation scenarios.

A summary of supported input modalities in XRL-Bench is provided below:

Module	Tabular Data Support	Image Data Support
Standard RL Environments	Yes	Yes
State-Explaining Techniques	Yes	Yes
Evaluation Framework	Yes	Yes

4. TabularSHAP: A New State-Explaining Method

TabularSHAP is introduced as a novel and competitive XRL technique within XRL-Bench. The method is tailored for tabular state spaces and computes feature importance scores aligning with well-established SHAP paradigms, adapted for the RL setting. TabularSHAP operationalizes state explainability by quantifying the contribution of each state variable toward the agent’s policy decisions. The method’s practical efficacy is demonstrated in online gaming service scenarios, highlighting its empirical utility and performance within real-world environments (Xiong et al., 2024).

5. Applications and Use Cases

XRL-Bench facilitates rigorous empirical study of XRL methods across standard testbeds and real-world deployments. The framework’s support for both tabular and perceptual (image) state spaces broadens its applicability, enabling evaluation of explainers in simulated tasks as well as data-rich online domains. Deployment of TabularSHAP in online gaming services exemplifies how XRL-Bench can inform and verify agent behavior where transparency and rational action selection are essential (Xiong et al., 2024).

6. Open-Source Availability and Research Facilitation

XRL-Bench is distributed as an open-source platform, lowering the barrier to entry for implementing and testing novel XRL techniques on a standardized set of benchmarks. The open-access nature encourages consistent experimental protocols and reproducibility, fostering community-driven research progress and comparative studies within the XRL domain (Xiong et al., 2024).

7. Impact and Future Directions in XRL Evaluation

The introduction of XRL-Bench establishes foundational infrastructure for the systematic evaluation of state-explaining XRL methods, addressing previously unmet needs in benchmark design and metric standardization. Its modularity and broad input support suggest extensibility to future XRL scenarios and explanation paradigms. Continued adoption of the XRL-Bench framework is poised to accelerate methodological advancements and the integration of explainability into safety-critical RL deployments (Xiong et al., 2024).