Temporally-Aware Spurious Attractor Strength

• The paper introduces TASAS as a novel metric that differentiates between transient attractor regions and persistent terminal traps in reinforcement learning systems.
• It employs a persistence-weighted sum using finite-horizon rollouts, integrating escape probabilities to normalize spurious attractor strength against goal density.
• Empirical results across various benchmarks demonstrate TASAS's effectiveness in distinguishing genuine policy failure modes from misleading transient states.

Temporally-Aware Spurious Attractor Strength (TASAS) is a quantitative robustness metric introduced for formal verification of reinforcement learning (RL) policies within a dynamical systems framework. TASAS is specifically designed to distinguish between transient attractor-like regions—through which trajectories may simply pass en route to a goal—and true terminal “trap” states that persistently capture and retain system trajectories. By incorporating a temporal component absent from purely geometric metrics, TASAS resolves ambiguity in the failure analysis of RL agents and provides a direct measure of the persistent risk posed by spurious attractors away from the intended goal region (Nasir et al., 21 Aug 2025).

1. Context and Motivation

The metric arises in the context of viewing the closed-loop system formed by the RL agent and its environment as a discrete-time autonomous dynamical system. Within this framework, Lagrangian Coherent Structures (LCS), identified via Finite-Time Lyapunov Exponents (FTLE), are interpreted as the hidden “skeleton” governing long-term trajectory behavior. Repelling LCS correspond to safety barriers that prevent entry into unsafe regions, while attracting LCS identify convergence properties and highlight the presence of attractor basins—including undesirable ones, termed spurious attractors. Existing non-temporal metrics such as Aggregated Spurious Attractor Strength (ASAS) measure the geometric strength of spurious attractors but do not determine whether these are persistent traps or merely transient—potentially leading to false alarms in policy analysis. TASAS addresses this deficit by incorporating escape probabilities under finite-horizon rollout, thereby measuring failure risk in a dynamically and temporally meaningful way (Nasir et al., 21 Aug 2025).

2. Formal Definition and Computation

TASAS is formulated as a persistence-weighted version of ASAS. The process involves three primary steps:

1. Identification of Significant Spurious Peaks: Let $h(s)$ denote the final-state density map from extensive trajectory simulation, $G \subset S$ the goal region, $h_{\text{goal}} = \max_{s \in G} h(s)$ the maximum density in the goal, and $\alpha \in (0,1]$ a significance threshold. The set of significant spurious peaks is

$$\mathcal{P}_{\text{sig}} = \{\, s \in \mathcal{P}_{\text{spurious}} \mid h(s) \ge \alpha \cdot h_{\text{goal}} \,\},$$

where $\mathcal{P}_{\text{spurious}}$ are local maxima of $h(s)$ outside the goal region.

1. Persistence Estimation via Escape Ratio: For each peak $p \in \mathcal{P}_{\text{sig}}$, $N$ rollout simulations are performed with starting state $p$, each allowed to run up to a finite horizon $G \subset S$0. The fraction $G \subset S$1 that reach the goal defines the escape probability; persistence is $G \subset S$2. High persistence indicates a genuine terminal trap.
2. Persistence-Weighted Spurious Strength: The persistence-weighted sum,

$G \subset S$3

is then normalized by $G \subset S$4:

$G \subset S$5

Algorithmically, the complexity is $G \subset S$6 due to the need for repeated finite-length rollouts from each detected spurious attractor (Nasir et al., 21 Aug 2025).

3. Conceptual Interpretation and Distinction from ASAS

A high ASAS can result from both terminal traps and regions that merely direct trajectories transiently (e.g., highways). This conflation may mislead safety verification. TASAS resolves this by filtering for persistence:

• $G \subset S$7: Measures how strongly trajectories accumulate at peak $G \subset S$8 in the simulated density.
• $G \subset S$9: The likelihood a trajectory starting at $h_{\text{goal}} = \max_{s \in G} h(s)$0 escapes to the goal.
• $h_{\text{goal}} = \max_{s \in G} h(s)$1: The persistence or “stickiness” of $h_{\text{goal}} = \max_{s \in G} h(s)$2—critical for distinguishing traps.
• $h_{\text{goal}} = \max_{s \in G} h(s)$3: Quantifies how much of the attractor’s strength is associated with actual, persistent risk.
• Normalization by $h_{\text{goal}} = \max_{s \in G} h(s)$4: Ensures interpretability relative to the primary goal attractor.

TASAS thus quantifies the relative strength of persistent, non-goal attractors—emphasizing convergence correctness over surface reward metrics. Unlike ASAS, which is purely geometric and static, TASAS is temporally aware and more robust to misleading transient features (Nasir et al., 21 Aug 2025).

4. Usage and Significance in Policy Robustness

TASAS operates as the principal metric for diagnosing hidden convergence failure modes in RL policies within the dynamical-systems safety verification framework. It complements the Mean Boundary Repulsion (MBR) metric that measures obstacle avoidance margins. TASAS is specifically intended to:

• Detect local minima, trap states, and cyclic/looping behaviors in the closed-loop system.
• Flag regions acting as undesired persistent basins of attraction, even when episodic reward appears satisfactory.
• Discriminate between transient pathways and true terminal traps, answering, "How much of the spurious attraction observed in the density field is genuinely persistent and hazardous?"

A robust policy, by this criterion, should present a single dominant attractor (the goal) and negligible TASAS. Moderate or high TASAS values indicate the presence of potentially catastrophic terminal traps (Nasir et al., 21 Aug 2025).

5. Empirical Results Across Benchmarks

The metric is empirically validated on both discrete and continuous control environments.

Environment	ASAS (final)	TASAS (final)	Robustness Interpretation
Simple Wall (discrete)	—	0.0544	Trap-free after learning
Scattered Blocks (discrete)	—	0.0550	Minor persistent trap persists
U-Shape Trap (discrete)	1.2258	0.0727	Transient highways, not traps
MountainCarContinuous-v0	0.0000	0.0000	Ideally robust
Pendulum-v1	0.0000	0.0000	Ideally robust
LunarLanderContinuous-v2	3.9038	3.9038	Serious persistent traps present

Notably, in the U-Shape Trap, ASAS remains high even as TASAS falls to near zero—demonstrating that the dominant spurious attractors are transient highways, not true traps. In LunarLanderContinuous-v2, ASAS and TASAS coincide at 3.9038, indicating that spurious attractors are both strong and persistent, thus the policy is classified as non-robust. In MountainCarContinuous-v0 and Pendulum-v1, both metrics are identically zero, indicating monotonic convergence to the goal with no secondary attractors (Nasir et al., 21 Aug 2025).

6. Practical Thresholds, Limitations, and Assumptions

Interpretive thresholds highlighted in the paper:

• ASAS $h_{\text{goal}} = \max_{s \in G} h(s)$5: indicates ideal robustness;
• ASAS $h_{\text{goal}} = \max_{s \in G} h(s)$6: critical lack of robustness;
• TASAS $h_{\text{goal}} = \max_{s \in G} h(s)$7: absence of persistent traps;
• Nonzero TASAS: presence of at least one genuine terminal trap region.

Limitations and operational assumptions of TASAS include:

1. Deterministic policy evaluation at test time ($h_{\text{goal}} = \max_{s \in G} h(s)$8).
2. Use of grid or sampled state representations; in high-dimensional systems, 2D slices are analyzed.
3. Persistence estimated over finite simulation horizon $h_{\text{goal}} = \max_{s \in G} h(s)$9—thus, empirical rather than analytic.
4. Sensitivity to the $\alpha \in (0,1]$0 threshold controlling which attractors are included.
5. Empirical escape rates rely on finite $\alpha \in (0,1]$1 rollouts and are subject to variance.

These imply that TASAS, while informative, is dependent on simulation parameters and does not provide an analytic guarantee across the entire basin structure—A plausible implication is that undetectable traps could remain in regions missed by sampling or on unexamined state dimensions (Nasir et al., 21 Aug 2025).

7. Role within the Dynamical Systems RL Safety Verification Framework

TASAS forms the keystone quantitative metric in the multi-layered verification framework described in the paper. The workflow is structured as:

1. Field visualization (LCS/FTLE, attractor plots) to provide phase space geometry.
2. Quantitative metrics: MBR (for safety margin), ASAS (for geometric spurious strength), TASAS (for persistent trap risk).
3. Formal stability certificates, leveraging local FTLE bounds for analytic robustness quantification.

Within this workflow, TASAS diagnoses convergence failures that would escape detection by reward-based, episodic, or solely geometric analyses. It enables discrimination between superficially plausible but actually hazardous policies and those that are robust in the global dynamical-systems sense (Nasir et al., 21 Aug 2025).