- The paper models each deterministic greedy goal-conditioned policy as a functional graph, using attractors and basins to explain per-goal success and failure beyond aggregate metrics.
- A fixed local goal support threshold of 0.5 predicts low-success goals with F1 above 0.88 across five settings, while zero support provides an exact certificate that the goal cannot be reached.
- The paper’s taxonomy identifies competitor-dominated and fragmented basins as major residual failure modes, offering actionable post-hoc targets for improving local entry, suppressing attractors, and reshaping curricula.
Motivation: aggregate metrics conceal structural failure
Sparse goal-conditioned reinforcement learning (GCRL) is conventionally evaluated through aggregate quantities such as mean success rate, yet these statistics obscure why a policy succeeds for some goals and fails for others. This paper, "Functional Graphs for Predicting and Explaining Goal Failure in Sparse Goal-Conditioned RL" (2605.09335), addresses that gap by analyzing the deterministic greedy policy of a trained goal-conditioned value function as a finite discrete dynamical system. The motivating observation is concrete: in an open 8×8 GridWorld trained with TD learning under uniform goal sampling, mean evaluation success across 20 seeds is $0.44$, but seed-level success ranges from $0.26$ to $0.67$ despite identical environment, architecture, horizon, update rule, and sampling distribution. An edge-biased goal curriculum, which triples the sampling rate of boundary goals, fails to reduce this variability and leaves mean success essentially unchanged ($0.46$). The paper's central question is therefore structural: for each goal, what closed-loop dynamics has the greedy policy induced, and how do those dynamics determine goal-level success or failure?
The functional-graph framework
For a fixed goal g, the learned value function Vθ​(s,g) and the deterministic environment jointly define a successor map fg​(s)=T(s,πg​(s)) with exactly one outgoing edge per state. This map is a functional graph (mapping digraph), and the paper invokes the standard finite-map decomposition: every trajectory eventually enters a directed cycle, each weakly connected component contains exactly one cycle, and all remaining states form in-trees feeding into it. Cycles are attractors; predecessor sets are basins of attraction. Treating the goal as absorbing (fg​(g)=g), success corresponds to membership in the goal basin, while failure manifests as spurious fixed points, non-goal cycles, dominant competing basins, or fragmentation across multiple attractors.
The framework distinguishes eventual goal basins Bg​({g}) from the finite-horizon success set $0.44$0, noting that evaluation uses the training horizon $0.44$1, so states reaching the goal only after more than $0.44$2 steps are counted as failures even though they lie in the eventual goal basin. From the graph the paper extracts a compact set of metrics: finite-horizon success, local goal support, goal-basin fraction, largest competing-basin fraction, and a fragmentation index $0.44$3 over basin mass.
Local goal support: a provable necessary condition
The core local statistic is local goal support (LGS): the fraction of valid one-step neighbors of $0.44$4 whose greedy successor is $0.44$5. In a four-neighbor deterministic GridWorld, any trajectory reaching $0.44$6 from a non-goal start must pass through a valid neighbor, so zero LGS exactly precludes goal entry: $0.44$7 implies $0.44$8 for every horizon. This yields a hard, parameter-free failure certificate.
The paper is careful about the converse, proving an asymmetry (Proposition A.2): positive LGS guarantees only that at least one start reaches the goal in one step, and even full local support does not guarantee high success, since distant states may be captured by competing attractors. This is confirmed empirically: the appendix enumerates interior goals with $0.44$9 but $0.26$0 in every condition—for instance, 119 such cases in the $0.26$1 TD setting with mean success $0.26$2 and fragmentation $0.26$3.
Empirical strength of the LGS diagnostic
Despite its simplicity, LGS is a strong failure predictor. Using the fixed rule $0.26$4 to predict low-success goals ($0.26$5), with no fitted parameters, the main $0.26$6 TD setting achieves precision $0.26$7, recall $0.26$8, and F1 $0.26$9; F1 exceeds $0.67$0 in all five settings (TD, MC, edge-biased, $0.67$1, bottleneck). The zero-support rule ($0.67$2) attains perfect precision everywhere, matching the theory, but low recall because many failures retain weak nonzero support.
Per-seed robustness analysis shows this is not a pooled artifact: mean within-seed AUC exceeds $0.67$3 in every condition (minimum $0.67$4), with mean Spearman correlation between LGS and success above $0.67$5. LGS also stratifies global structure monotonically: low-support goals exhibit success between $0.67$6 and $0.67$7 with competing-basin fractions of $0.67$8–$0.67$9, whereas full-support goals reach success of $0.46$0–$0.46$1 with competitors reduced to $0.46$2–$0.46$3.
A post-hoc taxonomy of residual failure modes
Because local support is not sufficient for success, the paper introduces a rule-based taxonomy assigning each seed–goal functional graph to one of four regimes using goal-basin fraction, largest competing basin, and fragmentation. Across 7,780 seed–goal graphs, the regimes separate sharply:
| Regime |
Share |
Mean success |
Mean LGS |
Comp. basin |
Frag. |
| Goal-dominant |
17.4% |
0.944 |
0.978 |
0.037 |
0.104 |
| Partial/contested |
4.0% |
0.554 |
0.767 |
0.392 |
0.097 |
| Competitor-dominated |
54.1% |
0.073 |
0.334 |
0.733 |
0.195 |
| Fragmented |
24.5% |
0.244 |
0.537 |
0.326 |
0.565 |
The regime distribution varies systematically by condition: the $0.46$4 setting is 64.9% competitor-dominated, suggesting that larger state spaces make global basin organization harder; the bottleneck environment is 49.3% fragmented, consistent with constrained geometry splitting the state space into separated routing regions; and MC learning yields far fewer goal-dominant graphs (5.7%) than TD (34.1%), with much higher fragmentation. Notably, edge-biased sampling shifts where structure forms—edge-goal success rises from $0.46$5 to $0.46$6 while interior success falls from $0.46$7 to $0.46$8—without eliminating competitor-dominated failures. The taxonomy is explicitly descriptive rather than a fitted classifier.
Limitations and open questions
The paper's claims are bounded by its experimental scope. All results assume deterministic greedy evaluation, finite discrete state spaces, four-neighbor movement, and exhaustive enumeration of the successor map; the environments are small GridWorlds ($0.46$9 to g0) with a small MLP UVFA trained for only 500 episodes per seed. The authors state plainly that extension to stochastic policies, continuous states, richer action spaces, and high-dimensional observations will require approximate graph construction or local dynamical diagnostics, and that the framework has not been tested beyond the bottleneck variant. Two specific questions remain open: whether the same structural failure modes (competitor dominance, fragmentation) persist in multi-room and MiniGrid-style geometries scaled in a controlled sequence, and whether the diagnostic taxonomy can be converted into targeted interventions—improving local entry, weakening competing attractors, or reshaping curricula to reduce fragmentation—that causally improve success rather than merely explaining it.
Conclusion
This paper contributes a structural evaluation framework, not a training algorithm. By decomposing the greedy goal-conditioned policy into goal-indexed functional graphs, it establishes that zero local goal support is an exact obstruction to goal entry, that a fixed threshold rule on LGS predicts goal-level failure with F1 above g1 across five settings without any fitted parameters, and that residual failures among locally supported goals are explained by competitor-dominated and fragmented basin structure. The result reframes sparse GCRL failures as structured, exactly computable policy-induced dynamics, and provides a cheap post-training diagnostic that complements aggregate success rates with a mechanistic account of per-goal failure.