Expandable Decision-Making States (EDMS)

• EDMS is a modular semantic state representation that replaces raw kinematic data with tactical descriptors like space control and offside context.
• The framework divides state representation into Absolute and Relative (on-ball and off-ball) components, enabling expandable, tailored tactical features.
• EDMS employs domain-driven action masking within multi-agent reinforcement learning, achieving robust performance across diverse soccer datasets.

Expandable Decision-Making States (EDMS) denotes, in its explicit arXiv usage, a modular semantic state representation introduced for player-level multi-agent deep reinforcement learning in soccer tactical analysis. In that formulation, EDMS replaces a purely kinematic description based on raw Cartesian positions and velocities with semantically organized descriptors such as space control, defensive pressure, pass-lane accessibility, scoring opportunity, offside context, and transition status, and pairs those descriptors with action masking so that on-ball and off-ball agents do not share the same feasible action set (Ide et al., 1 Oct 2025). In a broader interpretive usage, several adjacent literatures have been mapped onto EDMS-like concerns—state construction, state refinement, uncertainty states, goal-state abstraction, and latent interaction modes—but these works generally do not themselves use the term “Expandable Decision-Making States.”

1. Named EDMS: definition, motivation, and scope

The named EDMS framework was proposed for soccer tactical analysis under offline independent multi-agent reinforcement learning. Its stated problem is that prior player-level deep reinforcement learning models, especially the baseline inherited from Nakahara et al. and OpenSTARLab, used PVS: Position and Velocity States and allowed every player to choose from the same action set, even when some actions were impossible for that player. The paper isolates two consequences: poor interpretability, because learned values are attached to low-level physical inputs rather than soccer concepts, and poor tactical realism or unstable learning, because off-ball players can be assigned on-ball actions such as shot or pass (Ide et al., 1 Oct 2025).

EDMS addresses that problem by reorganizing state description around “time and space,” which the authors identify as fundamental to soccer decision-making. The representation is divided into Absolute State and Relative State. Relative State is further divided into On-ball State and Off-ball State, and On-ball State is split into Intra-Possession State and Inter-Possession State. This organization is central to the paper’s definition of “expandable”: EDMS is built from modular, per-player decision-making descriptors rather than a single vector of coordinates, so users can add tactical variables, adapt to more players or different data providers, preserve a consistent per-agent schema, and keep downstream reinforcement-learning architectures unchanged (Ide et al., 1 Oct 2025).

That expandability is also framed as a response to heterogeneous data sources. The paper explicitly notes variation across leagues, providers, frame rates, event schemas, and visibility conditions, and presents EDMS as more provider-agnostic than raw provider-specific coordinate or event encodings. The intended benefit is not only statistical robustness but also analyst-facing interpretability: a model that attributes Q-values to open space, weak marking pressure, and strong passing lanes is more actionable than one that attributes them to coordinate patterns alone (Ide et al., 1 Oct 2025).

2. State composition and semantic variables

The soccer process is modeled as an MDP over possessions, with one episode defined as a single possession sequence from possession start to end. Within that structure, EDMS enumerates state variables by category rather than by a single compact indexed tuple (Ide et al., 1 Oct 2025).

The Absolute State contains two variables: distance from the ball to each team’s offside line and formation. The offside line is defined as the halfway line if all attacking players are in their own half; otherwise it is the line of the second-to-last opponent relative to the defending team and the ball. Formation is taken directly from the data and does not account for attack or defense phase shifts or substitution-induced changes (Ide et al., 1 Oct 2025).

The Off-ball State is calculated for each of the 10 off-ball players on the attacking team and consists of six variables: $dist\_ball$, $time\_to\_reach\_player$, $time\_to\_reach\_passline$, $space\_score$, change in $space\_score$ when moving 1 meter in 8 directions, and $pass\_score$. The paper provides the explicit construction

$$f_{pass\_score} = k_1 dist\_ball + k_2 space\_score + k_3 time\_to\_reach\_player + k_4 time\_to\_reach\_passline,$$

with $k_1 = 0.5$, $k_2 = 0.3$, $k_3 = 0.2$, and $time\_to\_reach\_player$0, chosen from a preliminary pass-success prediction study (Ide et al., 1 Oct 2025).

The On-ball State — Intra-Possession contains six variables: time for the nearest opponent to reach the ball, distance to the opponent’s goal, angle to the opponent’s goal, $time\_to\_reach\_player$1, $time\_to\_reach\_player$2, and $time\_to\_reach\_player$3. The paper specifies that $time\_to\_reach\_player$4 is calculated only within 30 meters of the opponent’s goal by considering defending players excluding goalkeeper within the triangle formed by the on-ball player and both goalposts, modeling each defender’s probability of blocking a shot angle, computing overall block probability from multiple defenders, and numerically integrating over goalmouth angles. The $time\_to\_reach\_player$5 divides the field into right, central, and left zones and assigns score $time\_to\_reach\_player$6 to the zone containing the tallest player (Ide et al., 1 Oct 2025).

The On-ball State — Inter-Possession contains five variables: time for the closest player from each team to reach the ball, distance from the ball to each team’s goal, angle from the ball to each team’s goal, ball speed, and a binary transition variable, where $time\_to\_reach\_player$7 denotes players transitioning from attack to defense and $time\_to\_reach\_player$8 denotes players transitioning from defense to attack (Ide et al., 1 Oct 2025).

The representation’s semantic layer is carried by variables such as space score, time-to-reach features, pass score, dribble score, shot score, long ball score, distance or angle to goal, distance to offside line, transition, formation, ball speed, and the change in space score for 8 directional movements. The paper explicitly links these features to interpretations in terms of marking pressure, passing lanes, ball accessibility, scoring opportunities, counterattack potential, and danger in defensive transition (Ide et al., 1 Oct 2025).

A particularly important feature is space score. Although the paper gives no closed-form formula, it describes its construction as a Voronoi-like region based on player position and velocity, weighted by pitch importance. Pitch importance is modeled as the product of a sigmoid vertical value increasing toward the opponent goal and a Gaussian horizontal centrality value peaking at the center and decreasing toward the sidelines. The resulting importance field maximizes the central area in front of goal, so the feature encodes not only occupancy but valuable occupancy (Ide et al., 1 Oct 2025).

The paper also gives concrete examples of expandability. Additional variables could include individual player passing abilities in Intra-Possession State, pass maps between players in Off-ball State, and match score in Absolute State. The intended point is that semantic feature blocks can be extended without redesigning the learning architecture (Ide et al., 1 Oct 2025).

3. Action feasibility, MDP formulation, and learning architecture

EDMS is paired with a discrete action space and an explicit masking rule. The model uses 16 actions, split into on-ball and off-ball actions. On-ball actions are pass, through pass, shot, cross, dribble, and defensive action, where defensive action aggregates interception, clearance, tackle, and block. Off-ball actions are move in 8 directions and stay. The paper itself notes a slight arithmetic inconsistency in the total count, but its intended structure is unambiguous: on-ball and off-ball players have separate decision sets (Ide et al., 1 Oct 2025).

The action-masking mechanism is domain-driven. For an on-ball player, the Q-values of off-ball actions are set to a very large negative number, e.g. $time\_to\_reach\_player$9; for an off-ball player, the Q-values of on-ball actions are set to $time\_to\_reach\_passline$0. The paper argues that, without masking, impossible cases such as an off-ball player choosing shot or pass enlarge the search space and especially harm SARSA-style temporal-difference learning because Q-values can become high for impossible actions, forcing wasteful corrective updates (Ide et al., 1 Oct 2025).

The underlying learning setup is an independent offline multi-agent RL model over possessions with shared team reward. The output of interest is the Q-value, and the discount factor is fixed at

$time\_to\_reach\_passline$1

so that all plays within a possession are treated equivalently. The learning algorithm is SARSA. Reward is zero at intermediate steps except for terminal or event-defined cases: $time\_to\_reach\_passline$2 for a goal, $time\_to\_reach\_passline$3 for a concession in the subsequent sequence, and otherwise EPV based on ball location; the paper’s extension adds EPV reward at the moment a shot is taken so that shot creation is not assigned zero reward mid-sequence (Ide et al., 1 Oct 2025).

Training minimizes a combined objective: $time\_to\_reach\_passline$4

$time\_to\_reach\_passline$5

$time\_to\_reach\_passline$6

with $time\_to\_reach\_passline$7, $time\_to\_reach\_passline$8, Adam optimizer, and learning rate $time\_to\_reach\_passline$9 (Ide et al., 1 Oct 2025).

The architecture used in the study is an RNN with a fully connected input layer of 64 dimensions and ReLU activation, a single-layer GRU with hidden dimension 64, and an output layer producing Q-values over all defined actions. Per-frame inputs include state variables for players and field, together with a one-hot vector of the action performed by each player (Ide et al., 1 Oct 2025).

4. Interpretability, empirical results, and reproducibility

A central claim of EDMS is that it relocates value interpretation from raw positions and velocities to tactical concepts. The paper’s interpretability argument is feature-based rather than attribution-based: Q-values can be read through time-to-reach features as marking pressure, through pass-line reach time as passing lanes, through $space\_score$0 and time-to-ball as ball accessibility, through $space\_score$1 as space occupation, through $space\_score$2 as scoring opportunity, through directional $space\_score$3 as dribbling potential, and through offside-aware space scoring as contextual legality (Ide et al., 1 Oct 2025).

The primary interpretability tool is Q-value visualization: a field plot with all players and the ball, together with directional Q-values for a selected off-ball player’s 8 movement options. In the case studies, EDMS highlights support movements, penetrative runs, counterattacks, defensive breakthroughs, risky or high-reward actions, and offside awareness. One sequence shows a forward in a Kyoto Sanga play having low Q-values while offside and sharply higher Q-values after becoming onside; the paper attributes this specifically to $space\_score$4 in offside position and positive $space\_score$5 after returning onside. A free-kick case is presented as a limitation, with nearly uniform directional Q-values and no clear best movement (Ide et al., 1 Oct 2025).

The main validation results are as follows.

Model	action loss	TD loss
PVS w/o mask	$space\_score$6	$space\_score$7
PVS w/ mask	$space\_score$8	$space\_score$9
EDMS w/o mask	$space\_score$0	$space\_score$1
EDMS w/ mask	$space\_score$2	$space\_score$3

These results support two points. First, action masking strongly reduces action loss for both PVS and EDMS. Second, EDMS w/ mask is the best overall model, with the lowest action loss and the lowest TD loss. The authors interpret this as a synergy between semantic state representation and realistic action constraints: richer tactical information helps once impossible actions are removed from the search space (Ide et al., 1 Oct 2025).

Cross-dataset evaluation also favors EDMS.

Data	action loss	TD loss
J1 League PVS	$space\_score$4	$space\_score$5
J1 League EDMS	$space\_score$6	$space\_score$7
LaLiga PVS	$space\_score$8	$space\_score$9
LaLiga EDMS	$pass\_score$0	$pass\_score$1
FIFA World Cup PVS	$pass\_score$2	$pass\_score$3
FIFA World Cup EDMS	$pass\_score$4	$pass\_score$5

The paper presents these results as evidence of generalizability across competition types and cross-provider robustness. It also reports that team-average Q-values had no significant correlation with season-level xG or $pass\_score$6, with $pass\_score$7, suggesting that the learned values may be capturing process-level tactical quality rather than simple shot-outcome statistics (Ide et al., 1 Oct 2025).

On reproducibility, the method was integrated into OpenSTARLab RLearn, with preprocessing handled by OpenSTARLab-Preprocessing and support for event plus tracking data in SAR format. The study reports compatibility with proprietary J.League data, commercial LaLiga data from StatsBomb plus SkillCorner, public FIFA World Cup 2022 event+tracking data, and SoccerNet GSR for broadcast-footage visualization. The code repository is given as https://github.com/open-starlab/ (Ide et al., 1 Oct 2025).

5. Broader EDMS-compatible theories and adjacent formulations

Beyond the named soccer method, the technical notes map several earlier and contemporaneous frameworks onto EDMS-like concerns. These mappings are interpretive rather than terminological: the papers below generally do not use the term EDMS themselves.

A probabilistic-variational reading appears in “Self-organization in complex systems as decision making,” which defines an admissible set of generalized states or prospects $pass\_score$8, assigns probabilities $pass\_score$9, and derives a Gibbs-like law

$$f_{pass\_score} = k_1 dist\_ball + k_2 space\_score + k_3 time\_to\_reach\_player + k_4 time\_to\_reach\_passline,$$0

from the principle of minimal information under constraints. The same paper also gives a dynamic selection rule $$f_{pass\_score} = k_1 dist\_ball + k_2 space\_score + k_3 time\_to\_reach\_player + k_4 time\_to\_reach\_passline,$$1 based on the principle of minimal expansion, and a behavioral decomposition $$f_{pass\_score} = k_1 dist\_ball + k_2 space\_score + k_3 time\_to\_reach\_player + k_4 time\_to\_reach\_passline,$$2 into utility factor and attraction factor. This suggests an EDMS interpretation in which decision states are not fixed utility-maximization primitives but probability-weighted macrostates that can be reweighted under changing constraints (Yukalov et al., 2014).

A sequential-preference reading appears in “Utility Theory for Sequential Decision Making.” That paper treats outcomes as finite trajectories, introduces memorylessness as the condition that future preference comparisons depend only on the current state, and proves that memoryless sequential preferences admit an affine recursion

$$f_{pass\_score} = k_1 dist\_ball + k_2 space\_score + k_3 time\_to\_reach\_player + k_4 time\_to\_reach\_passline,$$3

Its notes explicitly interpret state expansion as a way to restore preference-Markov sufficiency when a coarse state representation fails to contain all preference-relevant history. A plausible implication is that EDMS, when formalized beyond soccer analytics, aligns more naturally with state refinement toward memoryless preferences than with a fixed additive-reward MDP ontology (Shakerinava et al., 2022).

A constructive-state reading appears in “Constructive Decision Theory,” where state and outcome spaces are not primitive. Instead, primitive objects are syntactic programs built from tests and actions, and representation theorems recover subjective states from the decision maker’s language and preferences. Canonical state constructions such as

$$f_{pass\_score} = k_1 dist\_ball + k_2 space\_score + k_3 time\_to\_reach\_player + k_4 time\_to\_reach\_passline,$$4

show that state granularity depends on available tests and on how incomplete preferences are completed. This supports a strongly endogenous interpretation of expandability: new distinctions become part of the state space only when the descriptive vocabulary can express them (0906.4316).

A logical-abstraction reading appears in “Goal-Oriented Semantic Communication for Logical Decision Making,” which separates full world-state constituents from goal-oriented states $$f_{pass\_score} = k_1 dist\_ball + k_2 space\_score + k_3 time\_to\_reach\_player + k_4 time\_to\_reach\_passline,$$5 induced by task rules. There, each goal state is a disjunction over constituent world states,

$$f_{pass\_score} = k_1 dist\_ball + k_2 space\_score + k_3 time\_to\_reach\_player + k_4 time\_to\_reach\_passline,$$6

and semantic communication is optimized to preserve information about $$f_{pass\_score} = k_1 dist\_ball + k_2 space\_score + k_3 time\_to\_reach\_player + k_4 time\_to\_reach\_passline,$$7 rather than about raw observations. This is not an EDMS paper in name, but it provides a formal example of decision-relevant state abstraction as a separate layer over world states (Saz et al., 21 Apr 2026).

Two further papers illustrate structural state augmentation. “An evidential Markov decision making model” expands a binary action space by inserting explicit uncertain or hesitant states, so that a decision maker can occupy an unresolved action state before final commitment. In EDMS terms, this is a direct state-space expansion from committed choices to committed-plus-hesitant latent states (He et al., 2017). “Uncovering Interpretable Internal States of Merging Tasks at Highway On-Ramps for Autonomous Driving Decision-Making” learns three discrete, interpretable internal states of merge interaction with an HMM-GMR model, providing a compact, probabilistic, sequential latent state layer between observations and behavior; the paper does not call this EDMS, but it is an empirical prototype of interpretable expandable decision-state modeling in autonomous driving (Wang et al., 2021).

6. Applications, misconceptions, and open problems

A recurring misconception is to treat EDMS as a universally standardized formalism. That is not supported by the literature summarized here. The term Expandable Decision-Making States is explicitly defined in the soccer MARL paper, while many other works are better described as EDMS-compatible, EDMS-relevant, or EDMS-analogous. This distinction matters because several of those adjacent papers explicitly lack a full algorithm for automatic online state generation, splitting, or merging, even when they provide strong foundations for state construction or refinement (Ide et al., 1 Oct 2025).

The application surface is nevertheless broad. In optimization, “An Expandable Machine Learning-Optimization Framework to Sequential Decision-Making” uses recurrent prediction, local attention, and adaptive partial fixing to extend models trained on shorter or smaller instances to longer and larger ones; the paper does not define EDMS formally, but it is a practical analogue of expandable sequential decision-state commitment (Yilmaz et al., 2023). In interactive logic, “Interactive Model Expansion in an Observable Environment” formalizes partial states $$f_{pass\_score} = k_1 dist\_ball + k_2 space\_score + k_3 time\_to\_reach\_player + k_4 time\_to\_reach\_passline,$$8, the precision order $$f_{pass\_score} = k_1 dist\_ball + k_2 space\_score + k_3 time\_to\_reach\_player + k_4 time\_to\_reach\_passline,$$9, and the distinction between contingent solutions and definite solutions, thereby giving a rigorous semantics for partial states that remain safely extendable under environmental observation and decision revision (Carbonnelle et al., 2023). In multimodal autonomous-driving data, “D2E-An Autonomous Decision-making Dataset involving Driver States and Human Evaluation” supplies ego state, surrounding-agent state, environment context, physiology, eye attention, first-person video, and human evaluation scores, making it a plausible substrate for expanded decision-state design that incorporates latent driver-side variables and post hoc evaluative signals (Ke et al., 2024).

The limitations of the named EDMS formulation are explicit. The soccer paper states that EDMS captures only the fundamentals of soccer decision context; it does not define a complete tactical ontology. It is weak in static set-piece situations, its formation variable is simplistic, it uses independent MARL rather than richer cooperative dependence, its interpretability is feature-based rather than causally validated, off-ball motion is discretized into 8 directions plus stay, and SoccerNet broadcast use required imputing off-screen players to $k_1 = 0.5$0 (Ide et al., 1 Oct 2025). The broader EDMS-compatible literature exhibits parallel gaps: preference-theoretic accounts usually assume a fixed state space even when they motivate expansion; constructive and logical approaches provide foundations for endogenous or task-induced states but do not, in general, supply online state-growth operators or multiresolution consistency theorems (Shakerinava et al., 2022, 0906.4316, Saz et al., 21 Apr 2026).

Future directions in the named EDMS line are concrete. The soccer paper proposes more advanced MARL methods such as QMIX / value decomposition, game-theoretic modeling of strategic dependence, domain-specific variables for set pieces, and extensions with player passing ability, pass maps, and match score (Ide et al., 1 Oct 2025). A plausible implication from the broader literature is that future EDMS research will need to combine three ingredients that are only partially unified at present: semantically interpretable state blocks, explicit rules for state refinement or expansion, and decision-theoretic criteria for when additional state distinctions are necessary rather than merely available.