Relative Action Space Reduction

Updated 6 July 2026

Relative Action Space Reduction is a technique that redefines a global action set to a smaller, context-dependent subset using norms, state feasibility, or structural grouping.
It improves decision-making in multi-agent scenarios, reinforcement learning, and robotics by ensuring actions remain safe, interpretable, and causally relevant.
Methods such as causal effect estimation, continuous masking, and action grouping demonstrate its practical benefits through faster convergence and enhanced transfer.

Relative action space reduction denotes a family of operations in which decision making is performed over an action set that is smaller than the nominal action space, but smaller in a context-dependent or baseline-dependent sense rather than by unconditional deletion alone. In the cited literature, the reduction may be defined relative to a norm such as a Move de Rigueur, relative to a state-dependent set of relevant or feasible actions, relative to the current control state through incremental commands, or relative to a structural factorization of a large action into groups, codewords, or lower-dimensional embeddings (George et al., 2023, Stolz et al., 2024, Delavari et al., 7 Jul 2025, Majeed et al., 2020). The resulting methods appear in multi-agent spatial interaction, reinforcement learning, robot manipulation, autonomous driving, and action abstraction, but they share a common technical move: replacing globally available actions with a smaller effective action space that is sufficient, valid, safe, interpretable, or causally meaningful in the current context.

1. Scope of the concept

The expression is used for several related, not identical, constructions. In one line of work, it means feasible action-space reduction: one agent is causally responsible for another when its action reduces the other agent’s feasible action set relative to a normative baseline. In another, it means state-dependent restriction: a policy is constrained to a relevant subset $\mathcal{A}^r(s)\subseteq\mathcal{A}$ or to dynamically valid intervals. In another, it means structural compression: actions are grouped, sequentialized, or embedded in a lower-dimensional space. In control-oriented work, it often means relative parameterization: the policy outputs a delta or local correction rather than an absolute command (Guenov et al., 25 Feb 2026, Stolz et al., 2024, Li et al., 2023, Delavari et al., 7 Jul 2025).

Setting	Reduction principle	Representative formulations
Multi-agent spatial interaction	Compare feasible actions under actual behavior to feasible actions under MdR	FeAR, group FeAR (George et al., 2023, George et al., 23 May 2025, Guenov et al., 25 Feb 2026)
RL with constraints or relevance knowledge	Restrict execution to $\mathcal{A}^r(s)$ or to valid intervals	Continuous action masking, interval restrictions (Stolz et al., 2024, Grams, 2023)
Large discrete action spaces	Replace actions by groups or codewords	Grouped MDPs, binarization (Li et al., 2023, Majeed et al., 2020)
Robotics and driving control	Use incremental commands relative to current state	Relative steering, delta action spaces (Delavari et al., 7 Jul 2025, Aljalbout et al., 2023)
Action abstraction	Replace high-dimensional policies by low-dimensional interfaces	Body-affordances, ASRSE3 (Guttenberg et al., 2017, Wang et al., 2020)

This suggests that “relative” modifies different objects across the literature: a norm, a state, a local control frame, or an action representation. The unifying feature is not a single formula, but the replacement of a global action set by a smaller effective one whose meaning depends on context.

2. Feasible action-space reduction and causal responsibility

In multi-agent spatial interaction, relative action space reduction is formalized by Feasible Action-Space Reduction (FeAR). The discrete formulation models a state space $\StateSpace$, a set of agents $\Agents=\{1,\dots,k\}$, individual action spaces $\ActionSpace{i}$, a joint action $\jointAction$, and a Move de Rigueur $\mdr{i}(\aState)$ that serves as a normatively prescribed default action. For $i\neq j$ , the metric compares the number of feasible actions available to agent $j$ under the actual joint action with the number available if agent $i$ had instead followed its MdR. The resulting quantity is clipped to $\mathcal{A}^r(s)$ 0, and positive values indicate assertive influence while negative values indicate courteous influence (George et al., 2023).

This framing makes responsibility explicitly relative to a norm. The same physical movement may be assertive under $\mathcal{A}^r(s)$ 1 and courteous under $\mathcal{A}^r(s)$ 2, because the counterfactual baseline changes. The measure is also asymmetric: $\mathcal{A}^r(s)$ 3 and $\mathcal{A}^r(s)$ 4 need not match, because one agent can reduce another’s options more strongly than the reverse. The original grid-world work therefore treats causal responsibility not as direct collision production alone, but as the degree to which one agent narrows another’s maneuvering room (George et al., 2023).

The continuous extension replaces action counting by feasible-action hypervolume. Agents act over a time window $\mathcal{A}^r(s)$ 5 with constant accelerations $\mathcal{A}^r(s)$ 6, where $\mathcal{A}^r(s)$ 7 and $\mathcal{A}^r(s)$ 8. The action space of an affected agent is partitioned into magnitude and direction intervals, feasibility is determined by collision checking over forward-simulated trajectories, and the feasible volume is the sum of the volumes of those subsets for which all actions are collision-free. FeAR is then a normalized comparison between feasible volume under actual behavior and feasible volume under the MdR counterfactual. Positive values again indicate assertive reduction, negative values courteous expansion, and diagonal terms quantify how much feasible action space remains to the agent itself (George et al., 23 May 2025).

A central limitation of individual FeAR is causal overdetermination. In some interactions, no single agent has $\mathcal{A}^r(s)$ 9, yet a coalition clearly restricts the affected agent. The group extension addresses this by lifting feasible-action reduction from individuals to non-empty groups $\StateSpace$0. It also defines four types of assertive influence: solo influence, mediated influence, coupled influence, and mediated coupled influence. The associated tiering algorithm first removes courteous agents, then performs a minimal-group search over the remaining candidates; at tier $\StateSpace$1, a group $\StateSpace$2 is declared assertive if adding it to the already identified set strictly increases the affected agent’s FeAR. This yields tiers such as $\StateSpace$3, where higher tiers are more directly constraining and lower tiers depend on them (Guenov et al., 25 Feb 2026).

The simulations make the difference between individual and group reduction explicit. In a three-agent case, individual FeAR detects solo influences such as $\StateSpace$4 and $\StateSpace$5, but group FeAR reveals mediated influence $\StateSpace$6 via agent $\StateSpace$7 and coupled influence $\StateSpace$8 even though $\StateSpace$9. In the larger robot-crossing-pedestrians scenario, group FeAR uncovers coupled and mediated influences such as $\Agents=\{1,\dots,k\}$0 affecting the robot and $\Agents=\{1,\dots,k\}$1 affecting pedestrian $\Agents=\{1,\dots,k\}$2. Across Aggressive, Directed, and Random scenarios, group effects are strongest when agents are spatially close and dynamically entangled: as median Manhattan distance increases, the difference between group FeAR and individual FeAR rankings declines, and Kendall’s $\Agents=\{1,\dots,k\}$3 agreement rises (Guenov et al., 25 Feb 2026).

3. State-dependent restriction and masking in reinforcement learning

A second major meaning of relative action space reduction is state-conditioned restriction to relevant actions. In continuous control, the nominal action space $\Agents=\{1,\dots,k\}$4 is often much larger than the set of actions that are useful or safe in a given state. Continuous action masking therefore introduces a relevant action set $\Agents=\{1,\dots,k\}$5 and three exact mappings from an unconstrained policy to a relevant policy: the ray mask, the generator mask, and the distributional mask. The ray mask radially projects actions from the center of $\Agents=\{1,\dots,k\}$6; the generator mask uses the latent generator space of a zonotope with linear map $\Agents=\{1,\dots,k\}$7; the distributional mask truncates the policy density to $\Agents=\{1,\dots,k\}$8. All three ensure that only relevant actions are executed, and PPO experiments on control tasks show higher final rewards and faster convergence than the baseline without masking (Stolz et al., 2024).

The formulation is explicitly geometric. Relevant action sets are represented as convex sets, especially polytopes or zonotopes, and are computed from system dynamics and a control-invariant relevant state set. For Seeker, the mean episode return improves from $\Agents=\{1,\dots,k\}$9 for the baseline to $\ActionSpace{i}$0 with the ray mask, $\ActionSpace{i}$1 with the generator mask, and $\ActionSpace{i}$2 with the distributional mask. In the 2D and 3D quadrotor tasks, the relevant action set is about $\ActionSpace{i}$3 and $\ActionSpace{i}$4 of the global space respectively, and masking again yields faster or more reliable learning (Stolz et al., 2024).

Related work replaces heuristic pruning by causal effect estimation. In CEE, an inverse dynamics model and an N-value network are used to estimate the one-step causal effect

$\ActionSpace{i}$5

Actions below a threshold are masked as redundant, and a second stage performs relative reduction within action clusters: actions are grouped by similarity

$\ActionSpace{i}$6

then retained only if their normalized relative causal effect exceeds a threshold within the cluster. On Maze, MiniGrid, and Atari, CEE outperforms PPO, NPM, NPM-Random, and CEE-WOC, and the method defines redundancy in quantitative causal terms rather than by hand-crafted rules (Liu et al., 24 Jan 2025).

Dynamic obstacle-avoidance work studies a different but related setting in which the valid action set at time $\ActionSpace{i}$7 is a union of disjoint intervals,

$\ActionSpace{i}$8

Two extensions are proposed for arbitrary numbers of intervals: Parameterized Action Masking (PAM) and Multi-Pass Scaled TD3 (MPS-TD3). The main empirical conclusion is sharp: discrete masking of action-values is the only effective method when constraints did not emerge during training. When restrictions are learned, the choice between projection, masking, and the ConstraintNet modification depends on the task at hand. PAM performs poorly because of coarse binning and aggressive masking of partially invalid bins, whereas MPS-TD3 performs strongly overall (Grams, 2023).

4. Structural reduction by grouping, sequentialization, and abstraction

Relative action space reduction can also be achieved by changing the representation of actions rather than masking them online. One approach is action grouping. A surjective grouping map

$\ActionSpace{i}$9

partitions a large discrete action space into a smaller set of groups, and planning is carried out in a grouped MDP $\jointAction$0. Actions are grouped by similarity in transition kernels and rewards through a linear decomposition model with deviation parameters $\jointAction$1 and $\jointAction$2. The central theoretical result is a decomposition of performance loss into approximation error and estimation error, leading to a counter-intuitive tradeoff: finer grouping reduces approximation error but can worsen estimation when samples or computation are limited. The best grouping is therefore not always the finest one (Li et al., 2023).

A second approach is exact sequentialization or binarization. Instead of choosing one action from a large set $\jointAction$3, an action is encoded as a codeword $\jointAction$4, often with $\jointAction$5, and the agent reveals the codeword one symbol at a time. The sequentialized environment reproduces the original environment when the full codeword is completed, and the discount must satisfy

$\jointAction$6

This reduces instantaneous branching factor at the cost of increasing effective horizon. In the non-MDP setting, the construction improves the dependence of ESA bounds on action-space size from exponential in $\jointAction$7 to logarithmic in $\jointAction$8 (Majeed et al., 2020).

A third line learns low-dimensional action interfaces. Body-affordance learning introduces a low-dimensional embedding $\jointAction$9 indexing high-dimensional, time-extended, reactive policies. The user specifies a target sensor space $\mdr{i}(\aState)$0 and a distance metric $\mdr{i}(\aState)$1, and the proposer network is trained so that outcomes from affordance grid points are as separated as possible while remaining smooth enough for interpolation. In the reported experiments, an 8-DOF reacher control problem is reduced to a 2-DOF affordance space, and a 180-dimensional hexapod locomotion policy space is likewise reduced to 2 DOF (Guttenberg et al., 2017).

Spatial manipulation work applies an analogous factorization to $\mdr{i}(\aState)$2 action spaces. ASRSE3 transforms an original MDP with high-dimensional action space into a new MDP with reduced action space and augmented state space by factorizing pose selection as

$\mdr{i}(\aState)$3

Later decisions are made relative to earlier selected partial pose components through cropping and transformation functions, and the method is combined with SDQfD for large action spaces. On real robot evaluation, reported success rates include $\mdr{i}(\aState)$4 on 4H1, $\mdr{i}(\aState)$5 on ImH2, $\mdr{i}(\aState)$6 on H3, $\mdr{i}(\aState)$7 on ImH3, and $\mdr{i}(\aState)$8 on H4 (Wang et al., 2020).

5. Relative control semantics in autonomous driving and robotics

In autonomous driving, relative action space reduction can mean changing the semantics of an action from an absolute command to a local adjustment. The driving study in CARLA compares full action spaces, fixed reduced spaces, dynamic masking, and relative reduction. The relative method defines steering by

$\mdr{i}(\aState)$9

with relative steering options $i\neq j$ 0 and throttle choices $i\neq j$ 1. The action vocabulary is fixed, but its effect is interpreted relative to the current steering state. Negative deltas are masked near the left boundary, positive deltas near the right boundary, and steering is clipped if needed. This preserves action consistency because action “ $i\neq j$ 2” always means “decrease steering by $i\neq j$ 3,” rather than changing meaning across states (Delavari et al., 7 Jul 2025).

The empirical outcome is that Rel-0.5 converged fastest, with a reported step-to-target of $i\neq j$ 4 steps, compared with $i\neq j$ 5 for Fix-21012, $i\neq j$ 6 for Rel-1.0, $i\neq j$ 7 for F-0.5, and $i\neq j$ 8 for Dyn-0.5. On straight routes, Dyn-0.5 and Rel-0.5 both achieved $i\neq j$ 9 success, while on the full route F-0.5 had highest reward and $j$ 0 success, but Rel-0.5 had the lowest lane deviation and the best efficiency score (Delavari et al., 7 Jul 2025).

Robot manipulation work studies a broader design space of absolute and relative control spaces. Thirteen action spaces are evaluated, including joint torques, joint velocities, joint positions, Cartesian velocities, Cartesian positions, and one-step or multi-step delta variants. The delta spaces use

$j$ 1

for OI $j$ 2 and

$j$ 3

for MI $j$ 4. Evaluation is not limited to return: the study defines Episodic Reward, Expected Constraint Violations, Normalized Tracking Error, Task Accuracy, and Offline Trajectory Error. The main pattern is that velocity-based action spaces generally perform best in simulation and transfer better than position-based spaces, JT has the highest OTE, and delta action spaces usually transfer better than their base counterparts (Aljalbout et al., 2023).

The practical implication is not merely that smaller spaces are better, but that action spaces that naturally limit tracking error transfer better. The study explicitly states that transfer can be improved by reducing the magnitude of jumps of the corresponding control targets, and that joint-velocity-based action spaces provide the best overall balance of learning, safety, and transfer (Aljalbout et al., 2023).

6. Trade-offs, misconceptions, and limitations

A recurring misconception is that action-space reduction is uniformly beneficial. The grouping literature proves the opposite: finer grouping decreases approximation error but can increase estimation error under fixed sample and compute budgets, so intermediate group counts can be optimal (Li et al., 2023). Game-oriented action-space shaping reaches a related conclusion empirically. Removing actions often helps, discretizing continuous actions is usually helpful, but converting MultiDiscrete to Discrete is not reliably beneficial, and in ViZDoom increasing the number of available actions improves results in harder scenarios. Atari results likewise show no clear average difference between minimal, full, and multi-discrete action spaces (Kanervisto et al., 2020).

Another misconception is that reduction always means global pruning. In FeAR, reduction is relative to MdR; in masking, it is relative to the current state; in delta control, it is relative to current or previous targets; in sequentialization, no actions are removed at all, because complexity is shifted from branching factor to planning horizon (George et al., 2023, Stolz et al., 2024, Aljalbout et al., 2023, Majeed et al., 2020). This suggests that the central design question is not only “how many actions remain,” but also “relative to what baseline, representation, or local frame is the reduced space defined.”

The methods also have distinct limitations. Continuous action masking requires task knowledge sufficient to compute a tractable relevant set, the distributional mask needs Hit-and-Run MCMC and approximate normalization, and the generator mask requires a zonotope with fixed generator dimensions (Stolz et al., 2024). Continuous FeAR is pairwise and does not handle group responsibility, while group FeAR was introduced precisely to fill the responsibility gaps created by overdetermination (George et al., 23 May 2025, Guenov et al., 25 Feb 2026). Dynamic interval methods are sensitive to whether constraints appear during training, and PAM inherits discretization artifacts from binning (Grams, 2023).

A plausible implication is that relative action space reduction is best understood as a design pattern rather than a single algorithmic family. Across causal responsibility, RL, and robotics, the pattern is to preserve task sufficiency while shrinking the effective decision set by conditioning it on norms, feasibility, relevance, structure, or local control semantics. The cited literature therefore treats action-space design not as a minor implementation detail, but as a primary mechanism for determining causal attribution, exploration efficiency, safety, smoothness, and transfer robustness (Guenov et al., 25 Feb 2026, Stolz et al., 2024, Aljalbout et al., 2023).