Contract-Based Compositional Shielding for Safe Multi-Agent Reinforcement Learning

Abstract: Safe coordination problems surface in multi-agent reinforcement learning when global safety cannot be enforced by any agent unilaterally: the admissibility of one agent's action may depend on the dynamics of other agents. Decentralised shields can enforce safety at runtime, but purely factorised permissions often exclude optimal team behaviour that is safe only through coordination. We study deterministic safety guarantees for agents trained and deployed under decentralised execution, recovering team-optimal safe behaviour without centralised runtime control. Agents have a shared global specification $φ$ in the safety fragment of Linear Temporal Logic ($\mathsf{LTL}{\mathsf{safe}}$ ), and select among tuples of local $\mathsf{LTL}{\mathsf{safe}}$ obligations whose conjunction implies the global specification $φ$. Each agent may rely on the other agents' local obligations as assumptions because the whole contract tuple is certified simultaneously and allows projection into local action masks. At learning time, a non-stationary multi-armed bandit chooses among a library of local $\mathsf{LTL}_{\mathsf{safe}}$ obligations to select the tuple that optimises team reward, all without forgoing end-to-end safety. We evaluate the approach across 6 environments and 15 algorithmic variants.

• The paper introduces a novel contract-based framework that decomposes global safety objectives into local obligations to ensure deterministic safety in multi-agent reinforcement learning.
• It employs circular assume-guarantee reasoning and fixed-point computation to certify local action masks, enabling coordinated optimal behaviors under strict safety constraints.
• Empirical results on six benchmarks demonstrate that the approach recovers high-return coordinated actions, outperforming conventional decentralized and centralized shielding methods.

Contract-Based Compositional Shielding for Safe Multi-Agent Reinforcement Learning

Motivation and Problem Formulation

The paper addresses the challenge of enforcing safety in cooperative Multi-Agent Reinforcement Learning (MARL) where admissibility of actions depends on coordinated behavior. Conventional decentralized shielding approaches, based on Cartesian decomposition of a central safety shield, tend to be overly conservative: they reject optimal behaviors that are only safe through coordination, since unilateral action filtering cannot encode assumptions about the policies of other agents. The core objective here is to achieve deterministic safety for decentralized agents without relying on central runtime control, while still enabling the team to realize return-optimal, globally safe coordinated behaviors.

Methodology: Contract-Based Decentralized Shielding

The proposed framework employs compositional assume-guarantee contracts, decomposing a shared global safety objective $\phi$ (expressed in the safety fragment of Linear Temporal Logic, LTL) into tuples of local agent obligations. Each tuple, or "contract", is jointly certified such that the conjunction of local obligations ensures the global specification is met. The key technical elements are:

1. Local Obligations and Contracts: Each agent is assigned a local $\varphi_i$ over its dedicated local alphabet. The contract $$\mathcal{C} = \langle \varphi_1, \dots, \varphi_n \rangle$$ is valid when $\bigwedge_{i=1}^n \varphi_i \implies \phi$ universally over global execution traces.
2. Assume-Guarantee Reasoning: Certification leverages a circular fixed-point approach, unlike approaches that necessitate an acyclic dependency graph for obligations (e.g., (Brorholt et al., 2024)). The circular reasoning engine simultaneously certifies all agent obligations and local action masks, allowing each agent to rely on commitments from teammates as encoded in the current contract.
3. Automata Compilation: Each local obligation is compiled into a deterministic safety automaton; collective action spaces are modeled as contract products, and a fixed-point computation determines the "winning region" from which agents can select nonempty Cartesian products of local admissible actions ensuring no symbolically unsafe state is ever reached. This construction yields deterministic action masks (shields) per agent.
4. Learning-Time Selection: A non-stationary multi-armed bandit selector chooses, at learning time, among a finite library of certified contracts. Rewards guide selection, with correctness ensured via prior certification; no new safety certificates are needed online.
5. Profile Search and Permissiveness: The contract library is constructed by bounded enumeration over local formulae with a specified maximum depth. Return-optimal behaviors are recovered, contingent on the expressivity of the contract library.

Theoretical Results

The approach admits several strong theoretical properties:

• Circular Compositional Soundness: Under a certified contract $\mathcal{C}$, any execution that never deviates from the projected local masks is guaranteed to satisfy the global objective $\phi$.
• Optimal Safe Policy Recoverability: If a deterministic team-optimal safe policy can be represented by some contract in the library, the approach can realize maximal attainable safe team return.
• Certified Contract Selection: Switching contracts at episode boundaries—always among pre-certified tuples—guarantees deterministic safety, regardless of how the outer loop selector explores or exploits contracts.

Empirical Evaluation

The methodology is empirically validated on six multi-agent benchmarks: Flatland, Connector, Level-Based Foraging, RWARE, Pressure Plate, and Car Platoon. The environments are characterized by safety objectives that preclude direct realization via ordinary factorized shields (e.g., coordinated access to regions, mutual trail/queue management, and collaborative physical interaction tasks).

Figure 1: Renders of the six benchmark environments used in the evaluation.

Quantitative results (see Figure 1 and corresponding analysis) indicate that contract-based compositional shields consistently match or outperform both conventional shielded baselines and centralized approaches in terms of final team reward under strict safety constraints. For benchmarks such as RWARE, Pressure Plate, and Connector—where factorized masks yield empty or overly restrictive action spaces—contract-based shields are uniquely able to recover and sustain coordinated high-return behaviors.

Contrasts with Prior Work

• Decentralized Shielding: Basic local shielding (as in [elsayed2021safe], [alshiekh2018safe], [melcer2022shield]) cannot admit behaviors that are safe only under coordination. These methods either synthesize central shields or use conservative factorizations, thus eliminating optimal yet coordinated safe actions.
• Assume-Guarantee Compositionality: Closest in spirit to (Brorholt et al., 2024), but that work imposes an acyclic order on dependencies and a cascading-learning protocol. In contrast, the present approach supports fully general, circular dependencies, and leverages simultaneous certification over the complete tuple via contract-product fixed points.
• No Online Certification: Compared to probabilistic/logical shield learning (e.g., (Court et al., 17 Oct 2025, Asadi et al., 17 Nov 2025)), every selected contract is fully certified upfront, resulting in deterministic safety guarantees that hold for all executions, not in expectation or on average.

Implications and Future Directions

The contract-based framework substantially expands the space of behaviors available to decentralized MARL systems under strong safety requirements. By supporting constraint-based negotiation via pre-certified contracts, it enables fine-grained relaxation of conservatism. Beyond deterministic safety, the methodology could be adapted to weighting-probabilistic contracts, automated contract synthesis via formal methods, or continuous-state abstractions.

Potential future work includes:

• Extension to probabilistic or quantitative logics, for encoding risk metrics or cost-bounded safety.
• Heuristic and learning-based contract synthesis, mitigating the search space explosion for large systems.
• Integration with inter-agent communication (see (Chatterji et al., 2024, Aydeniz et al., 2024)) to further expand admissible coordination structures.
• Application to partially observable, asynchronous, or nonstationary environments.

Conclusion

This paper introduces a contract-based compositional shielding mechanism for safe MARL that balances decentralized execution, deterministic safety, and coordinated optimality. By synthesizing, certifying, and selecting among local obligation tuples, the framework enables decentralized agents to rely on teammates’ commitments for safety-critical coordination, unlocking behaviors that are inaccessible to conventional local or factored shields. Empirical results across demanding benchmarks verify the approach's ability to enforce formal safety properties while maximizing team performance.