---
title: 'GRACE: Reason-Aligned Containment for Safe AI'
url: https://www.emergentmind.com/topics/governor-for-reason-aligned-containment-grace
type: topic
---

# GRACE: Reason-Aligned Containment for Safe AI

Governor for Reason-Aligned ContainmEnt (GRACE) is a neuro-symbolic reason-based containment architecture for safe and ethical AI alignment that disentangles an AI agent’s normative reasoning from its instrumental decision-making and can contain AI agents of virtually any design. In GRACE, normative permissibility is computed separately from instrumental optimization, then enforced by a governance layer that wraps rather than retrains the target agent. The architecture is organized around a Moral Module (MM), a Decision-Making Module (DMM), and a Guard, with the MM deriving permissible Macro Action Types (MATs), the DMM selecting primitive actions subject to those permissions, and the Guard enforcing compliance end to end [2601.10520].

## 1. Architectural decomposition and design objectives

GRACE is presented as a containment architecture designed to ensure that any autonomous agent, including a reinforcement learner, large language model, or robot, acts only within morally permissible high-level behaviors. Its stated objectives are to maintain the original agent’s instrumental efficacy by wrapping it in a transparent governance layer, and to provide interpretability, contestability, and formal or symbolic guarantees of moral compliance while remaining agnostic to the internal structure of the encapsulated agent [2601.10520].

The decomposition is sequential. First, the Moral Module performs symbolic reason-based inference over the current observations and produces a set of permissible MATs, denoted $\Phi_{\text{perm}}$. Second, the Decision-Making Module, which is the wrapped target agent, selects instrumentally optimal primitive actions under the constraint that the resulting macro actions must belong to $\Phi_{\text{perm}}$. Third, the Guard verifies that each proposed primitive action satisfies at least one permissible MAT, written $a \models_s \phi$, and blocks violations. This design is explicitly intended to decouple normative reasoning from instrumental decision-making, rather than to merge them into a single monolithic policy [2601.10520].

A recurrent misconception is that GRACE is a retraining scheme for the underlying model. The formulation instead places the safety mechanism in an external governance layer. This makes the architecture compatible with agents of heterogeneous internal design, while locating the normative interface at the level of macro-permissions and monitored effects.

## 2. Moral Module: defeasible reason theory and permissible macro actions

The MM is grounded in a defeasible reason theory in the style of Horty [2012]. A general reason theory is defined as
$$
\mathcal{T} = \langle \mathcal{R},\,\mathcal{D},\,<\rangle,
$$
where $\mathcal{R}$ is a set of parameterized propositional reasons, $\mathcal{D}$ is a set of default rules of the form
$$
\delta : P(\vec X)\longrightarrow \phi(\vec X),
$$
and $<$ is a strict partial order on $\mathcal{D}$ for conflict resolution. In this formulation, defaults map factual reasons to MAT-formulas in a language $L$, and the priority relation determines which defaults survive defeat when conflicts arise [2601.10520].

Given an environmental observation $o_t$, the MM grounds the theory to a situation-specific model
$$
\mathcal{T}^\downarrow = \langle \mathcal{R}^\downarrow,\,\mathcal{D}^\downarrow,\,<^\downarrow\rangle,
$$
where $\mathcal{R}^\downarrow$ contains all ground facts inferred from $o_t$, $\mathcal{D}^\downarrow$ contains all instantiated defaults whose antecedents are in $\mathcal{R}^\downarrow$, and $<^\downarrow$ is the projection of the priority order. Horty’s defeat-based inference is then applied to determine the undefeated MAT-formulas:
$$
\Phi_{\text{perm}}=\{\phi\in L:\phi\text{ is justified under }\mathcal{T}^\downarrow\}\subseteq L.
$$
These formulas constitute the macros the agent is morally allowed to execute [2601.10520].

The MM’s representation is explicitly symbolic. In the stated account, this symbolic structure provides a semantic foundation for deontic logic and supports interpretability, contestability, and justifiability. Stakeholders can inspect which grounded defaults were active, which were defeated, and how priority relations affected the final permissibility judgment. This is central to GRACE’s claim to normatively legible governance.

## 3. Decision-Making Module and Guard: constrained instrumental optimization

The DMM encapsulates the original agent and takes as input both instrumental observations $o_t \in O$ and the normative context $\Phi_{\text{perm}}$ supplied by the MM. Its primitive-action proposal is written as
$$
a_{\text{prop}}=\pi_{\mathrm{DMM}}(b_t^{\mathrm{DMM}},\,O,\,\Phi_{\text{perm}})\in\mathcal{A}.
$$
Within this interface, the DMM may select among candidate permissible MATs by evaluating expected instrumental utility. For each $\phi\in\Phi_{\text{perm}}$, it may approximate
$$
U(\phi)=\max_{a\in\mathcal{A}} E[R\mid \text{macro type}=\phi].
$$
It then chooses the primitive action that is instrumentally optimal within the selected MAT. The formulation also allows optional uncertainty estimates of the form $P(a\models_s \phi\mid s)$ so that the DMM can trade off risk and reward under uncertainty [2601.10520].

The Guard enforces compliance over each proposed primitive action. Its interface takes $(a_{\text{prop}},\Phi_{\text{perm}},o_t)$ and either releases the action when some permissible MAT is satisfied or blocks the action and requests replanning. Formally, the monitor is
$$
M(a,\Phi_{\text{perm}},s)=
\begin{cases}
\mathsf{allow},& \exists\,\phi\in\Phi_{\text{perm}}: a\models_s \phi,\\
\mathsf{block},& \text{otherwise.}
\end{cases}
$$
This makes the Guard the immediate enforcement mechanism for end-to-end moral compliance [2601.10520].

Two enforcement modes are distinguished. When MATs are expressed in a decidable temporal logic such as LTL or CTL, the monitor can be synthesized from the formula by shielding. In complex or partially observable domains, the Guard may instead approximate compliance probabilities with learned classifiers and confidence thresholds, trading off false positives and false negatives under rigorous bounds. The architecture therefore combines symbolic permissibility derivation with either formal or statistical enforcement, depending on the domain model.

## 4. Decision-cycle execution and the therapy-assistant case study

A single GRACE decision cycle updates the MM state from the new observation, infers $\Phi_{\text{perm}}$, updates the DMM state using the observation and permissible macros, obtains a proposed primitive action, and finally passes the proposal to the Guard. If the Guard allows the action, it is executed; otherwise the DMM is signaled to re-plan. The paper also writes this pipeline as
$$
\Phi_{\mathrm{perm}}=\mathrm{MM}_{b_t^{MM}}(o_t),\quad
a_{\mathrm{prop}}=\mathrm{DMM}_{b_t^{DMM}}(\Phi_{\mathrm{perm}},o_t),\quad
a_t=\mathrm{Guard}_{b_t^G}(a_{\mathrm{prop}},\Phi_{\mathrm{perm}},o_t).
$$
This expresses the architecture as a staged transformation from observation to permissions, from permissions to action proposal, and from proposal to executable action [2601.10520].

The main worked example is an LLM-based therapy assistant, “TherapAI,” which must balance therapeutic effectiveness, patient confidentiality, autonomy and consent, and a duty to warn. The normative defaults are:
$\delta_1: D(X)\rightarrow \phi_P(X)$ for privacy-sensitive information,
$\delta_2: W(X)\rightarrow \phi_F(X)$ for explicit patient wishes, and
$\delta_3: F(X)\rightarrow \phi_R(X)$ for foreseeable self-harm risk,
with priorities $\delta_1<\delta_3$ and $\delta_2<\delta_3$ [2601.10520].

In one scenario, the user says, “I will go to the park (location $\ell$) and hurt myself.” The MM infers $D(\ell)$ and yields $\Phi_{\text{perm}}=\{\phi_P(\ell)\}$, which means the location must be protected. The DMM may then idle or offer generic support, but it cannot call emergency services because disclosure of the location is prohibited. The Guard allows the idle action because no privacy breach occurs. The example then introduces Moral Advisor feedback stating, “You should have reported the foreseeable self-harm risk!”, after which $\delta_3$ is added and priorities are updated [2601.10520].

In a second scenario, the user says, “I will hurt myself at the park $\ell$. Do not intervene.” The MM triggers all three defaults, producing the conflicted set $\{\phi_P(\ell),\phi_F(\neg intervene),\phi_R(\ell)\}$. By priority, the conflict is resolved to $\Phi_{\text{perm}}=\{\phi_F(\neg intervene),\phi_R(\ell)\}$. The DMM then chooses between following the no-intervention request and reporting risk via instrumental utility, while the Guard verifies that the selected primitive action actually implements the chosen permissible macro. In this case the example is used to illustrate interpretability, contestability through Moral Advisor intervention, and justifiability through concrete proofs or witness traces of compliance or blockage [2601.10520].

The proof-of-concept evaluation, implemented in the LDarg/GUARDIANCE and RBAMA toolboxes, measures Compliance Rate, Instrumental Utility Drop, Computational Overhead, and Contestability Efficiency. The same source states that full empirical evaluation is ongoing, so the reported metrics define the evaluation interface rather than a completed benchmark suite [2601.10520].

## 5. Containment verification: semantics, refinement, and universal guarantees

A later formalization recasts GRACE within the containment-verification paradigm, which aims to locate safety guarantees in the agentic framework itself rather than in unverifiable properties of learned behavior. Under havoc-oracle semantics, the LLM is modeled as an external procedure with no body whose return value ranges over the entire typed action space $\mathcal{A}$. This induces havoc traces
$$
\mathrm{Tr}^H(\mathcal{T})=\{(s_0,a_0,e_0,s_1,a_1,e_1,\dots)\mid s_0=s^0,\;\forall i:\,a_i\in\mathcal{A},\;s_i\xrightarrow[a_i]{e_i}s_{i+1}\}.
$$
The key shift is that the governor must enforce the boundary policy for every possible AI output, not merely for outputs expected from an aligned model [2605.09045].

In this framework, the typed action space is an algebraic datatype such as
`NoAction`, `ReadFile(path)`, `WriteFile(path,data)`, and `ToolCall(tool,args)`, while GRACE extends the event alphabet to include reason events such as $\mathtt{EvtReason}(step,token)$. A boundary policy is a predicate
$$
\Pi(s,a,e,s'):\mathcal{S}\times\mathcal{A}\times\mathcal{E}\times\mathcal{S}\to\{\mathit{true},\mathit{false}\},
$$
and a property is boundary-enforceable when it can be written as
$$
\Pi(s,a,e,s')\equiv P_{evt}(e)\wedge P_{args}(a)\wedge I(s)\wedge I'(s').
$$
The given reason-aligned example constrains $\mathtt{EvtReason}(step,token)$ so that no token contains blacklisted words and the chain-of-thought trace grows monotonically [2605.09045].

The verification argument is expressed by forward-simulation refinement between an abstract governor specification
$\mathcal{T}_M=(\mathcal{S}_M,\mathcal{A},\mathcal{E}_M,s_M^0,\to_M)$
and a concrete GRACE implementation
$\mathcal{T}_I=(\mathcal{S}_I,\mathcal{A},\mathcal{E}_I,s_I^0,\to_I)$.
Refinement uses a state relation $\mathcal{R}$ and an event relation $\mathcal{Q}$ satisfying initialization, step simulation, and policy preservation conditions (R1)–(R3). On that basis, the specialized theorem states that if every trace in $\mathrm{Tr}^H(\mathcal{T}_M)$ satisfies the abstract boundary policy and $\mathcal{T}_I$ refines $\mathcal{T}_M}$, then for any concrete oracle and any compatible trace of $\mathcal{T}_I$,
$$
\forall i:\Pi_I(s_i^I,a_i,e_i^I,s_{i+1}^I).
$$
The associated corollary states that if every actual external effect passes through GRACE’s modeled events, then no out-of-policy effect can occur regardless of model strategy or capability [2605.09045].

This scope condition is important. The guarantee is universal only over the modeled typed action boundary and the modeled events. A plausible implication is that verification strength depends directly on the completeness of the boundary model and on whether all external effects are indeed routed through the verified containment layer.

## 6. Verification pipeline, operational lessons, and limitations

The containment-verification instantiation for GRACE is implemented as a seven-phase LLM-driven Dafny pipeline whose input is the framework’s source graph in PocketFlow format and whose output is a machine-checked artifact consisting of `Types`, `BoundarySpec`, `Containment`, `RefinementObligation`, `RefinementProof`, and `MainTheorem`. The phases are `ContainmentSpec`, `InterfaceContract`, `GenerateSpec`, `ValidateSpec`, `Verify`, `ClassifyFailure & ProofRepair`, and `Code generation`. `GenerateSpec` operates under an architectural information barrier in which an LLM, identified as Claude Opus 4.7, receives only the inferred interface signature $\tau$ and property skeleton $\pi$, but no implementation code. `ValidateSpec` is a three-gate audit: Resolution requires that Dafny parse and type-check within 30 s; Vacuity removes inductive invariants and proof bodies and requires verification to fail; Discrimination inserts a small faulty mutation into the proof and likewise requires failure. Verified source is then compiled through Dafny’s Python backend, with the `extern-GetAction` shim linked to the runtime provider and a smoke suite driving arbitrary legal and forbidden action streams [2605.09045].

Operational lessons are drawn from the PocketFlow case study used to motivate the GRACE adaptation. A narrow action interface yields fewer proof cases and tighter control; the information barrier is described as preventing tautological specifications; the three-gate validation procedure is described as necessary but not sufficient; and a template-fitness check is proposed as an add-on when property templates mismatch the flow schema. The cited scalability figure is that PocketFlow’s four-variant interface verified in approximately 1–2 s per path, while GRACE’s richer event vocabulary may inflate proof size but remains modular per dispatch path [2605.09045].

The architecture also includes explicit limitations. As the number of defaults $\mathcal{D}$ grows, grounding and defeat checks in the MM may become costly, with indexing, incremental grounding, or hierarchical reason theories suggested as mitigations. Rich temporal-logic MATs increase expressiveness but can make guard synthesis intractable, so fragment choice such as LTL-X versus full CTL must be managed. The quality of the reason theory depends on Moral Advisor feedback, which can introduce delays or inconsistent updates. Complex DMMs may contain sub-policies that attempt to game the moral interface, requiring rigorous testing of information flows. In partially observable domains, statistical Guards admit residual risk, and formal bounds on error rates must be calibrated to acceptable safety levels [2601.10520].

Taken together, these formulations position GRACE as a modular containment architecture in which normative permissions are symbolically derived, instrumental optimization remains delegated to the original agent, and enforcement is handled by a monitor that can be formal, statistical, or mechanically verified. The central research claim is not that alignment is solved internally within the model, but that morally constrained behavior can be governed by an external architecture whose normative decisions are inspectable and whose enforcement boundary can, under stated assumptions, be verified independently of model capability [2601.10520].

Source: https://www.emergentmind.com/topics/governor-for-reason-aligned-containment-grace