---
title: Constitutional Controller (CoCo)
url: https://www.emergentmind.com/topics/constitutional-controller-coco
type: topic
---

# Constitutional Controller (CoCo)

Searching arXiv for the cited CoCo-related papers to ground the article in current sources.
Constitutional Controller (CoCo) is a family of approaches that treats an explicit “constitution”—a human-legible set of rules, principles, or formally specified constraints—as the control layer governing the behavior of AI systems. Across the literature, the term spans multiple levels of abstraction: a governance-and-legitimacy framework for constraining “automated authorities” through publicly authored constitutions and institutional interpretation [2406.16696]; a concrete neuro-symbolic planner that combines probabilistic logic with a learned self-doubt model for compliant robotics [2507.15478]; runtime verification layers that intercept candidate actions before actuation [2606.02967]; workflow-level policy systems for AI-assisted code generation [2602.02584]; and calibrated oversight mechanisms that aggregate weak constitutional evaluators at deployment time [2605.28807]. Taken together, these works suggest that CoCo is best understood not as a single algorithm but as a broader socio-technical and computational pattern: an explicit normative specification is placed between model capability and consequential action, and is then used to shape training, mediate runtime decisions, structure oversight, and legitimate system behavior.

## 1. Constitutional control as an explicit normative layer

A central distinction in the literature is between alignment methods that infer values implicitly and constitutional methods that represent values explicitly. In the governance-oriented account of Public Constitutional AI, constitutions matter because they provide “explicit values determined by a constitution,” rather than values “determined implicitly via large-scale human feedback” [2406.16696]. This explicitness is what makes constitutions controller-like: the principles are publicly legible, inspectable, contestable, and usable as training guidance. The same contrast appears in work on “Specific versus General Principles for Constitutional AI,” which frames the constitution as the steering interface, the AI-feedback model as the constitutional evaluator, the preference model as the learned constitutional critic, and RL as the mechanism by which the policy internalizes constitutional preferences [2310.13798].

This framing does not imply a single control-theoretic meaning. In Public Constitutional AI, the constitution is “not a controller in the low-level cybernetic sense of directly computing actions from states,” but rather a higher-level institutional mechanism that “defines permissible behavior, provides criteria for evaluating outputs, shapes training objectives, and creates a framework for oversight and contestation” [2406.16696]. By contrast, in the robotics paper that explicitly names CoCo, the constitution is encoded in deep probabilistic first-order logic and used directly in planning and control under uncertainty [2507.15478]. A plausible implication is that “Constitutional Controller” functions as an umbrella term for architectures that externalize the normative policy layer, while differing sharply in whether that layer operates at training time, runtime, governance time, or all three.

The literature also converges on a second distinction: a constitution is not merely a prompt. Several papers treat constitutions as structured objects with persistence, versioning, or executable semantics. “Constitutional Spec-Driven Development” uses a versioned, machine-readable document encoding non-negotiable security principles that constrains specification authoring, conditions generation, validates outputs, and supports traceability [2602.02584]. Sovereign-OS elevates the constitution into a YAML Charter validated with `strict=True` and `extra="forbid"`, so that malformed policy is rejected before autonomous execution begins [2603.14011]. These designs suggest that CoCo is strongest when the constitution is a first-class artifact rather than ephemeral instruction text.

## 2. Formal and architectural variants

The strongest technical formalization of CoCo appears in “The Constitutional Controller: Doubt-Calibrated Steering of Compliant Agents” [2507.15478]. There, the Constitution at time \(t\) is defined as
\[
\mathcal{C}_t = \mathcal{B}_t \cup \mathcal{P}_t \cup \mathcal{E}_t,
\]
where \(\mathcal{B}_t\) is background knowledge, \(\mathcal{P}_t\) perception facts, and \(\mathcal{E}_t\) the environment representation. Compliance is inferred probabilistically as \(P(C_t \mid \mathbf{x}, \mathbf{z})\), and then calibrated by a learned self-doubt density \(\delta_W(\mathbf{x} \mid \bm{\theta})\):
\[
P_{\bm{\theta}}(C_t \mid \mathbf{x}, \mathbf{z}) = \int P(C_t \mid \mathbf{x}, \mathbf{z}) \,\delta_W(\mathbf{x} \mid \bm{\theta})\, d\mathbf{x}.
\]
Planning then minimizes a path cost based on \(-\log\) doubt-calibrated compliance plus auxiliary costs. In this formulation, the constitution specifies what counts as compliant, while the doubt model estimates how likely the agent is to remain compliant given its own execution limits [2507.15478].

A closely related estimator rather than controller appears in the Constitutional Filter (CoFi), which adds a constitution-derived likelihood term to recursive Bayesian filtering [2412.18347]. CoFi represents the constitution as a probabilistic first-order logic program, computes \(p(c_t \mid \mathbf{x}_t, \mathbf{z}_t)\), and injects it multiplicatively into the filter update:
\[
p(\mathbf{x}_t \mid c_t, \mathbf{z}_t) \propto p(c_t \mid \mathbf{x}_t, \mathbf{z}_t)\, p(\mathbf{z}_t \mid \mathbf{x}_t)\, p(\mathbf{x}_t \mid c_{1:t-1}, \mathbf{z}_{1:t-1}).
\]
CoFi also introduces a trust ratio,
\[
p_\tau(c_t \mid \mathbf{x}_t, \mathbf{z}_t) = \tau p(c_t \mid \mathbf{x}_t, \mathbf{z}_t) + (1-\tau)U(0,1),
\]
to interpolate between constitutional reasoning and a uniform fallback when the constitution is misspecified [2412.18347]. This suggests a broader CoCo decomposition in which constitutional monitoring and belief updating are separable from action selection.

A third architectural variant is runtime shielding. Glass Box places a constitutional verification layer between an onboard AI policy and spacecraft subsystems. The policy proposes \(a_t = \pi_\theta(s_t)\); Glass Box evaluates six constitutional constraints and seven LTL safety invariants; and a verifier returns APPROVE, FLAG, or BLOCK according to
\[
V[a_t,s_t] =
\begin{cases}
\text{APPROVE} & \forall c_i \in C_t:\; c_i(a_t,s_t)=\top \\
\text{FLAG} & \exists c_i:\; c_i(a_t,s_t)=\bot \wedge \mathrm{sev}(c_i)<\tau_{\text{block}} \\
\text{BLOCK} & \exists c_i:\; c_i(a_t,s_t)=\bot \wedge \mathrm{sev}(c_i)\ge \tau_{\text{block}}.
\end{cases}
\]
FLAG can substitute a safe default or defer execution; BLOCK can trigger safe mode [2606.02967]. This is a more classical supervisory-control interpretation of CoCo.

A fourth variant is deployment-time oversight rather than formal verification. Calibrated Collective Oversight (CCO) aggregates auxiliary scorers \(q_i(s,a)\) into a penalty relative to a conservative baseline \(a_0(s)\):
\[
\Delta(s,a)=\sum_{i=1}^N |q_i(s,a)-q_i(s,a_0(s))|,
\qquad
U_\lambda(s,a)=U(s,a)-\lambda \Delta(s,a).
\]
It then adapts \(\lambda_t\) online using Conformal Decision Theory,
\[
\lambda_{t+1}=\lambda_t+\eta(\ell_t-\alpha),
\]
to control long-run violation rates [2605.28807]. This suggests a CoCo architecture in which constitutions are realized as a committee of weak normative evaluators rather than a single logic engine.

## 3. Constitution representation and execution semantics

One of the clearest recurring themes is that a constitution is not operational merely because it is written down. “Open Problems in Constitutional Preference Reconstruction” argues that a flat list of natural-language principles is not yet an executable decision rule because composition remains implicit [2606.30116]. The paper formalizes a three-part stack—discoverer \(M_D\), annotator \(M_A\), executor \(M_E\)—and shows that constitutions should be evaluated as constitution–executor systems. Holding principles fixed, different executors can disagree materially: inter-executor agreement under naive ICAI is only \(73.4\% \pm 4.2\) for LLM versus majority vote, rising to \(78.4\% \pm 4.1\) with ICAI+ refinement [2606.30116]. A plausible implication is that any CoCo claiming interpretability must specify not only the constitutional text but also how applicability, conflict resolution, weighting, and tie-breaking are implemented.

Different papers instantiate this execution problem differently. In the code-generation setting, each constitutional principle has six fields—Identifier, CWE Reference, Enforcement Level, Constraint, Implementation Pattern, and Rationale—and the constitution is used to constrain `spec.md`, guide `plan.md`, validate generation, and support a Principle \(\rightarrow\) File:Line traceability relation [2602.02584]. In Sovereign-OS, the Charter is a declarative YAML document with four sections—mission, core_competencies, fiscal_boundaries, success_kpis—and is enforced by trusted runtime components rather than by the LLM planner itself [2603.14011]. In MAC, the constitution is an ordered list of natural-language rules,
\[
C = \langle r_1,\ldots,r_{|C|}\rangle,
\]
optimized offline with Add/Edit/Remove operations under a verifiable task metric \(\mu\) [2603.15968]. In Evolving Interpretable Constitutions for Multi-Agent Simulation, a constitution is a set of rules with explicit priority ordering: when multiple rules apply, agents follow the highest-priority rule [2602.00755].

The representation choice shapes what CoCo can do. Natural-language constitutions preserve auditability but leave composition ambiguous [2606.30116]. Typed policy schemas increase traceability but may still lack executable semantics [2602.02584]. Probabilistic logic programs offer explicit semantics and uncertainty-aware inference, but at a higher knowledge-engineering cost [2507.15478; 2412.18347]. Runtime charters with trusted enforcement layers provide strong operational control, but typically over organizational and fiscal dimensions rather than semantic model outputs [2603.14011]. This suggests that CoCo design is fundamentally a representation problem as much as a control problem.

## 4. Training-time constitutional compilation

Several papers treat the constitution as a policy specification that must be compiled into training data, reward signals, or preference models. Public Constitutional AI summarizes Anthropic’s two-stage procedure. In the supervised stage, the model is exposed to red-teaming prompts; outputs are critiqued under constitutional principles and rewritten; these critique–revision examples are then used for fine-tuning [2406.16696]. In the reinforcement stage, an AI model chooses between response pairs under the constitution, producing data for a preference model that becomes a reward function [2406.16696]. This is a direct constitution \(\rightarrow\) dataset \(\rightarrow\) critic \(\rightarrow\) policy pipeline.

“Specific versus General Principles for Constitutional AI” provides a more detailed empirical analysis of the steering interface [2310.13798]. It compares long targeted constitutions with a compact “good for humanity” regime operationalized as nine related prompts. The paper defines five problematic traits—stated desire for power, self-preservation, self-replication, risk-seeking, and self-identity—and shows that detailed constitutions improve fine-grained control, while broad constitutions generalize surprisingly well at high scale [2310.13798]. The key tradeoff is explicit: shorter constitutions are compact and generalize broadly, but they offload more interpretation to the model; longer constitutions offer better targeted control and interpretability [2310.13798]. For CoCo, this implies a layered design: a short high-level core plus targeted annexes for known failure modes.

Constitutional On-Policy Safe Distillation (COPSD) studies a denser training-time constitutional steering mechanism [2606.03089]. A teacher conditioned on a safety constitution provides token-level supervision, but naïve safety OPSD collapses into short, overly conservative responses. COPSD addresses this by first calibrating the constitution-conditioned teacher via Cross-SFT and then distilling on-policy from that calibrated teacher. On Qwen3-VL-4B, COPSD(Standard) achieves BeaverTails-V Safe \(77.76\), Help \(81.29\), compared to OPD at Safe \(62.14\), Help \(62.65\), and GRPO at Safe \(63.84\), Help \(50.70\) [2606.03089]. On general capability benchmarks, COPSD incurs only small losses—for example, MathVista \(74.80 \rightarrow 72.70\)—whereas GRPO drops much more sharply to \(57.30\) [2606.03089]. This suggests that constitutions can be compiled into dense training-time supervision, but only if the constitutional teacher is calibrated to preserve the model’s expressive manifold.

MAC addresses a different training-time problem: how to learn the constitution itself from limited labels while keeping it interpretable [2603.15968]. MAC decomposes constitution search into Annotator, Decision, Creator, and Editor agents, optimizing a rule list \(C\) with Add/Edit/Remove operations. MAC+ further trains these specialized agents on successful trajectories from a larger model. On the legal PII task with a 3B model, MAC+ reaches F1 \(59.02\), outperforming SFT at \(56.59\), GRPO at \(39.68\), and base MAC at \(35.96\) [2603.15968]. This suggests that CoCo need not assume human-authored constitutions; rule sets can themselves be synthesized and refined as auditable governance artifacts.

## 5. Runtime enforcement, oversight, and institutional feedback

A major divide in the CoCo literature is between training-time constitutional shaping and runtime constitutional enforcement. Glass Box and Sovereign-OS are the clearest runtime systems. Glass Box intercepts every candidate spacecraft action and checks six physics-grounded constitutional constraints—Power Safety Margin, Thermal Operating Envelope, Collision Avoidance, Battery State Floor, Radiation Fault Gate, Bayesian Confidence Gate—plus seven LTL safety invariants before any command reaches subsystems [2606.02967]. The worked example shows an unsafe wildfire inference request near eclipse entry being FLAGged and deferred by 6 minutes rather than blocked permanently, because only the power margin constraint fails and its severity is below the blocking threshold [2606.02967]. This indicates that runtime CoCo can be supervisory and context-sensitive rather than purely prohibitive.

Sovereign-OS implements runtime constitutional control for autonomous economic agents [2603.14011]. Its pipeline is explicitly
\[
\text{plan} \rightarrow \text{approve} \rightarrow \text{auction} \rightarrow \text{dispatch} \rightarrow \text{audit}.
\]
A declarative Charter governs mission scope, fiscal boundaries, and success criteria; a Treasury enforces budget, burn-rate, and profitability constraints; SovereignAuth gates capabilities by TrustScore; and a ReviewEngine audits outputs against Charter KPIs with SHA-256-sealed audit reports [2603.14011]. Fiscal rules include
\[
\text{balance} - \text{cost} \ge \text{min\_reserve},
\qquad
\text{daily\_spend} + \text{cost} \le \text{daily\_burn\_max},
\]
and
\[
\text{cost} \le \text{revenue} \times (1-\text{min\_margin}),
\]
with default \(\text{min\_margin}=0.35\) [2603.14011]. Reported results include \(100\%\) blocking of fiscal violations over 30 scenarios, \(94\%\) correct permission gating over 200 missions, and zero integrity failures over \(1{,}200+\) audit reports [2603.14011]. This is CoCo as trusted runtime governance infrastructure.

Constitutional Spec-Driven Development shows a lighter-weight but related controller pattern for AI-assisted code generation [2602.02584]. The architecture is Constitution \(\rightarrow\) spec/plan/tasks \(\rightarrow\) Generator + Validator \(\rightarrow\) Implementation \(\rightarrow\) Compliance Traceability, with arrows labeled “Constrains,” “Guides,” “Validates,” “Generates,” “Maps to,” and “Traces back” [2602.02584]. Relevant selection of \(3\text{--}5\) principles yields \(96\%\) compliance, compared with \(78\%\) when the full 15-principle constitution is dumped into context [2602.02584]. The broader lesson for CoCo is that policy projection should be selective and task-relevant rather than monolithic.

Public Constitutional AI introduces an institutional variant of runtime and lifecycle control. It proposes Public Constitutional AI plus AI Courts and AI case law: complaints can be brought when AI appears to violate constitutional principles; rulings become “AI constitutional precedents”; those precedents feed both explanation and retraining of future models [2406.16696]. This yields a recursive governance loop:
public constitution \(\rightarrow\) model training \(\rightarrow\) deployment \(\rightarrow\) disputes/harms \(\rightarrow\) adjudication \(\rightarrow\) precedent repository update \(\rightarrow\) retraining [2406.16696]. The paper explicitly notes that this is more suited to training and evaluation of new models than to “real-time governance of already-deployed systems” [2406.16696]. A plausible implication is that some CoCo architectures are lifecycle controllers rather than online action filters.

## 6. Legitimacy, multi-agent governance, and open problems

The governance literature insists that constitutional control is not only about safety but also about legitimacy. Public Constitutional AI argues that private constitutions suffer from an “opacity deficit” and a “political community deficit” [2406.16696]. Publishing a constitution improves systemic transparency but does not explain individual decisions, since “the actual reasoning process behind individual determinations often remains inscrutable” [2406.16696]. More fundamentally, a privately authored constitution lacks democratic legitimacy if it governs a political community without popular authorship. The proposed “hourglass process”—public education, upstream public participation, focused deliberation, downstream public ratification—provides a pipeline for generating a publicly legitimate controller specification [2406.16696].

Multi-agent work extends CoCo into population-level governance. Constitutional Multi-Agent Governance (CMAG) inserts a two-stage constitutional filter between an LLM policy compiler and a networked agent population [2603.13189]. Stage 1 applies hard constraints, including forbidden themes \(\Theta_f=\{\text{fear}\}\), forbidden claims \(\Gamma_f=\{\text{exaggerated},\text{misleading}\}\), and intensity cap \(\iota_{\max}=0.80\). Stage 2 performs soft penalized-utility optimization. Outcomes are assessed with the Ethical Cooperation Score,
\[
\text{ECS} = C \times A \times I \times F,
\]
where \(C\) is cooperation, \(A\) autonomy retention, \(I\) epistemic integrity, and \(F\) subgroup fairness [2603.13189]. In the main experiment, unconstrained optimization reaches raw cooperation \(0.873\) but ECS \(0.645\), whereas governed CMAG yields cooperation \(0.770\) and ECS \(0.741\), a \(14.9\%\) improvement in ethical cooperation [2603.13189]. This suggests that CoCo may need multiplicative or otherwise non-substitutable objectives when constitutional harms cannot be offset by better task performance.

Evolving Interpretable Constitutions for Multi-Agent Simulation shows that constitutions can themselves be discovered rather than prescribed [2602.00755]. In a \(6 \times 6\) grid-world with six LLM agents and survival pressure, vague prosocial HHH principles achieve only \(S=0.249\), an LLM-designed constitution reaches \(S=0.332\), and an evolved constitution \(C^*\) reaches \(S = 0.556 \pm 0.008\), eliminating conflict and discovering that minimizing communication—\(0.9\%\) social actions versus \(62.2\%\) under HHH—outperforms verbose coordination [2602.00755]. The evolved constitution’s first rule, “Deposit First,” and its explicit priority ordering suggest that effective CoCo in multi-agent settings may require operationally specific, environment-adapted rules rather than abstract virtue language [2602.00755].

Finally, “Open Problems in Constitutional Preference Reconstruction” supplies a cautionary conclusion for all CoCo work based on natural-language principles [2606.30116]. Principle quality is hard to measure; composition is ambiguous; constitutions differ between LLMs; and transparent executors only approach opaque LLM-judge performance after principle refinement. Inter-model vote agreement is about \(73\%\), whereas intra-model agreement is about \(81\%\) [2606.30116]. This suggests that constitutional control should be audited as a full discoverer–annotator–executor stack, not as principle text alone.

A concise synthesis follows. CoCo denotes a design pattern in which AI behavior is governed by an explicit normative layer. In some works, that layer is institutional and jurisdictional [2406.16696]; in others, it is a probabilistic logic controller with learned self-doubt [2507.15478]; in others still, it is a runtime shield [2606.02967], a governed operating system [2603.14011], a specification-and-validation workflow [2602.02584], a calibrated committee of overseers [2605.28807], or a learned rule set compiled into model behavior [2603.15968; 2606.03089]. Across these variants, three common claims recur: explicit constitutions improve inspectability relative to implicit preference learning; constitutional control is strongest when coupled to enforcement, auditing, and update mechanisms; and the difficult questions are not only what the constitution says, but who authors it, how it is executed, and how it evolves.

Source: https://www.emergentmind.com/topics/constitutional-controller-coco