System 3: Meta-Cognitive Agent Layer

• System 3 is the meta-cognitive layer that manages narrative identity, long-term survival, and self-alignment in computational agents.
• It integrates mechanisms such as process-supervised thought search, dual-store narrative memory, and hybrid reward signaling to optimize performance.
• Empirical results demonstrate improvements in cognitive efficiency and task success, reinforcing persistent agent behavior in dynamic environments.

System 3 denotes a third computational stratum in agent architectures, distinct from the canonical System 1 (fast, perception-action or heuristic response) and System 2 (deliberative, model-based planning) layers. As formalized in "Sophia: A Persistent Agent Framework of Artificial Life" (Sun et al., 20 Dec 2025), System 3 presides over the agent’s narrative identity, long-horizon survival, meta-cognition, and self-alignment. Its mechanisms operationalize core constructs from psychology and artificial life—including autobiographical memory, user and self modeling, meta-cognitive process supervision, and hybrid reward signaling—enabling persistence, identity continuity, and transparent explanation in long-lived artificial agents.

1. Three-Layer Cognitive Agent Architecture

System 3 is organized atop Systems 1 and 2 in a compositional stack. The overall cognitive agent is structured as follows:

• System 1 (Perception–Action): Encoders $E$ process sensory input $o_t$ into event vectors $x_t$; low-level policy $\pi_1$ maps commands to primitive actions $a_t$.
• System 2 (Deliberative Planning): A LLM or similar planner $\pi_2$ receives $\{x_{1:t}, m_t, g_t\}$ (history, memory, current goal) and outputs high-level commands $c_t$. The LLM is optionally fine-tuned or augmented by reinforcement learning, with policy parameters $\theta_2$.
• System 3 (Persistence and Meta-Cognition): An Executive Monitor asynchronously observes all internal events $(x_t, a_t, r_t, \text{trace}_t)$, supervises reasoning, maintains and verifies narrative identity, dynamically generates new sub-goals $g_{t+1}$, and synthesizes a hybrid intrinsic/extrinsic reward $R^{\mathrm{int}}_t$ that modulates ongoing agent behavior (Sun et al., 20 Dec 2025).

The Executive Monitor orchestrates the agent’s introspection loop, feeding outputs of System 3 into System 2’s deliberative core, thereby closing a persistent self-improvement cycle.

2. Core Computational Mechanisms of System 3

System 3 is composed of four synergistic modules:

2.1 Process-Supervised Thought Search

Goal expansion is formalized as a Tree-of-Thought (ToT) search. Each node $\mathbf{v}$ in the ToT carries a partial plan and a value estimate

$$\hat{V}(\mathbf v) = \lambda_{\mathrm{ext}}\hat{V}_{\mathrm{ext}}(\mathbf v) + \lambda_{\mathrm{int}}\hat{V}_{\mathrm{int}}(\mathbf v) - \kappa\, \mathrm{Cost}(\mathbf v)$$

where $\hat{V}_{\mathrm{ext}}$ is predicted extrinsic value, $\hat{V}_{\mathrm{int}}$ encodes intrinsic signals (curiosity/mastery), and $\mathrm{Cost}$ penalizes LLM resources. The search supplements LLM-generated beams with meta-cognitive pruning, retaining only nodes passing a self-critique filter. Reflection at episode boundaries further aligns predicted and realized returns via updates

$$\Delta \hat{V}(\mathbf v) \leftarrow \eta(r_{\mathrm{realized}}-\hat{V}(\mathbf v)).$$

2.2 Narrative Memory

Narrative memory is a dual-store, consisting of:

• An episodic buffer $\mathcal{B}_{\mathrm{mem}}$ that logs tuples $$\langle t,o_t,a_t,r^{\mathrm{tot}}_t,g_t,\text{trace}_t\rangle$$.
• A short-term cache $\mathcal{B}'_{\mathrm{mem}}$ for the current problem.

Memory queries leverage vector-embedded retrieval with cosine similarity

$$\mathrm{sim}(x, q) = \frac{\phi(x)\cdot\phi(q)}{\|\phi(x)\|\,\|\phi(q)\|}.$$

High-similarity episodes are injected as needed; aged entries may be summarized and compressed into high-level narratives via reflection.

2.3 User and Self Modeling

User goals are modeled as a belief distribution $p_u(g)$, updated using Bayesian inference: $p_u^{(t+1)}(g) \propto p(o_t\mid g)p_u^{(t)}(g).$ Self-modeling is encoded as a capability dictionary $\{(k_i, s_i)\}$ where $s_i$ is an estimated proficiency updated after each task: $$s_i \leftarrow s_i + \alpha_{\mathrm{self}}(r_i - s_i).$$

2.4 Hybrid Reward System

Total per-timestep reward is a sum: $$r^{\mathrm{tot}}_t = \alpha r^{\mathrm{ext}}_t + \beta_t r^{\mathrm{int}}_t$$ where $r^{\mathrm{int}}_t$ aggregates curiosity (novelty), mastery (skill improvement), and coherence (narrative consistency). Weighting $\beta_t$ is dynamically modulated via self-critique.

3. Autobiographical Identity and Meta-Cognitive Integrity

Sophia’s System 3 enforces narrative identity by requiring all episodic entries $e$ to reference at least one immutable “creed” from the self model: $C_e \neq \varnothing.$ A sliding-window analysis of narrative memory computes mean pairwise similarity on creed-tagged episodes: $$\frac{1}{\binom N2}\sum_{i<j}\mathrm{sim}(e_i, e_j) < \epsilon_{\min}$$ triggers re-alignment if narrative coherence deteriorates. Meta-cognitive subroutines can inject bridging episodes or creed reminders to maintain identity continuity.

4. Prototype Implementation and Empirical Insights

A browser-based, forward-learning prototype demonstrates System 3’s efficacy over a 36-hour continuous run (Sun et al., 20 Dec 2025). Key measured outcomes:

• Cognitive Efficiency: Chain-of-Thought step count per episode reduces by 80% on recurring tasks ($\text{CostReduction} = 0.8$).
• Performance Gains: For high-complexity (“Hard”) tasks, first-attempt success rises from 20% at $T=0$ to 60% after 36 hours ($+40$ percentage points by paired t-test, $p<0.01$).
• Narrative Consistency: The agent exhibits a stable autobiographical thread and transparent task organization, even under diverse, evolving user-supplied objectives.

The table below summarizes key task-level outcomes:

Task Difficulty	T=0 (h)	T=36 h	Δ (%)
Easy	95%	96%	+1
Medium	70%	78%	+8
Hard	20%	60%	+40

5. Theoretical Mapping and Broader Significance

System 3 formalizes several psychological and artificial life constructs as concrete modules:

Psychological Construct	Module Implementation
Meta-cognition	Executive Monitor (reflection)
Theory-of-mind	User Model (goal inference)
Intrinsic motivation	Hybrid Reward (curiosity/mastery)
Episodic/autobiographical memory	Memory Module (RAG retrieval, summarization)

System 3 provides a pathway for agents to continuously audit, re-align, and explain their reasoning, aiming for persistent alignment and long-horizon adaptation. This meta-layer is architecturally orthogonal and modular, allowing integration with varied System 1/2 stacks.

Sophia’s persistent agent wrapper exemplifies how self-directed improvement, identity auditing, and meta-cognitive reward shaping can be embedded in practical LLM-centric frameworks, establishing a foundation for research into computational artificial life and autonomous agent alignment (Sun et al., 20 Dec 2025).