CheetahClaws: Modular Harness for Agentic AI

• CheetahClaws is an open-source, Python-native reference harness that modularizes six key agentic AI components, including memory, context, and skill routing.
• It prioritizes transparency and verifiability by enabling independent activation and measurement of each module, thus supporting persistent and auditable agent behavior.
• The framework facilitates rigorous research by providing precise metrics for memory precision, context efficiency, communication fidelity, and safe evolution of agent systems.

CheetahClaws is an open-source, Python-native reference harness for agentic AI systems, explicitly architected to expose and modularize the six principal system-level components that translate the raw capabilities of foundation models into persistent, auditable, and verifiable agent behavior. Developed as part of the research outlined in "From Model Scaling to System Scaling: Scaling the Harness in Agentic AI" (Gu, 25 May 2026), CheetahClaws embodies the thesis that future advances in agentic AI will depend as much on the design and evaluation of agent harnesses as on improvements in model scale. It stands in contrast to production and multitool harnesses by prioritizing transparency, reproducibility, and fine-grained control over all axes of harness-level design.

1. System-Level Architecture and Design Rationale

CheetahClaws is predicated on the notion that, once the reasoning substrate ($\mathcal{R}$) of foundation models achieves sufficient capability, the dominant bottleneck in agentic performance migrates to the surrounding harness: memory ($\mathcal{M}$), context-constructor ($\mathcal{C}$), skill-router ($\mathcal{S}$), orchestration loop ($\mathcal{O}$), and governance layer ($\mathcal{G}$). The system-level objective is formalized as

$$P_H = \Phi(\mathcal{R}, \mathcal{M}, \mathcal{C}, \mathcal{S}, \mathcal{O}, \mathcal{G}),$$

where $P_H$ denotes overall harness performance, and $\Phi$ encapsulates the joint system effect. Unlike previous systems, CheetahClaws is engineered to allow researchers to independently activate, measure, and experiment with each harness axis. The harness uniquely records per-entry memory confidence and recency, operationalizing the trust axes—memory precision, durability, retrievability, and verifiability—rather than relying on indirect access patterns.

2. Core Components and Failure Modes

2.1 Context Governance ($\mathcal{C}$)

Context governance in CheetahClaws addresses the challenge of constructing model input that is maximally relevant, compact, and traceable, with a refresh policy that avoids "exposure without access." Instead of a traditional token-buffer, the context at time $\mathcal{M}$0, $\mathcal{M}$1, is determined by a selection policy $\mathcal{M}$2:

$\mathcal{M}$3

given user prompt $\mathcal{M}$4, memory pool $\mathcal{M}$5, and token budget $\mathcal{M}$6. $\mathcal{M}$7 scores each candidate memory snippet $\mathcal{M}$8 by

$\mathcal{M}$9

and selects top-$\mathcal{C}$0 items with provenance metadata. Traceability is enforced via unique IDs and source logging; freshness is monitored via verification timestamps.

2.2 Trustworthy Memory ($\mathcal{C}$1)

Memory in CheetahClaws is evaluated not only on precision and durability but also on runtime verifiability. The system guards against "stale-but-confident" memory—information that was once correct but has since drifted. Trust in a memory entry $\mathcal{C}$2 is a function of its stored confidence and recency:

$\mathcal{C}$3

with $\mathcal{C}$4 the stored confidence, $\mathcal{C}$5 the elapsed time since last verification, and $\mathcal{C}$6 a staleness penalty. Memory retrieval is ranked as

$\mathcal{C}$7

with all retrieved facts treated as hypotheses pending live re-verification (e.g., via tool or file read) before trust is finalized.

2.3 Dynamic Skill Routing ($\mathcal{C}$8)

Skill routing is directed by a policy $\mathcal{C}$9 mapping current agent state to a probability distribution over available skills. CheetahClaws avoids "confident-but-unchecked" failures by coupling routing to explicit post-condition verification. At each step, the orchestrator computes

$\mathcal{S}$0

selecting skill $\mathcal{S}$1, which produces output $\mathcal{S}$2. Each skill $\mathcal{S}$3 defines a Boolean post-condition $\mathcal{S}$4 that must succeed before effects are committed.

2.4 Orchestration Loop ($\mathcal{S}$5)

The orchestration loop interleaves context selection, reasoning, skill routing, and memory management. A canonical pseudocode representation is: $\mathcal{O}$2

2.5 Verification and Governance ($\mathcal{S}$6)

All tool invocations and memory writes in CheetahClaws are first staged and then subject to tool-specific rules, post-conditions, and audit logging enforced by the governance module. Only upon successful verification are changes committed; failures may trigger rerouting or aborts.

3. Interfacing Foundation Models and Tool Plugins

CheetahClaws provides a unified interface to foundation models by abstracting over model backends (e.g., OpenAI, Anthropic, self-hosted LLaMA variants): $\mathcal{O}$3 Tool plugins are registered with specific names, capability scopes, and post-condition checkers, supporting shell commands (shutil, subprocess), file I/O, grep-like operations, git introspection, and HTTP APIs. The harness’s key execution cycle assembles context, queries the LLM, parses requested actions, routes tool calls, verifies outcomes, and performs memory consolidation with governance: $\mathcal{O}$4

4. Evaluation Metrics, Qualitative Comparisons, and Benchmark Axes

The harness is designed to enable, not optimize, harness-level evaluation. Six principal benchmark axes are defined:

Axis	Metric Description	Typical Failure Mode
Trajectory quality	Successful completion under repeated trials	Dead-ends, non-reproducibility
Memory hygiene	$\mathcal{S}$7	Stale, imprecise, or incorrect entries
Context efficiency	$\mathcal{S}$8	Overlong or irrelevant context windows
Communication fidelity	Fraction of agent messages passing consistency checks	Contradictions, miscoordination
Verification cost	Average number of checks per action	Excessive runtime overhead
Safe evolution	Drift in memory precision over $\mathcal{S}$9 sessions	Memory contamination, reward gaming

Qualitative comparisons show that, relative to Claude Code (closed-source) and OpenClaw (TypeScript), CheetahClaws offers structured memory with explicit confidence/recency, context governance at several granularities, and full auditability through open-source Python. As a research scaffold, it supports controlled studies isolating effects of varying $\mathcal{O}$0 (context) and $\mathcal{O}$1 (skill routing) on outcome metrics. Full empirical benchmarking is reserved for future work (Gu, 25 May 2026).

5. Research Agenda and Open Challenges

The primary research agenda articulated for CheetahClaws centers on formalizing and benchmarking harness-level phenomena that lie outside traditional model-centric evaluation, including:

• Process-level metrics: Memory retrieval precision and drift, minimal-context efficiency, multi-agent communication fidelity, recovery strategies under failed verification, and adaptation under dynamic tool sets.
• Longitudinal evaluation: Benchmarking over sequences of related tasks to reveal drift, contamination, and proxy-gaming.
• Agent evolution standards: Introducing audit questions separating what persists (memory, skills, guardrails), what updates online versus with review, what is measured, and what remains auditable.
• Open challenges:
  • Automating context selection with guarantees of minimality/relevance.
  • Bounded-staleness memory with low tool-call overhead.
  • Learning skill routing policies optimized for verified progress (not fluency).
  • Subagent communication with consistency/conflict resolution.
  • Formal models for safe evolution under noisy rewards and adversarial inputs.

These directions reflect the increasing centrality of the harness in delivering reliable and safe agentic AI, with CheetahClaws serving both as a testbed for methodologies and a baseline for future system-scaling research (Gu, 25 May 2026).

6. Significance Within Agentic AI Research

CheetahClaws operationalizes the system-scaling paradigm shift by providing an experimental platform that foregrounds auditability, modularity, and process-level measurement. Its architecture and approach exemplify the transition from monolithic, model-centric evaluation to the dissection of emergent agent phenomena into harness-controlled, verifiable processes. By making every module and metric explicit, CheetahClaws offers a foundation for rigorous comparative studies and a development path for next-generation harnesses capable of supporting robust, scalable, and trustworthy agentic AI (Gu, 25 May 2026).