Automatic Oracle Transfer from Natural-Language Specifications

Develop methods to automatically generate domain-specific regression oracles from natural-language specifications within the Kitchen Loop framework, eliminating the need for manual per-domain engineering of oracles such as Anvil chain-fork execution for decentralized finance systems or multi-layer signal quality gates for signal platforms.

Background

The Kitchen Loop’s deployments rely on bespoke, domain-specific regression oracles to determine whether the system remains at least as good as before each iteration. Examples include executing demo strategies on Anvil chain forks for a DeFi SDK and applying deterministic quality gates plus anti-signal canaries for a signal intelligence platform.

While this oracle-centric trust model underpins safe autonomy, building each oracle currently requires substantial domain engineering effort. Automating oracle creation from specifications would reduce adoption friction and broaden applicability beyond the two validated domains.

References

Our deployments expose four open problems that we believe warrant dedicated research: OP1: Oracle Transfer. The Kitchen Loop relies on a bespoke regression oracle per domain (chain fork execution for DeFi, quality-gate pipeline for signals). Automatic generation of regression oracles from natural-language specifications --- eliminating the per-domain engineering cost --- is an unsolved challenge.

— The Kitchen Loop: User-Spec-Driven Development for a Self-Evolving Codebase  (2603.25697 - Roy, 26 Mar 2026) in Subsection "Open Problems" (Production Safety Record)

Other classes of representational load, such as novel string-formatting contracts, alternate data layouts, or project-local API conventions, are equally important in practice, but the strict metamorphic-testing guarantee does not directly transfer, because constructing a closed bijection on non-numeric input spaces is itself an open problem.

— Memorization Diagnostics for Code LLMs Should be Scale-Aware  (2608.12771 - Rajput et al., 13 Aug 2026) in Section 7, subsection “Scope of the Metamorphic Claim”

Nevertheless, testbench component generation alone does not complete the verification loop when expected-behavior modeling, scoreboard/checker construction, and protocol-aware coverage closure remain open.

— AgentDV: Closed-Loop Agentic AI for Hardware Design Verification  (2608.27148 - Goli et al., 27 Aug 2026) in Introduction, Section 1