Negative Knowledge as Failure-aware Shared Memory for AutoResearch

Abstract: AI-assisted research systems generate many failed attempts, but those failures rarely become a durable, shared knowledge asset. We propose a negative knowledge memory layer: a curator agent converts each failed attempt into a bounded, typed record in a shared bank, and a downstream research agent explicitly adopts or rejects those records before proposing its next experiment. We evaluate this layer in two settings: same-task retry on ScienceAgentBench and cross-task scientific research on two nonlinear math-physics PDE problems. The negative knowledge layer outperforms vanilla AutoResearch baselines while using fewer tokens; agents with the negative knowledge bank solve new tasks that all baselines fail to solve in PDE systems research. We also show that the previous negative knowledge bank can transfer and enhance AutoResearch on different PDE problems. These results suggest that structured negative knowledge is a knowledge asset that should be explicitly maintained in broader AI-engaged scientific research beyond a memory-compression or debugging aid, alongside positive findings, as a collective infrastructure for scientific memory. Code is available at https://github.com/hch-wang/Negative_Knowledge.

• The paper demonstrates that negative knowledge records, derived from failed experiments, significantly improve AutoResearch workflow efficiency.
• It introduces a structured memory layer that reduces token memory cost by up to 73.3% and increases coding pass rates.
• Negative records enable successful cross-task transfer in complex numerical systems, showcasing their modular utility in AI-driven research.

Negative Knowledge as Failure-aware Shared Memory for AutoResearch

Motivation and Architecture

"Negative Knowledge as Failure-aware Shared Memory for AutoResearch" (2606.21024) addresses the structural inefficiency in current AutoResearch workflows: failures from agent-driven scientific experiments are transiently used as local debugging signals but rarely persistent knowledge assets. The paper proposes and empirically validates a negative knowledge memory layer—a system-level protocol wherein a dedicated curator agent converts artifacts from failed experiments (e.g., code, logs, traces, and reasoning outputs) into bounded, typed records, stored in a shared project-level bank. Downstream research agents must ground new experiment proposals by inspecting the negative knowledge bank, adopting or rejecting records, thereby transforming failures from mere debugging aids into reusable epistemic infrastructure for agent-based scientific research.

The negative knowledge layer is agnostic to architectural specifics and formalizes records via closed, auditable vocabularies for failure classification. This schema enforces portability, auditability, and operational utility across agents and rounds.

Figure 1: The negative knowledge memory layer embeds curated failure records into a multi-agent AutoResearch pipeline, establishing persistent constraints for future experiment planning.

Numerical Evaluation: ScienceAgentBench Retry Performance

Evaluation on ScienceAgentBench, using Claude Sonnet 4.6, demonstrates quantitatively that negative knowledge memory improves performance on scientific coding tasks beyond baseline and token-intensive self-debug protocols. The results are as follows:

• Baseline and vanilla retry: Pass rate remains constant at $31.6\%$; retrying without structured memory does not improve outcome.
• Negative knowledge retry: Single curated record from one failed attempt increases the pass rate to $36.8\%$, with only 296 tokens, representing a $73.3\%$ reduction in memory size compared to self-debug.
• Deep negative knowledge retry: Condensing three failed rounds into one deep negative record yields $47.4\%$ pass rate, outperforming three-round self-debug ($44.7\%$), while using $28.3\%$ fewer tokens.

Strong numerical claims are established: negative knowledge records preserve useful repair information in compact form and allow agents to repair tasks whose repeated self-debugging fails. The explicit diagnosis and prescriptive recommendations encoded in negative records are predictive of failure repairability, supporting reduced token cost and operational efficiency.

Case Study: PDE Research and Cross-task Transfer

A comprehensive case study on coupled nonlinear PDE systems (BKdV and BNLS) evaluates negative knowledge as a reusable asset in mathematical physics research. The study proceeds in two stages:

• Stage 1: Bank Construction—Agents conduct multi-round stress tests on Burgers, KdV, Gardner, and BKdV systems, resulting in a bank with 58 records (15 positive, 43 negative). Negative knowledge captures not only failed routes but operational boundaries, e.g., Gibbs-driven instability thresholds for certain numerical schemes.
• Stage 2: Cross-task Transfer—For three novel tasks (soliton stability, soliton-packet fission, bore-soliton fusion) in the BKdV system, only agents given the negative bank solve all tasks; no-bank and positive-only-bank baselines fail or achieve partial success. Negative knowledge entries are not mere vetoes but actively constrain and prescribe next-step experiment selection, fundamentally redirecting research trajectories.

Figure 2: Compound solitons on the $u = v^2/2$ manifold, providing the target context for soliton stability (Test-A).

Figure 3: Soliton fission—a Gaussian packet decomposes into a soliton train (Test-B context for agent benchmarking).

Figure 4: Bore-soliton fusion—Burgers bore overtakes and fuses KdV solitons (context of Test-C for agent evaluation).

Trace analysis reveals that negative records are not only reused as constraints but carry prescriptive recommendations leading to successful repair. In contrast, positive knowledge fails to generalize between related numerical regimes due to path-dependent failure modes. Furthermore, the negative knowledge bank achieves partial transfer to a mechanistically distinct Burgers-NLS system: agents with only the BKdV bank solve two BNLS tasks, evidencing the modular utility and transfer of negative records across systems.

Practical and Theoretical Implications

The research establishes that negative knowledge—i.e., structured, auditable records of failures and non-working routes—is a knowledge asset with operational value for scientific memory infrastructure, not just memory compression or debugging. This contrasts with the historic publication bias toward positive findings and reconceptualizes the collective reward structure for agent-assisted research.

Practically, the negative knowledge bank enables amortized failure cost and targeted experimentation, accelerating research progress in open-ended and unstructured domains (such as nonlinear PDE research). Theoretically, it posits the scientific community as a large-scale AutoResearch system wherein negative knowledge can be maintained and leveraged as public infrastructure, especially as AI agents mitigate individual productivity constraints.

Speculation on Future Developments

Future directions include systematizing negative knowledge schemas for broader domains (e.g., chemistry, biology), integrating negative banks with provenance ontologies (PROV-O, ORKG, RO-Crate), and optimizing reward allocation across agent roles to sustain continual negative record production and sharing. Large-scale collective agent architectures may further realize negative knowledge as a constraint-based search accelerator for open-ended discovery, influencing reproducibility and collaborative dynamics in AI-driven science.

Conclusion

The proposed negative knowledge memory layer converts transient experiment failures into reusable, bounded, typed records indispensable for AutoResearch workflows. Structured negative knowledge demonstrably improves retry repairability, reduces token memory cost, and enables cross-task transfer in complex numerical research. The findings support establishing negative knowledge as shared epistemic infrastructure for AI-engaged scientific research, beyond its conventional role as a debugging artifact.