Discovery Loop Methodology

• A Discovery Loop is a closed, iterative process with a feedback mechanism that continuously updates hypotheses, experiments, observations, models, and decisions, involving human agents and/ higher-level systems in disciplines such as materials science, robotics, and legal technology.
• Discovery loops are distinguished from static workflows by their adaptive nature, retaining and incorporating negative results to guide future predictions and update candidate selection across various applications and research areas.
• Specific methodologies such as mechanism-discovery systems, LLM-AutoSciLab, LLM-ACES, CAMEO, and AIMBio-Mat use different implementations involving data, history, constraints, and candidate equations to advance the discovery loop, each authentically pursuing autonomous outcomes

Discovery loop is a closed, iterative process in which hypotheses, experiments, observations, models, and subsequent decisions continuously update one another. Unlike open-loop workflows that generate predictions from static data, a discovery loop selects actions or measurements, incorporates their outcomes, revises its internal state, and uses the revised state to guide further exploration. The concept appears across materials science, scientific machine learning, formal verification, process discovery, robotics, legal technology, and autonomous research. Its implementations range from human–machine systems, in which experts remain responsible for objectives and validation, to agentic or automated systems that coordinate hypothesis generation, experiment selection, execution, analysis, and memory.

1. Conceptual foundations and defining properties

A discovery loop consists of a state, a hypothesis or candidate space, an action-selection mechanism, an observation process, and an update rule. In abstract form, a system maintains accumulated data, prior hypotheses, evidence, and constraints; proposes a candidate action; observes the resulting data; and updates the state before selecting the next action. This structure is explicit in mechanism-discovery systems such as LLM-AutoSciLab, where the state includes observations, structured memory, and a current hypothesis set (Kabra et al., 21 May 2026), and in LLM-ACES, where symbolic operator priors, candidate equations, trajectories, and an experience buffer co-evolve through repeated acquisition (Abhyankar et al., 23 Jun 2026).

The defining property is feedback. In materials discovery, experimental outcomes—including positive and negative results—are added to the information available for subsequent prediction and candidate selection (Pogue et al., 2022). In autonomous materials environments, every expensive oracle evaluation updates the evaluated history, convex hull, stability labels, and future proposal policy (Malik et al., 28 Jan 2026). In scientific hypothesis discovery, rejected hypotheses are retained because they indicate directions that have already been explored unsuccessfully or that require reformulation (Zhao et al., 1 Jul 2026).

A discovery loop is therefore distinguished from several related workflows:

• Static prediction: a model is trained on a fixed dataset and produces predictions without using subsequent observations to alter the search.
• Open-loop screening: candidates are generated and ranked, but experimental feedback does not systematically change later candidate selection.
• One-shot optimization: a fixed initial design is evaluated without adaptive modification of the candidate or experiment space.
• Closed-loop discovery: candidate generation, evaluation, and subsequent decision-making are coupled through state updates.
• Agentic discovery: specialized agents coordinate objective setting, knowledge acquisition, prediction, experimentation, analysis, publication, planning, exploration, and enforcement (Pauloski et al., 15 Oct 2025).

The loop may be closed at different levels. CAMEO autonomously controls synchrotron characterization and selects subsequent measurements, but synthesis is performed beforehand and optical-property extraction remains partly human-assisted (Kusne et al., 2020). The Discovery Loop in AIMBio-Mat includes human review, laboratory measurements, uncertainty-aware optimization, provenance, and governance, but is explicitly positioned as exploratory and preclinical infrastructure rather than clinical decision-support software (Mei et al., 20 May 2026). In contrast, MADE provides a benchmark abstraction in which generators, filters, planners, selectors, agents, and oracles can be composed and evaluated under a finite query budget (Malik et al., 28 Jan 2026).

A loop is not necessarily fully autonomous. Human researchers may define objectives, supply priors, approve candidates, perform difficult measurements, validate discoveries, or override recommendations. The human role often shifts from manually choosing every action to supervising a system that proposes and executes actions while preserving expert judgment and responsibility (Kusne et al., 2020, Pauloski et al., 15 Oct 2025).

2. Hypothesis formation, representation, and search-space construction

Discovery loops require a representation of candidate explanations, designs, or actions. The representation determines what the system can discover and how efficiently it can search.

In LLM-ACES, the LLM proposes operator priors rather than final equations. Each prior defines a constrained symbolic subspace, within which PySR fits candidate ODEs using observed data and a complexity penalty (Abhyankar et al., 23 Jun 2026). This separates structural inductive bias from numerical fitting. The LLM contributes operator families and exploration of structurally distinct regions, while symbolic regression estimates coefficients and evaluates candidates on validation data.

LLM-AutoSciLab maintains an ensemble of competing mechanisms, including symbolic equations, kinetic laws, and signed regulatory graphs (Kabra et al., 21 May 2026). The ensemble is intended to preserve structural uncertainty rather than prematurely collapse to a single explanation. Candidate mechanisms may differ in functional family, variable inclusion, nonlinearities, graph topology, edge orientation, or activation and repression signs. Experiments are then selected where these candidates make divergent predictions.

DiscoPER represents each hypothesis as an executable program that consumes data and returns a judgment and statistical evidence (Zhao et al., 1 Jul 2026). This creates a broad hypothesis space capable of expressing interaction, mediation, moderation, conditional relationships, spatial patterns, image-derived attributes, and multimodal analyses. Hypotheses are accepted only after their code executes and satisfies effect-size, significance, and held-out validation criteria.

In materials discovery, the search object may be a composition, crystal structure, synthesis condition, processing trajectory, or waveform. MADE represents candidates as compositions combined with crystal structures and uses an oracle to return formation energies (Malik et al., 28 Jan 2026). The closed-loop workflow filters invalid, redundant, or chemically implausible candidates before selecting one for an expensive oracle evaluation. In closed-loop processing-protocol discovery, a 4-second scanning-probe bias waveform is represented by 33 Fourier coefficients, allowing evolutionary mutation and crossover to generate executable protocol variants (Liu et al., 11 Jun 2026).

Synthesis discovery systems combine literature-derived knowledge with numerical process variables. For the thin-film synthesis of $\mathrm{Rb_3BiI_6}$, an LLM extracted synthesis information, troubleshooting guidance, and related-compound knowledge, then proposed bounds and initial conditions for spin speed, precursor concentration, annealing temperature, solvent ratio, and solute ratio (Sheng et al., 16 Aug 2026). The LLM therefore influences both the initial candidate distribution and the explored parameter bounds.

The representation can also be a graph over samples or categories. CAMEO represents a materials composition spread as a graph whose nodes are samples and whose edges connect neighboring compositions. GRENDEL infers phase-map hypotheses under physical constraints, while Gaussian random-field harmonic energy minimization propagates labels to unmeasured compositions (Kusne et al., 2020). In generalized category discovery, Loop uses BERT representations, K-means clusters, local neighborhoods, and LLM-selected semantic neighbors to refine the representation geometry (An et al., 2023).

Representation quality imposes an upper bound on discovery. Composition-only superconductivity models omit crystal structure, defects, disorder, pressure, doping, and synthesis history (Pogue et al., 2022). Similarly, a waveform model may be unable to capture protocol effects that fall outside its executable representation, and an image-derived feature may be too noisy to support statistically useful discovery (Zhao et al., 1 Jul 2026). Discovery loops therefore require representations that preserve the variables, context, constraints, and mechanisms relevant to the target phenomenon.

3. Adaptive acquisition and experimental selection

The principal function of a discovery loop is to determine what should be measured next. Acquisition strategies usually balance exploitation of promising candidates against exploration of uncertain or structurally distinct regions.

Mechanism-discovery systems select experiments by predictive disagreement. LLM-AutoSciLab generates candidate mechanisms and searches for observations on which they disagree; low confidence activates a disambiguation mode, while higher confidence shifts the system toward refinement within the supported mechanism family (Kabra et al., 21 May 2026). LLM-ACES similarly integrates candidate ODEs from possible initial conditions and selects the condition maximizing average pairwise trajectory disagreement (Abhyankar et al., 23 Jun 2026). This targets identifiability rather than merely reducing prediction error on already observed trajectories.

The distinction is important because multiple mechanisms may agree locally while diverging elsewhere. A passive learner can obtain low interpolation error while recovering an incorrect governing law. An adaptive loop instead searches for an intervention or initial condition at which competing explanations make different predictions. The resulting observation has high discriminatory value even if it contributes little to average error on the existing dataset (Kabra et al., 21 May 2026, Abhyankar et al., 23 Jun 2026).

Materials loops use related but not identical acquisition rules. CAMEO initially performs risk minimization for phase-map learning, selecting structural measurements expected to reduce phase-label error. After the phase map reaches an Fowlkes–Mallows Index of at least 80%, it switches to phase-region-specific Gaussian-process optimization of $\Delta E_g$ using predictive mean, uncertainty, and distance to phase boundaries (Kusne et al., 2020).

The superconductivity discovery loop combines ML filtering with expert selection. RooSt screens candidates from Materials Project and OQMD, after which experts assess stability, synthesis feasibility, metallicity or dopability, nearby phases, safety, and impurity risks. Experimental results then refine subsequent predictions and candidate selection (Pogue et al., 2022). The workflow illustrates that acquisition can depend on information absent from the predictive model.

In autonomous materials benchmarking, acquisition occurs through a modular pipeline:

1. select a composition or region;
2. generate structures;
3. filter invalid and duplicate candidates;
4. score or rank candidates;
5. select one structure for the expensive oracle;
6. update the convex hull and discovery statistics;
7. use the result to guide the next proposal.

MADE evaluates this process under a fixed oracle budget and distinguishes final discovery count from early discovery through the area under the discovery curve (Malik et al., 28 Jan 2026).

For multi-objective synthesis, noisy expected hypervolume improvement selects batches expected to improve the Pareto front. In the $\mathrm{Rb_3BiI_6}$ campaign, hyperspectral imaging produced coverage, uniformity, and phase-purity objectives; Gaussian-process models were updated after each experimental round, and qNEHVI selected subsequent conditions (Sheng et al., 16 Aug 2026). In processing-protocol discovery, an upper-confidence-bound criterion combines predicted reward and uncertainty, while a diversity filter prevents a batch from collapsing onto near-duplicate waveforms (Liu et al., 11 Jun 2026).

Acquisition may also target uncertainty in local cluster structure. Loop for generalized category discovery uses Local Inconsistent Sampling, which intersects high neighborhood-inconsistency and high cluster-assignment-entropy samples. An LLM then selects the most semantically appropriate neighbor among candidates from competing clusters, and the result becomes contrastive-learning supervision (An et al., 2023).

4. Updating models, memory, and scientific state

A discovery loop requires more than appending new observations. It must update models, hypotheses, search priorities, and memory in a way that preserves provenance and failed attempts.

In symbolic and mechanistic discovery, newly acquired trajectories or perturbation responses are added to the dataset, candidate models are refit or rescored, and unstable or contradicted hypotheses are removed or downgraded. LLM-ACES also updates an experience buffer containing high-scoring and low-scoring symbolic candidates, allowing later LLM prompts to exploit successful operator patterns while avoiding previously unsuccessful structures (Abhyankar et al., 23 Jun 2026). LLM-AutoSciLab stores validated and failed mechanisms, bootstrap confidence, fit statistics, and prior search regions (Kabra et al., 21 May 2026).

DiscoPER adds a second-order update. Every five iterations, the Reflect module analyzes accepted and rejected claims to identify gaps, confounds, contradictions, recurring variable clusters, and interaction hints. The resulting guidance redirects future hypothesis generation (Zhao et al., 1 Jul 2026). This is distinct from ordinary conversational memory: the system treats previous discoveries as empirical data about the search process itself.

Materials systems update both predictive models and the state of the physical search space. In MADE, each oracle result changes the convex hull, potentially revising the stability status of previously evaluated materials (Malik et al., 28 Jan 2026). In synthesis optimization, measured film properties update Gaussian-process surrogates and the Pareto front. In the ferroelectric waveform loop, the full archive of waveform–reward pairs is retained, and the DKL surrogate is retrained for each generation (Liu et al., 11 Jun 2026).

AIMBio-Mat defines a more extensive scientific record. Each learning object includes composition or structure, processing, structural descriptors, context, measured outcomes, uncertainty, governance metadata, and provenance links to raw data, protocols, instruments, samples, and model versions (Mei et al., 20 May 2026). Failed syntheses, adverse responses, null results, and protocol deviations are retained. The platform also proposes model registries, model cards, datasheets, applicability-domain reports, uncertainty calibration, and decision logs.

The role of negative information is recurrent. A failed synthesis may identify an infeasible region; a negative superconductivity result is informative only when the intended phase was successfully synthesized and characterized; a rejected statistical hypothesis can prevent repeated testing of an unproductive direction; and a failed candidate equation can expose a spurious operator family (Pogue et al., 2022, Zhao et al., 1 Jul 2026, Abhyankar et al., 23 Jun 2026).

Updating is also necessary for safety and compliance. In human-on-the-loop legal discovery, a human correction must invalidate dependent summaries, queries, evidence sets, and downstream decisions rather than being recorded only as a terminal override (Sinha et al., 18 Jun 2026). The system must preserve branchable state, rollback information, and provenance for each action.

5. Validation, stopping, and evaluation

Discovery loops require evaluation criteria that measure more than predictive fit. Relevant metrics include mechanism recovery, discovery count, discovery timing, diversity, uncertainty calibration, resource consumption, and failure or risk rates.

For mechanism discovery, symbolic accuracy distinguishes recovery of the governing structure from numerical interpolation. LLM-AutoSciLab reports symbolic accuracy, numerical exact accuracy, RMSLE, edge F1, sign accuracy, and exact graph recovery across NewtonBench, ActiveSciBench-Chem, and ActiveSciBench-GRN (Kabra et al., 21 May 2026). LLM-ACES evaluates reconstruction, held-out generalization, out-of-distribution extrapolation, symbolic accuracy, and expression complexity (Abhyankar et al., 23 Jun 2026).

For autonomous materials discovery, MADE uses mSUN, discovery curves, area under the discovery curve, acceleration factor, enhancement factor, compositional diversity, unique compositions, unique space groups, and structural discrepancy (Malik et al., 28 Jan 2026). These metrics distinguish a system that finds many discoveries early from one that eventually finds the same number only after exhausting its budget.

For materials synthesis, hypervolume measures the dominated region of the multi-objective space. The $\mathrm{Rb_3BiI_6}$ workflow compared LLM-assisted initialization with Latin hypercube sampling and found more Pareto-optimal samples and higher hypervolume for the LLM-assisted arm at matched trial counts (Sheng et al., 16 Aug 2026). In the reported campaign, a good sample satisfied coverage above $0.90$, uniformity roughness below $1.0$, and phase-purity proxy above $0.90$.

Closed-loop systems also require calibration and risk metrics. AIMBio-Mat proposes metadata completeness, expected calibration error, confidence-interval coverage, applicability-domain detection, failed trials avoided, cost per validated candidate, expert-override frequency, and audit completeness (Mei et al., 20 May 2026). The legal discovery loop evaluates privilege-waiver risk, escalation rate, first-error position, and rollback recovery rate rather than relying only on endpoint precision and recall (Sinha et al., 18 Jun 2026).

Stopping conditions vary by system. MADE primarily stops when the oracle budget is exhausted (Malik et al., 28 Jan 2026). CAMEO transitions from phase mapping to property optimization at an FMI threshold and reports the GST optimum after 19 iterations (Kusne et al., 2020). DiscoPER uses 100 iterations with reflection every five iterations (Zhao et al., 1 Jul 2026). LLM-ACES and LLM-AutoSciLab use fixed acquisition rounds or budgets, while confidence changes the acquisition regime rather than necessarily terminating the process (Kabra et al., 21 May 2026, Abhyankar et al., 23 Jun 2026). AIMBio-Mat proposes governed stopping based on target satisfaction, exhausted budget, unacceptable risk, applicability-domain failure, insufficient expected improvement, or expert recommendation (Mei et al., 20 May 2026).

A formal stopping rule is often absent. This is a significant distinction between a demonstrated workflow and a fully specified autonomous discovery policy. Many systems provide acquisition functions and evaluation metrics but leave campaign termination to a fixed budget, a planned number of iterations, or human judgment.

6. Applications, limitations, and governance

Discovery loops have been applied or proposed across diverse domains. In materials science, they have identified an epitaxial nanocomposite phase GST467 with $\Delta E_g=0.76\pm0.03$ eV, discovered a superconducting phase in the Zr–In–Ni system, optimized ferroelectric conditioning waveforms, and explored thin-film synthesis of $\mathrm{Rb_3BiI_6}$ (Kusne et al., 2020, Pogue et al., 2022, Liu et al., 11 Jun 2026, Sheng et al., 16 Aug 2026). In formal verification, the Invariant Analysis Engine turns failed proof attempts into actionable invariant-discovery tasks, combining human or agent-generated predicates with theorem-prover feedback (Walter et al., 2021). In category discovery, sparse LLM neighbor judgments refine BERT representations and generate semantic names for novel clusters (An et al., 2023). In wireless robotics, the discovery loop describes a protocol-level feedback process in which delayed ROS 2/DDS discovery messages generate reliability traffic that further increases contention (Choi et al., 3 Aug 2026).

Agentic scientific architectures generalize the loop to objective formulation, knowledge retrieval, experiment planning, execution, analysis, publication, memory, exploration, and enforcement (Pauloski et al., 15 Oct 2025). Embodied-science proposals extend this architecture toward perception, language reasoning, physical action, and experimental feedback, although the supplied document associated with the proposed PLAD framework is a manuscript template and does not itself report a demonstrated system (Zhuang et al., 20 Mar 2026).

The principal limitations are methodological and infrastructural. Models may be misspecified, candidate spaces may exclude the true mechanism, LLMs may generate poor hypotheses, and adaptive data selection can introduce bias. Simulator-based benchmarks may overestimate performance relative to physical laboratories. Composition-only models omit structure and processing history. LLM-derived image features may be noisy. Statistical validation does not establish causality, and open-ended hypothesis search creates multiple-testing concerns. Expert filtering improves practical outcomes but introduces human selection bias and can reduce reproducibility (Pogue et al., 2022, Kabra et al., 21 May 2026, Zhao et al., 1 Jul 2026).

Physical experiments add failure modes absent from static datasets: failed synthesis, phase mixtures, contamination, batch variability, instrument drift, changing sample state, safety constraints, and irreversibility. The ferroelectric waveform study mitigates state dependence through local pre/post normalization, fresh measurement locations, diversity constraints, and uncertainty-aware selection, but it does not guarantee calibrated uncertainty under severe distribution shift (Liu et al., 11 Jun 2026). The superconductivity study emphasizes that a negative result may reflect failed synthesis rather than absence of superconductivity (Pogue et al., 2022).

Governance is therefore integral to the discovery loop. AIMBio-Mat requires FAIR metadata, provenance, uncertainty, governance fields, negative-result capture, access control, model documentation, and risk-tiered review (Mei et al., 20 May 2026). Human-on-the-loop legal discovery places planning validation, reasoning checkpoints, execution sandboxing, and uncertainty-gated escalation inside the operational loop. In its synthetic evaluation, a threshold of $0.5$ reduced privilege-waiver risk from $\Delta E_g$0 to $\Delta E_g$1 while routing $\Delta E_g$2 of documents to attorney review (Sinha et al., 18 Jun 2026). The result illustrates a general principle: autonomy should be constrained by consequence, reversibility, uncertainty, and the cost of human review.

The broader significance of discovery loops lies in their redefinition of scientific automation. The central task is not merely to generate predictions faster, but to select informative actions, preserve failed and successful evidence, revise hypotheses, coordinate heterogeneous tools, and maintain an auditable relationship between decisions and observations. The strongest implementations combine adaptive acquisition, explicit uncertainty, structured memory, physical or formal validation, and human oversight. Their effectiveness depends less on any single model than on the integrity of the entire feedback system.