Collaborative AI agents for autonomous experimental research with solid-state quantum spins

Abstract: An autonomous research laboratory could conduct an entire study, from formulating questions to preparing manuscripts. AI offers one route towards this goal. Recent AI systems take on several research roles, but may draw conclusions that the evidence does not support. Here, we present an autonomous research laboratory in which agents based on LLMs assess one another's work and can stop or redirect investigations. They can generate and reject research questions, require separate theoretical checks and conduct experiments designed to challenge their claims. We demonstrate these capabilities by investigating ensembles of quantum spins in diamond. The agents derived the minimum number of calibration settings needed to predict responses to a defined family of microwave pulse sequences. They checked the result computationally and designed experiments to distinguish competing predictions. An initial experiment agreed with the theoretical predictions, but repeated reference measurements showed that the signal was drifting during data collection. The agents therefore treated the result as inconclusive and revised the measurement protocol without relaxing the acceptance criteria. The revised experiment met those criteria and supported the theoretical predictions. This study demonstrates that collaborating AI agents can select research questions, derive theoretical results and test their predictions experimentally.

• The paper demonstrates a collaborative AI framework where agents autonomously formulate, verify, and experimentally test theoretical claims in solid-state quantum research.
• The study identified $L^{2}+1$ necessary and sufficient linearly independent calibrations to predict the ensemble response of quantum spins under variable pulse sequences.
• The agents rejected proposals, invalidated measurements, and adapted experimental designs to maintain rigorous scientific standards, resulting in successful validation of the calibration law for certain error models (Mexican Institute of Astrophysics)

The paper presents an autonomous research laboratory in which specialized LLM agents jointly formulate a research problem, derive and verify a theoretical result, design a discriminating experiment, operate a solid-state quantum-spin platform, and determine whether the resulting evidence supports the claim. Its central contribution is methodological as much as physical: scientific criticism is implemented as an active control mechanism within the research workflow. Agents can reject proposals, require independent verification, invalidate experimentally non-discriminating designs, classify apparently successful measurements as inconclusive, and revise the protocol without weakening precommitted acceptance criteria.

The investigation concerns ensembles of nitrogen-vacancy (NV) centres in diamond subjected to phase-controlled microwave pulse sequences. The physical system is sufficiently structured to permit exact analysis, but sufficiently heterogeneous to make ensemble prediction nontrivial: individual spins differ in detuning and drive strength, and the experiment observes averages over an unknown distribution of persistent pulse errors. This setting allows the agents’ theoretical claims to be tested against withheld measurements and against simpler interpolation rules.

Architecture of the collaborative laboratory

The system distributes the investigation across six principal roles: proposers, a principal investigator (PI), a computational researcher, an experimentalist, a manager, and independent reviewers. The proposers generate candidate questions independently. The PI evaluates them for novelty, logical validity, significance, theoretical tractability, and experimental feasibility. Once a question is selected, the PI develops the theory, the computational researcher performs independent symbolic and numerical checks, and the experimentalist develops control software and acquires data from the NV apparatus. The manager coordinates document exchange and contributes to experimental planning, while independent reviewers assess the resulting manuscript.

This division is not merely an organizational convenience. The workflow separates claim generation from claim evaluation and assigns different contexts and computational environments to the agents. Research outputs are exchanged as written documents, and progression through the workflow requires explicit verdicts. Rejected proposals must be revised, mathematical derivations must be independently checked, experimental predictions must be locked before target analysis, and experimental results must satisfy both accuracy criteria and controls. The design therefore treats scientific judgement as a sequence of decision points rather than as a final post hoc review.

Human involvement remained substantial at the infrastructure level. Human researchers assembled and configured the microscope, diamond, microwave, optical, and safety systems. They also established preset operating limits for the safety agent. Within those constraints, the agents managed the intellectual workflow and the measurements. The resulting autonomy is consequently bounded autonomy over a prepared experimental platform, not unrestricted laboratory operation.

Selective question generation and adaptive rejection

The proposal-selection process provides one of the clearest demonstrations that rejection changed the direction of the investigation. Over approximately 30 hours, the PI rejected 75 proposals, accepting none during the first 17 batches. The decisive objections included logical invalidity, prior establishment, insufficient significance, and lack of a feasible experimental test. Independent proposers also repeatedly converged on similar unsuccessful ideas.

The manager responded by extracting recurring rejected mechanisms and excluding them from subsequent proposal instructions. Those mechanisms accounted for 52 of the first 56 fresh proposals. After their exclusion, nine new proposals explored seven previously unrepresented directions. This is important because the system did not merely filter a fixed distribution of ideas; it altered the proposal-generation process using accumulated criticism.

The eventual research question emerged through revision rather than direct selection. A rejected proposal was identified by the PI as a useful seed, but its leading-order approximation was replaced by a formulation retaining the full dependence on pulse sequences. The accepted question asked for the minimum number of calibration measurements needed to predict the response of an inhomogeneous NV ensemble to every phase-controlled sequence of up to $L$ pulses. The problem has both an information-theoretic and physical character: its answer depends on the dimension of the response family induced by persistent pulse errors.

The large number of rejected proposals is also a limitation of the result. It demonstrates selectivity and adaptive search, but it does not establish that the generated proposal distribution is broadly novel or representative. The dependence on prompt instructions, model families, batch structure, and the manager’s exclusion list remains material.

Exact calibration law for persistent pulse errors

The central theoretical result is that $L^2+1$ calibration measurements are necessary and sufficient to predict the ensemble response for every phase-controlled sequence of at most $L$ pulses, under the assumption that each spin retains the same pulse-error parameter throughout a sequence.

For an individual spin, the pulse action is represented by a parameter $a$ encoding the drive and detuning response. An arbitrary ensemble is described by an unknown probability distribution over admissible values of $a$, together with an unknown fluorescence offset and contrast. The observed response to a sequence is therefore an affine transformation of the ensemble average of the sequence-dependent return probability.

The PI’s derivation identifies the real vector space spanned by all response functions for sequences of length at most $L$. Its dimension is exactly

$d_L=L^2+1.$

A set of $d_L$ linearly independent calibration responses spans this space. Every unmeasured sequence response can consequently be expressed as a linear combination of the calibration responses, with coefficients determined solely by the pulse phases and sequence structure. Applying the same combination to the measured ensemble means yields a predictor for the target response without reconstructing the microscopic distribution of spin errors.

The necessity proof is as important as the sufficiency construction. If fewer than $L^2+1$ independent calibrations are used, there exist physically admissible ensembles that agree on all calibration measurements but disagree on at least one unmeasured sequence. Thus, no more flexible predictor can guarantee universal reconstruction from an underspecified calibration set. The result is therefore an identifiability theorem, not simply a convenient interpolation scheme.

The reduction is numerically substantial. The initial proposal supplied the sufficient upper bound $8L^2+6L+1$, which gives 91 settings for $L^2+1$0. The PI reduced this to the exact minimum of 10. More generally, the number of required calibrations grows quadratically with sequence length, while the number of possible sequences grows exponentially for a fixed discrete phase alphabet. The implication is that a complete response family can be predicted with a substantially smaller calibration set, provided the persistent-error model and linear-independence conditions hold.

The calibration law does not imply that ten arbitrary settings suffice for three-pulse sequences. The settings must provide linearly independent responses. Repeated acquisitions do not increase the response-space dimension; they only improve estimation precision. The unknown fluorescence background and contrast are accommodated by the affine structure of the response representation, but the theorem remains conditional on the specified pulse model and on a common, sequence-persistent error parameter for each spin.

Independent computational verification and robustness analysis

The computational researcher independently reconstructed the response-space structure and verified dimensions of 26 and 37 for sequences of up to five and six pulses, respectively, consistent with $L^2+1$1. The verification was separate from the PI’s derivation and was conducted through a coding-agent workflow using the TSUBAME 4.0 supercomputer.

The agents also examined departures from the persistent-error assumption. The PI specified stress tests involving hidden phase rotations inserted between pulses. The computational researcher generated 252,620 exact certificates bounding calibration ambiguity and prediction error across the prescribed perturbation scenarios. For the six-pulse test, the certified prediction error remained at the photon-noise level for hidden rotations of approximately $L^2+1$2 or less.

This result qualifies, rather than removes, the principal theoretical assumption. Small violations of persistence may be practically tolerable under the tested conditions, but the robustness analysis does not establish universal validity under arbitrary temporal drift or correlated control errors. It uses a particular perturbation family, parameter domain, acquisition budget, indistinguishability criterion, and certification procedure. The theorem itself applies to the idealized persistent-error model; the numerical certificates assess only selected deviations from it.

Experimental design as a test of competing explanations

The transition from theorem to experiment required the agents to distinguish the calibration law from simpler interpolation procedures. The first proposed design failed this requirement. It used a complete grid of nine three-pulse calibration sequences and five shorter sequences to predict four withheld targets. Because the nine three-pulse measurements already determined the targets through Fourier interpolation, agreement between the experiment and the proposed law would not have been diagnostic. The PI rejected the design despite its apparent experimental feasibility.

The revised design combined five three-pulse calibrations with five sequences of zero, one, or two pulses. These ten settings span the $L^2+1$3 response space while leaving four three-pulse responses withheld. The shorter sequences contain information that is absent from the five three-pulse settings, so the calibration law and the interpolation baselines make different predictions for the withheld targets.

The prediction transfer was fixed through a precomputed matrix mapping the ten calibration means to the four target predictions. The agents also specified rival interpolation models, accuracy margins, control measurements, variance-validity conditions, and classifications for support, contradiction, and inconclusive outcomes before target analysis. The measured target values could therefore not influence the numerical prediction rule.

This design makes the empirical claim narrower and more precise. The experiment tests whether information from shorter sequences transfers to withheld three-pulse responses according to the derived response-space structure, and whether that transfer differs from the specified interpolation alternatives. It does not test every possible calibration method, every pulse-error model, or the universal applicability of the theorem outside the selected pulse family.

Rejection of the first apparently successful acquisition

The first main acquisition satisfied all ten precommitted accuracy criteria, but the PI classified it as inconclusive because independent controls failed. The global-phase and reference-pulse controls tested invariances that should have held under the intended dynamics. Their failure revealed signal drift during acquisition: the signal increased by 7.6% over the morning and exhibited a further 2.9% step when room lights were switched on at 11:48 a.m.

The acquisition alternated between two programs sampling different global-phase replicas at different times. Consequently, temporal drift was confounded with the phase condition. Agreement with the theoretical prediction was therefore insufficient evidence. The agents preserved the inconclusive classification rather than treating the failed controls as a nuisance or selectively retaining the accuracy results.

The protocol was revised without relaxing the acceptance criteria. Pilot acquisitions tested shorter alternation intervals and different sequence orders. The first pilot continued to fail the phase controls; the second, which performed phase switching within a single program, passed. The final protocol used randomized sequence ordering, within-program phase toggling, additional null diagnostics, a warm-up period, and six hours of acquisition.

The final run produced 21,600 valid one-second records and passed all accuracy, control, and variance-validity requirements. The withheld-target contrast was

$L^2+1$4

weighted counts per shot, compared with the calibration-law prediction of

$L^2+1$5

The interpolation alternatives predicted zero. The result therefore supported the law within the predefined tolerance and distinguished it from the two specified interpolation rules. The small experimental standard error relative to the prediction uncertainty reflects the fact that the prediction itself depends on noisy calibration means; uncertainty in the calibration-derived prediction was propagated, including covariance with the target residuals.

The episode is methodologically stronger than a single successful run would have been. It shows that the agents distinguished predictive agreement from experimental validity, diagnosed a concrete confound, changed the acquisition procedure, and retained the original decision rule. The evidence supports the transfer identity under the tested NV conditions, but it does not independently establish the complete mathematical theorem; the empirical data cover a finite design with four withheld targets.

Limitations and open questions

The autonomous laboratory depends on human-prepared infrastructure, safety limits, apparatus characterization, and scientific oversight. Humans assembled the apparatus, configured operating bounds, and retained responsibility for the claims and data integrity. The experiment therefore demonstrates agentic coordination within a constrained physical environment rather than complete end-to-end laboratory independence.

The result also depends on the correctness of the pulse model, the admissible parameter domain, and the assumption of sequence-persistent errors. The stress tests provide certified robustness for selected hidden rotations, but do not characterize arbitrary drift, non-Markovian dynamics, pulse-to-pulse correlations, or model misspecification. An open technical question is whether analogous finite-dimensional response spaces and exact lower bounds can be derived for broader error processes.

The proposal search exhibited substantial mode collapse before adaptive exclusions were introduced: 52 of 56 fresh proposals belonged to repeatedly rejected idea families. The manager’s intervention improved directional diversity, but the study does not quantify the novelty or quality of the excluded ideas, nor does it compare the resulting search with human-led or nonadaptive baselines.

The empirical validation is similarly bounded. It distinguishes the calibration law from two specified interpolation alternatives on one room-temperature NV platform and one pulse family. It does not test competing physical models that might also reproduce the measured contrast, nor does it establish performance under substantially different noise regimes or experimental platforms. Finally, although the codebase Auctopus is released, the study does not provide a controlled ablation isolating the effects of role specialization, independent contexts, prediction locking, manager intervention, and explicit acceptance criteria.

Conclusion

The paper demonstrates a collaborative AI workflow that operationalizes scientific criticism throughout an experimental investigation. Its strongest result is the exact calibration law: under persistent pulse errors, $L^2+1$6 linearly independent measurements are necessary and sufficient to predict all responses of phase-controlled sequences up to length $L^2+1$7. The agents independently verified the derivation, rejected a non-discriminating experiment, invalidated an apparently successful acquisition because controls failed, and obtained supporting evidence after revising the protocol without weakening the criteria. The study therefore provides a concrete example of LLM agents performing not only research tasks, but also the evaluative decisions that determine whether a theoretical claim has received adequate experimental support (2610.40589).