Praxist: From Experimental Artifacts to Solution Lineages

Abstract: Autonomous R&D agents now write, run, and improve executable artifacts under automated evaluation---but largely as laboratory instruments: shown on curated benchmarks, with gains that are hard to trace to a cause and costs well above what sustained engineering practice absorbs. The limitation is structural. Most systems treat each attempt as nearly self-contained, so logs, memories, and search trees record what happened without establishing which design element produced an improvement, whether its evidence survived validation, or how it recombines with others. Long campaigns therefore keep re-learning the same lessons. We introduce Praxist, a lineage-centered generational system that converts reproducible artifacts and evaluator outcomes into a typed evidence graph of findings, lane-structured frontiers, and agendas. Separating local artifact construction from cohort-level evidence synthesis lets later attempts inherit validated mechanisms, unresolved claims, and useful constraints, and leaves results attached to an inspectable lineage. On the standardized 75-task MLE-bench suite, the finalized official-grader results give Praxist 60 medals (80.0\%), 49 of them gold, against 55 medals (73.3\%) and 34 gold for a Claude Code baseline on Claude Opus 4.8---at a recorded model spend of US$3,054 versus US$38,370, roughly a twelfth of the cost. Four case studies---quantitative trading, LiDAR-inertial-visual SLAM, tokamak magnetic control, and rocket landing---carry the same process into open-ended engineering problems, improving on each task-native baseline in headline accuracy, survival, or resource cost, with the discovery path on record. Stronger artifacts at an order of magnitude less spend, each backed by an auditable lineage, are, to our knowledge, first brought together here: the operating profile production research requires, not the one a benchmark demonstration establishes.

• The paper demonstrates a new approach to autonomous research, focusing on converting artifacts from experimental campaigns into inheritable, structured knowledge.
• The Praxist system shows optimized performance in complex tasks, such as solving 80% of MLE-bench tasks (compared to the baseline's 73.3%), and achieving 100% success in a rocket-landing scenario starting from a 4.03% success rate only.
• The results validate the utility of converting initial scores and artifacts into durable, replicable, and inherited evidence, allowing more effective accumulation of research knowledge across experiments.

Praxist addresses a specific weakness in evaluator-grounded autonomous research: systems can generate and score many artifacts without converting those results into durable, operational research knowledge. Its central claim is that long-horizon campaigns should inherit structured evidence rather than merely retain transcripts, scalar scores, or a ranked population of candidate artifacts. The proposed system represents research as a recurrent transformation from artifact to finding, frontier, agenda, and lineage. The paper evaluates this design on all 75 MLE-bench tasks and four open-ended engineering and scientific case studies (2608.25955).

Research problem and central thesis

Existing autonomous research systems commonly organize search around candidate programs, solution trees, or iterative agent memory. This representation is effective when the primary objective is to identify a high-scoring artifact, but it can discard intermediate mechanisms whose value is only revealed through later recombination. A candidate may contain a useful actuator allocation rule, data-processing choice, diagnostic, or control schedule even when its aggregate score is poor. Conversely, a high score may be immature, invalid, or difficult to reproduce. Praxist therefore separates the artifact that was evaluated from the claim that should be inherited.

The paper formalizes the campaign state as four components: a frontier of inheritable evidence, an agenda for the next generation, optional compressed lessons called Gems, and an accumulated lineage trace. Each peer receives only the evidence relevant to its assigned direction. It constructs a reproducible artifact under a design contract, submits that artifact to an external evaluator, and produces typed findings. A synthesis stage then determines which findings become confirmed, candidate, diagnostic, or validation evidence, and converts those decisions into the next agenda.

Figure 1: One generation of Praxist converts inherited evidence into deliberately diverse experiments, evaluated artifacts, typed findings, frontier updates, agendas, and lineage records.

This design makes a strong methodological claim: the reusable unit of autonomous research is not the score-ranked artifact but the artifact-grounded finding. Findings preserve the intervention, outcome, evidence, maturity, limitations, and recommended inheritance action. The resulting system can retain a failed experiment as a diagnostic constraint, a promising but incomplete result as validation work, or a procedural lesson as durable memory. The implication is that search can postpone commitment about which complete artifact will eventually matter, provided that useful components remain recombinable and their provenance is preserved.

Praxist methodology

Design contracts and quantified diversity

Before construction, each peer receives a Deep Innovation Gate (DIG) contract. The contract identifies the mechanism family, intervention surface, parent lineage, research intent, expected evidence signature, forbidden changes, and validation or ablation hook. DIG is read-only: it does not itself generate or evaluate variants. Its purpose is to make the subsequent artifact interpretable as a test of a stated intervention rather than an unconstrained code modification.

Praxist applies Quantified Diversity (QD) at the cohort level. Design cells are defined by mechanism family, intervention surface, and intent. The allocator imposes caps on duplicate cells, mechanism families, parent lineages, and intents, while reserving some capacity for diagnosis and validation. In contrast to conventional quality-diversity optimization, QD is applied to research designs rather than final solutions. The aim is not simply to maintain diverse high-scoring artifacts, but to ensure that each generation covers exploitation, repair, falsification, diagnosis, and exploration.

The implementation distinguishes strict opening-generation constraints from later agenda-level targets. Hard caps govern the initial allocation, whereas later Chair-generated contracts treat coverage dimensions as soft targets. Consequently, diversity is explicitly planned throughout the campaign but is not guaranteed by a hard constraint after the first generation.

Artifact evaluation and evidence maturity

An artifact is the minimal reproducible unit required for inspection or evaluation. Its evaluation record stores the evaluator version, metric semantics, validity state, evidence stage, provenance, and limitations. Praxist explicitly separates score from maturity. Preliminary smoke or scout evaluations may provide useful directional information, but they cannot displace mature evidence on the frontier. This prevents a high score obtained from an incomplete or weakly validated run from being treated as a confirmed result.

Findings are assigned both a research type and an inheritance action. The five types are positive, negative, diagnostic, uncertain, and procedural. The six actions are reuse, validate, avoid, diagnose, preserve, and archive. This distinction is important: a negative finding can be highly valuable if it rules out an assumption, while a positive finding may remain unsuitable for inheritance if its evidence is immature.

The system also promotes failures to first-class evidence. Invalid artifacts, failed evaluations, and unsuccessful interventions can expose evaluator constraints, implementation errors, unstable mechanisms, or boundary conditions. This extends the role of reflection and self-refinement from within-episode feedback to cross-generation research-state management.

PI/Chair synthesis and lineage

After parallel peers publish their findings, Praxist performs cohort-level synthesis. Builder, Skeptic, and Portfolio PI roles independently examine the evidence from complementary perspectives; an optional External-validity role audits reproducibility and claim boundaries. The Chair arbitrates among these reports and emits the next agenda.

The frontier provides the operational status of inherited evidence. Confirmed findings can serve as parents or constraints; candidate findings require further validation; diagnostic findings shape subsequent search; and validation findings receive explicit reproduction or ablation assignments. Agenda dispositions include continue, stop, validate, and explore.

Lineage is accumulated during execution rather than reconstructed after final selection. The released representation uses several correlated ledgers—artifact indexes, finding edges, event trajectories, and per-generation frontier, agenda, and Gem state—with shared identifiers. Typed relations include derived_from, supports, challenges, updates, and related_to. Thus, the lineage records not only what was built but why a direction was retained, rejected, validated, or recombined.

Gems provide bounded cross-generation memory. When enabled, the system compresses recurring mechanisms, rejected assumptions, failure modes, procedural constraints, and evidence boundaries into a small active set. In the reported experiments, Gem compression is enabled only for the quantitative-trading campaign, with a six-generation period and at most four active Gems.

MLE-bench evaluation

The primary comparative evaluation covers all 75 MLE-bench competitions. Praxist uses DeepSeek V4 Pro as its research-agent model, while the baseline is a locally run Claude Code configuration using Claude Opus 4.8. Both arms use the same H100 hardware pool and the same MLE-bench grading harness, but each result is a single full-suite sweep rather than a multi-seed estimate.

Evaluation measure	Claude Code + Opus 4.8	Praxist
Any medal, all tasks	55/75, 73.3%	60/75, 80.0%
Gold medals	34	49
Low-tier medal rate	81.8%	90.9%
Medium-tier medal rate	76.3%	81.6%
High-tier medal rate	53.3%	60.0%
Recorded model spend	US$38,370 | **approximately US$3,054**

Praxist exceeds the baseline in every complexity tier and obtains 49 gold medals, compared with 34 for Claude Code. Its 49 gold outcomes constitute 81.7% of its medals, whereas gold accounts for 61.8% of the baseline’s accepted medals. The paper therefore reports a distinctive threshold effect: Praxist does not merely increase the number of tasks crossing a medal boundary; it more frequently reaches the highest medal tier.

However, the aggregate medal advantage does not imply broad raw-score dominance. On the 70 tasks for which both systems have accepted scores, Claude Code has the better raw metric on 36 tasks, Praxist on 33, and one task is tied. Praxist’s advantage is therefore concentrated in threshold crossings and medal composition rather than in winning most pairwise numerical comparisons. This distinction is central to interpreting the benchmark result.

The comparison is also limited statistically. Each system contributes one locally measured sweep, so the reported medal rates are not estimates with run-to-run confidence intervals. The Praxist ledger additionally involves integrity adjudication: 90,423 attempts were rejected, and clean fallback submissions were substituted on nine tasks. These procedures improve the credibility of the finalized ledger but also make the result dependent on the adjudication protocol.

Open-ended case studies

Rocket landing

The rocket study evaluates deterministic control of a frozen Swordfish C05 six-degree-of-freedom simulator. The plant, integrator, contact model, evaluator, and initial-state banks are fixed; the controller receives exact state feedback. Success is scored at interpolated first landing-leg contact and requires a joint predicate over lateral error, sink velocity, lateral velocity, tilt, angular rates, and remaining propellant.

Figure 2: The rocket benchmark combines fixed nominal and out-of-distribution initial-state banks with a deterministic hybrid guidance, attitude, and actuator-allocation controller.

The final controller combines rolling ZEM/ZEV guidance, a fuel-commit governor, phase guards, terminal-descent control, geometric attitude control, and a closed-form box-constrained allocator that divides pitch and yaw torque between engine gimbals and grid fins. The allocator enumerates clipped KKT candidates rather than solving an iterative optimization problem.

On the matched complete protocol, Praxist achieves 12,288/12,288 successful landings, or 100%, from a starting artifact scoring 4.03%. Weco, an autonomous code optimizer evaluated on the same task and objective, reports 17.12% from the same starting artifact. Praxist therefore reaches a 24.8-fold improvement over the starting controller, while Weco reaches 4.25-fold. The Praxist run records US$196.05 in model spend versus US$1,009.66 for Weco, although the systems use different models, budgets, and search surfaces.

The improvement concerns landing quality rather than first contact: the baseline already achieves first contact on every trajectory. Praxist reduces the 95th-percentile center-of-mass sink speed from 66.3928 to 0.32636 m/s, lateral speed from 1.40592 to 0.04705 m/s, and tilt from 4.56062 to 0.35184 degrees. Fuel depletion falls from 88.6393% of trajectories to zero. The actuator trade-off is explicit: grid-fin total variation decreases by 99.43%, but gimbal total variation increases by 80.43%, and roll-to-pitch/yaw coupling increases by 87.55%.

A post-run audit over all 122,880 rows across the three fixed source banks records 122,878 successes. The two failures are residual lateral-speed violations, not impacts or fuel failures. This audit broadens coverage over the fixed banks but does not establish population-level reliability because the banks were repeatedly reused during search, nominal and near-OOD samples are paired, and hard-OOD states jointly vary radius and velocity. The evaluation also stops at first contact, uses exact state feedback, and excludes disturbances, navigation error, post-contact dynamics, actuator failures, and model mismatch.

The lineage shows that the 100% success rate emerged cumulatively. The fuel-commit governor and guidance changes produced the largest early improvement; attitude retuning raised success to approximately 98.36%; narrow radius, slew, and sink guards closed the remaining failures by generation 9. The allocator was introduced only after success had already reached 100%. Its one-key ablation improves sink and lateral-speed statistics and reduces grid activity, but it does not explain the main success-rate gain. This is a particularly clear demonstration of the paper’s distinction between component attribution and lineage-level accumulation.

Quantitative trading

The quantitative-finance study uses a walk-forward evaluator over 28 quarters from 2019Q1 through 2025Q4. Each policy is retrained from scratch on a trailing 36-month window, separated from the test quarter by a five-trading-day gap. Execution occurs at the next open with 10 basis points of transaction cost per side.

Figure 3: The quantitative-trading evaluator retrains on rolling historical windows and compounds quarter-level returns under explicit execution costs.

The selected policy is an LSTM actor–critic trained with PPO, recurrent rollout replay, supervised warm-up objectives, and a deterministic execution adapter imposing top-$k$, cash, concentration, liquidity, and turnover constraints. It compounds to a 53.07% calendar-time CAGR, compared with 22.80% for the paired all-eligible equal-weight baseline. The policy produces a 1,864.5% cumulative return versus 320.8% for the baseline, is positive in 26 of 28 quarters, and exceeds the baseline in every calendar year of the principal evaluation period.

Additional reported statistics include a 1.56 quarterly zero-rate Sharpe ratio, a worst quarterly-window maximum drawdown of 33.48%, and a 2026 validation return of 21.85% with a 10.97% maximum drawdown. Under an additional 50 basis points per executed side, mean quarterly return decreases from 12.28% to 7.09% but remains positive.

The paper carefully narrows the status of this result. The selected policy is a post hoc metric winner, not the campaign’s promoted artifact. It was evaluated at the first tier, using one seed over 29 cells, and carries three hard constraint violations. The highest confirmed-lane policy completed the full five-seed, 145-cell tier but achieved a lower CAGR. The reported 53% figure therefore demonstrates a strong result under the walk-forward protocol, but not a clean, fully replicated superiority claim.

The lineage also shows incomplete success on the policy’s own repair objectives. A concentration repair increased diversification, yielding a mean effective number of 8.01 names and a maximum mean single-name weight of 21.69%, but it failed preregistered targets of more than ten effective names and a maximum weight below 0.15. The system retained the policy for return despite that failure, illustrating how Praxist can preserve a high-performing but diagnostically imperfect candidate rather than treating the primary metric as a complete research verdict.

SLAM

The SLAM study modifies FAST-LIVO2 with CovSched, a visual-resource policy that controls both when visual updates occur and which map points are admitted. The underlying ESIKF update sequence and residual definitions remain unchanged. CovSched schedules sparse-direct visual updates using a LiDAR translation-observability signal and suppresses map points that are spatially and geometrically redundant.

Figure 4: CovSched inserts observability-aware visual scheduling and geometry-aware map admission into an otherwise tightly coupled LIVO pipeline.

Across 14 NTU-VIRAL sequences, mean APE RMSE is 0.0937 m for FAST-LIVO2 and 0.0501 m for CovSched, with CovSched lower on 13 sequences and effectively tied on one. The paper explicitly declines to interpret this as an accuracy improvement because the two implementations use different pose-timestamp conventions. A same-binary timestamp control changes APE by a factor of 1.74–1.80, and re-association under a common timestamp rule reduces the mean relative difference to -0.09%. The supported conclusion is therefore not that CovSched improves accuracy, but that it reduces visual computation without an observed accuracy cost under the reported protocol.

The evaluator-captured visual-path processing time decreases by 72.4% on average, with a median reduction of 74.4%; every sequence shows a reduction, ranging from 43.2% to 81.6%. A summed LIO-plus-VIO thread-wall proxy decreases by 22.1%. These are not end-to-end latency or CPU-load measurements: timing is asynchronous, skipped frames enter the visual average as zero time, and the instrumentation is host-dependent.

The study is methodologically valuable because its strongest conclusion is negative and bounded. It does not establish which component—frame scheduling or map admission—causes the efficiency result, since the controls are sparse and confounded. Coverage also varies from 0.79 to 1.76 times the baseline’s ground-truth-associated samples. A paired fourteen-sequence rerun with identical source builds and timestamp conventions remains necessary to identify the causal contribution of the mechanism and to make a defensible trajectory-accuracy comparison.

Tokamak magnetic control

The fusion study evaluates 12 bounded voltage residuals added to nominal coil feedforward at 0.5-ms intervals in FreeGSNKE. The benchmark contains five scenarios, three initializations per scenario, and a 100-step horizon. Episodes terminate on hard physics violations such as plasma displacement, current loss, wall contact, coil-limit violations, or solver divergence.

Figure 5: The fusion benchmark evaluates bounded closed-loop coil-voltage corrections under nonlinear plasma dynamics and reports both survival and tracking precision.

The selected HybridJacobianPDV1 controller combines vertical PD control with counter-drain, radial-position control, an ohmic plasma-current loop, waypoint ramping, and sparse Jacobian-sign channels. Relative to a reconstructed MAST-U PCS-style baseline, Praxist survives 1,264 of 1,500 possible steps versus 1,222, and achieves lower common-horizon WNRMSE $p_{95}$, 2.86 versus 2.99. It also outperforms the baseline on common-horizon precision in four perturbed scenarios.

The result reverses under the benchmark’s original full-horizon metric. The PCS-style baseline achieves 4.42 versus 4.65 for Praxist and completes 11 of 15 episodes rather than 10. The paper attributes this disagreement to censoring: controllers that survive longer contribute more late, high-error steps to the full-horizon pool. On the common horizon, Praxist improves aggregate survival by 3.4% and precision by 4.4% relative to the baseline; on the full horizon, the baseline remains better. Neither controller passes the benchmark’s official classification thresholds.

This case study supports the paper’s emphasis on evidence interpretation. Metric choice changes the ordering of controllers, and the authors report both metrics rather than selecting the one favorable to Praxist. The comparison is also internally matched but uses privileged target values supplied by the harness rather than controller estimates derived from noisy probe observations. Moreover, the PCS-style comparator is an architectural reconstruction, not a direct port of the deployed MAST-U controller. The reported result thus establishes an in-simulator trade-off under a privileged-state protocol, not experimental tokamak-control performance.

Scheduling and mature-evidence supply

An appendix evaluates Praxist’s resource-scheduling layer through a synthetic discrete-event simulation involving 4,096,000 policy runs across 348 physically feasible scenarios. The scheduler tracks mature-evidence debt and preferentially launches mature-directed work when the quota is not yet satisfied, while retaining bounded redundancy against failures and heavy-tailed durations.

The mature-evidence debt controller achieves 99.85% quota success, compared with 98.65% for a Boolean maturity signal, 98.36% for a nonredundant thin token, and 27.03% for passive peers without backfill. Hardware utilization is similar across the feedback policies, indicating that utilization alone does not explain the evidence advantage. Under high CPU pressure, the debt controller reaches 99.49% success versus 95.74% for the Boolean policy; under high GPU-memory utilization, it reaches 99.76% versus 95.33%.

These results support the architectural separation between resource utilization and evidence supply. Nevertheless, they are conditional on a synthetic workload generator, linear GPU-capacity assumptions, a processor-sharing CPU model, a fixed plan inventory, and feasibility-filtered scenarios. The simulator was not released with the evidence package, so the scheduling results are design-study evidence rather than independently replayable benchmark results.

Limitations and open questions

The paper’s empirical claims are heterogeneous in evidentiary strength. MLE-bench offers broad task coverage but only one sweep per system and no variance estimate. Its medal advantage is partly a threshold phenomenon, and raw-score head-to-head comparisons favor the baseline on slightly more tasks. The cost comparison is informative but confounded by different model families and serving prices.

The rocket result is numerically strong within its frozen simulator, yet its evaluation distribution is fixed and adaptively reused. Exact state feedback, absent disturbances, first-contact scoring, and unmodeled post-contact dynamics substantially limit external validity. The selected generation-12 artifact was produced after the last committed generation boundary, although the paper reports that the committed generation-11 controller independently also achieved 100% on the complete protocol. The closed-form allocator is locally linearized and decouples pitch and yaw, so its behavior outside the evaluated regime is unresolved.

The quantitative-trading result is particularly vulnerable to selection and maturity concerns. The reported policy is selected post hoc, evaluated at one seed, violates three hard constraints, and is not the campaign’s clean confirmed-lane artifact. The protocol imposes a five-day gap and excludes forward-looking feature names, but the authors did not audit every engineered feature for leakage. The 2026 validation window is encouraging but short relative to the principal backtest.

The SLAM study does not support an accuracy-gain claim because timestamping, pose association, coverage, and accepted-run selection are confounded. The timing statistic is an evaluator-captured visual-path workload, not end-to-end latency. The fusion study relies on privileged target values, an approximate baseline reconstruction, lifecycle-sensitive evaluation, and a metric ordering that changes with the horizon definition. In both domains, factorial ablations are incomplete.

The broad open question is therefore not whether structured inheritance can produce useful artifacts—it can—but which components of Praxist are causally responsible for the gains. The paper does not isolate the contributions of QD allocation, typed frontier lanes, PI/Chair synthesis, Gems, lineage-aware prompting, and resource scheduling through a full factorial comparison. It also leaves open whether the same inheritance representation improves performance when evaluators are slower, noisier, non-deterministic, or only partially informative.

Conclusion

Praxist presents a coherent systems architecture for evaluator-grounded autonomous research in which reproducible artifacts are converted into typed, maturity-aware findings and selectively inherited through frontiers, agendas, Gems, and lineage records. Its strongest benchmark result is 60 medals and 49 golds on 75 MLE-bench tasks, compared with 55 medals and 34 golds for the local Claude Code baseline. The case studies demonstrate substantial within-protocol outcomes—a 100% rocket landing rate, 53% trading CAGR, 72.4% visual-path workload reduction in SLAM, and improved common-horizon fusion control—but also show why each result must be interpreted through its evaluation boundary.

The paper’s most consequential contribution is methodological: it treats failures, validation requirements, claim boundaries, and provenance as active research state. The empirical evidence supports this design as a practical approach to cumulative artifact construction, while leaving the causal value of its individual governance and inheritance mechanisms open for controlled ablation.