It's not the Language Model, it's the Tool: Deterministic Mediation for Scientific Workflows

Abstract: LLMs can produce convincing scientific analyses, but repeated generations on the same data do not guarantee the same result. A researcher may regenerate an identical query and receive a different fit, a different peak position or a different analysis procedure, without an obvious way to decide which output to trust. We propose typed mediation, a pattern in which the model orchestrates deterministic tools rather than generating analytical code. Each tool encodes one researcher's exact procedure for one instrument, ported through structured interviews. The model selects which tool to call and with what parameters. The tool produces the result. Regeneration does not change it. We evaluate this claim by running the same photoluminescence analysis on four platforms, including three commercial foundation models, four times each with the same prompt. The typed tool produces identical results across all runs. The commercial platforms either vary in numerical output and analytical methodology across runs, or fail to produce valid results on the task. We deploy this pattern on two instruments serving users over approximately six months, with very positive user feedback. Both cases are very challenging: they involve proprietary binary formats and per-seat licensed software, which force the tool to remain on local infrastructure alongside the data and the instrument it operates. We argue that deployment topology is not just a preference, but a structural requirement of scientific tool mediation. The result is a practical pattern for deploying LLMs in scientific workflows where reproducibility is mandatory, reducing analysis time from weeks to minutes while guaranteeing identical outputs across runs.

• The paper presents typed mediation which restricts LLMs to deterministic, schema-constrained tool orchestration, ensuring reproducibility of scientific workflows.
• It demonstrates successful deployment in photoluminescence spectroscopy and SEM analysis, achieving invariant analytical outputs and improved privacy.
• Empirical evaluation shows drastically reduced output variance and sub-minute execution latency compared to stochastic LLM-driven approaches.

Deterministic Mediation for Scientific Workflows: Enabling Reproducibility and Privacy via Typed Tool Orchestration

Motivation and Limitations of Stochastic LLM-Driven Analysis

Contemporary applications of LLMs in scientific data analysis face an intractable reproducibility barrier: repeated invocations of the same model on identical data can yield divergent analytical results due to inherent stochasticity in prompt interpretation, code generation, and workflow reconstruction. This is antithetical to the foundational requirement in scientific methodology that results must be verifiable and consistent under the same analytical conditions. Moreover, the privacy risks associated with transmitting proprietary or sensitive laboratory data via cloud-hosted AI systems compound the practical limitations of current LLM-driven approaches. Proprietary data formats, per-seat software licenses, and policy constraints further complicate the feasibility of cloud-based orchestration for many laboratory environments.

Typed Mediation: Architectural Pattern and Formal Properties

The proposed typed mediation paradigm reframes the role of LLMs in scientific workflows. Rather than permitting generative models to enact data analysis directly—often through code synthesis or workflow reconstruction—typed mediation restricts the model to the orchestration of deterministic, schema-constrained tools. These "typed tools" encapsulate a workflow specification derived via structured interviews with the domain expert responsible for the original analytical method. The typed schema, exposed through the Model Context Protocol (MCP), strictly defines valid parameters and invocation semantics. Thus, the tool instance executes the analysis deterministically, ensuring that repeated executions, regardless of which model orchestrates them, produce invariant results given fixed inputs and validated schema conformance.

This architecture enforces a strict separation of concerns:

• The LLM (mediator) is responsible only for interpreting user intent and mapping it to parameterized calls as specified by the skill configuration and schema.
• The typed tool, anchored to the instrument's local infrastructure, is the sole locus of analytical computation, guaranteeing determinism, privacy, and conformance to the researcher's validated workflow.

Deployment and Methodology: FORTHought Platform in Practice

The FORTHought platform operationalizes typed mediation and has been deployed over six months in an institutional setting with eleven active users. The implementation process for each workflow follows a reproducible methodology:

1. Structured Interview for Workflow Extraction: Detailed technical interviews with domain experts yield granular workflow specifications, resolving all methodological choices (e.g., fit model, spectral window adaptation, quality thresholds, parameter extraction logic). Discrepancies between assumed and actual procedure are iteratively corrected and validated.
2. Tool Implementation and Schema Definition: Each workflow specification is ported into a typed tool, which integrates directly with proprietary analysis software on local workstations. The MCP-exposed schema strictly restricts the tool's interface, enforcing deterministic execution and robust parameter validation.
3. Skill Document Configuration: Per-instrument skill files encode orchestration policies, tool access constraints, default arguments, and error-handling routines, further constraining LLM-mediated tool invocation and result presentation.

Figure 2: Typed mediation pipeline for photoluminescence spectroscopy, demonstrating end-to-end schema validation, deterministic execution, and representative outputs.

Two deployment cases are highlighted: photoluminescence (PL) spectroscopy and periodicity analysis in scanning electron microscopy (SEM).

Photoluminescence Spectroscopy Pipeline

The PL pipeline involved porting a multi-stage, proprietary-file-dependent analysis—previously consuming ~2 weeks per measurement campaign—into a deterministic tool accessible through natural language mediation. Workflow extraction via a 30-question structured interview revealed critical departures from default or naive analytical choices (e.g., Voigt vs. Lorentzian profiles, adaptive fitting windows, split fitting regimes). The resulting implementation encapsulates all tacit expert decisions into a canonical tool, yielding sub-0.02 discrepancies in derived exponents relative to manual gold standards and consistent outputs across all runs and users.

SEM Periodicity Analysis

The SEM case demonstrated generality across instrument classes, encoding device-specific calibration mappings and analysis logic as a deterministic, schema-typed tool. User-facing mediation via LLMs remained robust to model choice, as long as the schema was enforced.

Resource and Validation Considerations

Interview time rarely exceeded 90 minutes per workflow. Tool coding per implementation was approximately 1 hour, with most time expended on iterative validation against user corrections. Skill file authoring, while trivial in content, was critical in preventing plausible but incorrect LLM behavior.

Empirical Evaluation: Reproducibility Across Platforms

A controlled evaluation was performed using a fixed PL dataset and prompt, executed on four independent platforms (FORTHought, GPT-5.5 Extended Thinking, Claude Sonnet 4.6, and Gemini 3.1 Pro), each run four times with identical configurations.

Key results:

• FORTHought achieved perfect run-to-run reproducibility ($\sigma_b = 0$ for the intensity exponent), byte-identical output files, and sub-minute execution latency.
• GPT-5.5 and Claude Sonnet 4.6 delivered valid—but irreproducible—outputs, varying not only in numerical fit parameters (up to 54.8% spread in peak intensities for GPT-5.5), but in the underlying analytical procedures, including dynamic changes in emission peak selection and background subtraction methodology.
• Gemini 3.1 Pro consistently failed to deliver valid results due to missed preprocessing, centering fits on incorrect spectral regions.

Figure 4: Reproducibility evaluation of power-law fit exponent $b$ and completion time across four platforms and four runs; typed mediation exhibits perfect determinism and superior execution time.

The empirical evidence supports the claim that platform determinism is entirely governed by analysis localization within the tool; neither foundation model choice nor prompt stochasticity influence result reproducibility when orchestration is schema-gated.

Theoretical and Practical Implications

The pattern enforces a strict zero-variance analytical regime, critical for scientific contexts where silent methodological drift renders results unverifiable. Typed mediation requires that the operational locus of analysis—due to file formats, device calibrations, and license enforcement—remains on local infrastructure, unmediated by cloud-hosted LLM environments. This topology is not merely optimal, but a necessary precondition for privacy and license compliance in most laboratory settings.

Weakly constrained skill files or schema definitions are empirically shown to reintroduce LLM-induced variance, supporting related findings on schema-gated orchestration in scientific workflows (Strickland et al., 6 Mar 2026, Xu et al., 17 Feb 2026). Typed mediation thus defines a spectrum: as schema tightness increases, mediation variance ($\sigma^2_{mediation}$) asymptotes to zero, and one achieves perfect reproducibility both in workflow and result. This formalization aligns with the overarching requirement for scientific automation: decoupling reasoning variance from computational determinism.

Future Developments

The typed mediation pattern is most efficacious in domains characterized by complex, multi-stage analytical workflows with critical dependence on methodological fidelity. As MCP adoption expands (Pan et al., 25 Aug 2025) and tool specification languages mature, broader classes of laboratory workflows can be ported with reduced interview and validation overhead. Partial automation of workflow extraction and cross-institutional tool profilation are promising research directions. Longitudinal user studies assessing error rates and productivity impacts in a wider array of scientific contexts will further elucidate the generalizability of typed mediation.

Conclusion

Typed mediation offers a robust, practically validated solution for achieving deterministic, privacy-preserving, and reproducible scientific workflows mediated by LLMs. The critical insight is that analysis determinism is ensured not by model capability but by architectural pattern: local, researcher-encoded, schema-typed tools mediated—but not computed—by LLMs. This shift in locus from model to tool represents a structurally necessary refinement for scientific applications requiring strict reproducibility, especially within restrictive licensing and privacy regimes.

Figure 1: Schematic summary of typed mediation for PL spectroscopy and SEM periodicity analysis; both cases illustrate identical analytical outputs across runs and strict localization to instrument infrastructure.