- The paper introduces a containerised, multi-agent system that converts natural-language requests into reproducible Fiji/ImageJ workflows using retrieval-augmented domain knowledge, curated plugin environments, and human oversight.
- The paper demonstrates exact reproduction of a published tracking workflow with 29 tracks and 2,001 spots, while an autonomous apoptosis pipeline achieved 88.89% accuracy, 100% specificity, and 50% sensitivity.
- The paper embeds a quality-assurance agent that audits reproducibility during runtime, but highlights unresolved challenges including limited case-study validation, model stochasticity, sensitivity errors, and dependence on external APIs.
Agentic-J is a containerised, multi-agent AI system that allows biologists to perform reproducible image analysis in Fiji/ImageJ through natural-language instructions. The system addresses a well-documented gap: Fiji and ImageJ host a large, mature plugin ecosystem covering segmentation, tracking, and registration, but assembling these tools into reproducible pipelines requires programming knowledge, plugin-version management, and statistical tooling that few experimentalists command. A substantial share of image.sc forum posts concern environment and plugin-compatibility issues rather than scientific content, and recent benchmarks show that even frontier multimodal LLMs achieve only about 53% accuracy on microscopy-based scientific reasoning tasks. Agentic-J's central claim is that these gaps can be closed not by retraining models, but by combining sandboxed execution, curated plugin dependencies, retrieval-augmented domain grounding, and a human-in-the-loop interface in a single deployable package.
System design and operating philosophy
The system is deliberately positioned as a collaborator rather than an autonomous replacement for the analyst. The Fiji GUI, a conversational chat panel, and the agentic backend share a single browser tab (served over noVNC), so a researcher can request a generated Groovy pipeline, intervene through a plugin dialog, and ask for re-plots or parameter explanations within one session. The agent itself never touches the host operating system: all execution occurs inside a Docker container running as a non-privileged user with Linux capabilities dropped, with filesystem access confined to a dedicated data/ directory. This design responds to evidence that static rule-based guardrails (e.g., Llama-Guard-style pattern matching) can be obfuscated, and that long generated scripts may contain destructive file operations a non-expert cannot audit. Sandboxing is therefore the primary security mechanism rather than a supplementary one.
Architecturally, a LangChain deep agent acting as supervisor constructs a shared "state ledger" recording the plan, metadata, and scientific goal, and delegates to specialised sub-agents: a plugin manager, a coder–debugger pair, an optional Python data analyst, and a quality-assurance agent. The supervisor has no direct access to raw image data; it retrieves file metadata and invokes dedicated file tools, a privacy-relevant design choice. Extensibility is provided through the Model Context Protocol, with napari 3D visualisation as the first external integration.
Domain grounding without retraining
Three complementary knowledge mechanisms address the domain-knowledge deficit of general-purpose LLMs in bioimage analysis. First, a RAG knowledge base stores open-source training materials (Bankhead's textbooks, NEUBIAS resources, BioImageAnalysisNotebooks, plugin documentation) in a hybrid Qdrant collection with sparse and dense embeddings, fused via Reciprocal Rank Fusion. Second, a skills filesystem provides hierarchical, on-demand knowledge: each skill is a directory fronted by a SKILL.md header, letting the agent lift chunk-size constraints without overloading the context window. Third, persistent recipe and error databases accumulate working snippets and documented failure modes across sessions, giving the agent a form of competence that grows with use. The curated container ships the top 30 non-built-in Fiji plugins, selected empirically by citation analysis and ImageJ update-site download statistics (including TrackMate, StarDist, MorphoLibJ, and Labkit), each in an isolated environment — resolving the practical conflict that Cellpose and StarDist cannot coexist in a single installation.
Case study 1: reproducing a published tracking workflow
The first case study reproduces a CellTraxx-based cell-migration analysis from a peer-reviewed publication, using TrackMate through UI-guided instructions. Given a prompt encoding the paper's parameters (threshold 150, LAP tracker with max frame gap 4, linking max distance 60 px), the agent guided the user to exactly the published results: 29 tracks and 2001 spots, matching the original authors. A follow-up swarm plot of mean inter-frame velocities (with physical calibration of 0.645 µm/px and 5 min/frame) closely imitated the published figure. The authors candidly note that with a GPT-5.2-based plotting agent, styling must be prompted explicitly and repeated, since successive interactions may introduce unintended code changes — a concrete limitation of conversational iteration. The full tracking-plus-plotting task took approximately 7 minutes of agent runtime.
Case study 2: autonomous multi-step apoptosis workflow
The second, more demanding case study required the agent to autonomously build a five-stage pipeline for a Leishmania major study: rigid registration of a fixed TUNEL/DAPI image to a live-cell movie via TurboReg, background-corrected measurement of the parasite proliferation index $10 - R/G$, DAPI-based nucleus segmentation with Otsu-thresholded TUNEL calling, parasite-to-nucleus assignment, and statistical analysis with publication-ready plots. The workflow assigned 38 of 45 parasite ROIs and reproduced the expert-observed separation between TUNEL-positive and TUNEL-negative populations (Mann–Whitney p≈0.016; Cliff's delta ≈0.661). Overall classification accuracy was 88.89%, with 100% specificity but only 50% sensitivity — approximately half of the TUNEL-positive parasites could not be assigned to a nucleus.
Notably, the workflow autonomously identified and documented this sensitivity deficit, proposing plausible causes: an overly restrictive assignment distance, Otsu-thresholding inaccuracies, and non-rigid deformations from fixation that rigid registration cannot correct. The authors state plainly that expert-guided refinement remained critical for trustworthy outputs, and that the agent's QC overlays enabled a knowledgeable user to iterate on parameters. This case demonstrates both the feasibility of autonomous pipeline construction and the residual necessity of human judgement on ambiguous edge cases.
Reproducibility auditing as runtime infrastructure
A distinctive contribution is the QA agent, which the authors describe as the first runtime implementation of the community checklists for publishing images and image analyses (Schmied et al., Nature Methods 2024). Invoked automatically and read-only at project completion, it grades each checklist item (minimal/recommended/ideal tiers) as pass, partial, or fail, and emits a prioritised action list. The audit is honest rather than self-congratulatory: in the parasite workflow, the agent's own QA report recorded only 5/6 minimal workflow items and 4/12 minimal image-publishing items passed, with failures on missing scale bars, undocumented brightness/contrast adjustments, and unfilled public-data DOIs. Embedding the checklist in the analysis loop means reproducibility is produced alongside the analysis rather than retrofitted, and the QA mechanism doubles as a dissemination vehicle for community standards. The authors concede it is currently a proof-of-concept, with domain-specific guidelines (e.g., for migration data or generative-AI reporting) still to be integrated.
Limitations and open questions
Several limitations are acknowledged explicitly. The evaluation consists of two case studies derived from published work; no structured usability study with biologists has yet been conducted, and the authors note that benchmark accuracy alone cannot establish whether the system reduces the cognitive burden of translating biological questions into pipelines. They further invoke the "criteria drift" phenomenon: biologists' evaluation criteria emerge only in response to observed outputs, so evaluation protocols cannot be fully specified in advance. A second open question is stochasticity — LLM agents may produce syntactically distinct but semantically equivalent scripts across runs, and quantifying this variability (script-level semantic equivalence, tool-call sequence consistency, downstream biological concordance) is deferred to a planned repeatability study, for which the state ledger and versioned script archive provide instrumentation. Finally, the system depends on external model APIs; fully local deployment awaits open-source models of sufficient coding capability, which would strengthen the security posture for institutions with data-governance constraints.
Conclusion
Agentic-J demonstrates that sandboxed multi-agent orchestration, curated plugin environments, and retrieval-based domain grounding can convert natural-language descriptions into executable, documented, and auditable Fiji workflows without retraining the underlying LLM. The quantitative evidence — exact replication of a published tracking result and an 88.89%-accuracy autonomous apoptosis workflow that flags its own failures — supports the feasibility claim, while the 50% sensitivity result and the unvalidated usability and repeatability questions mark the boundaries of that claim. The architecture's model-agnostic skills filesystem and MCP adapter position it to absorb ongoing changes in the agent ecosystem, but its practical value rests on the planned collaborations and repeatability studies that remain to be carried out.