SciVisAgentSkills: Design and Evaluation of Agent Skills for Scientific Data Analysis and Visualization

Abstract: Recent advances in agentic visualization have enabled the translation of natural language into executable scientific visualization (SciVis) workflows. While general-purpose coding agents show strong capabilities, they often lack the tool-specific expertise required for SciVis tasks. In this work, we present SciVisAgentSkills, a collection of reusable agent skills that augment coding agents for scientific data analysis and visualization by encoding environment assumptions, tool usage patterns, and domain heuristics across scientific tools such as ParaView, napari, VMD, and TTK. We evaluate these skills on Codex and Claude Code using SciVisAgentBench, a benchmark of 108 expert-designed multi-step tasks. Results show that agent skills improve mean task scores across the evaluated suites, with token-efficiency benefits that depend on the agent harness and tool setting. These findings highlight the importance of structured procedural knowledge for enabling reliable, long-horizon SciVis workflows, while also showing that skills should be studied alongside the execution harness that loads and applies them. The skills are available at https://github.com/KuangshiAi/SciVisAgentSkills.

• The paper introduces a modular agent skills framework that enhances scientific visualization by providing procedural scaffolding, with notable gains such as a 60% improvement in topology tasks.
• The paper details a structured methodology, including environment pinning and documentation distillation, to construct robust and portable agent skills.
• The paper’s evaluation using SciVisAgentBench demonstrates improved task performance, token efficiency, and consistency in multi-step scientific data analysis.

SciVisAgentSkills: Structured Agent Skills for Scientific Data Analysis and Visualization

Motivation and Problem Context

The proliferation of agentic systems for scientific data analysis and visualization (SciVis) is driven by the rapid advancement of multimodal LLMs (MLLMs) and their integration with software tools via protocols such as the Model Context Protocol (MCP). While general-purpose coding agents now demonstrate competence in executing visualization workflows from natural language prompts, they remain bottlenecked by the lack of robust, tool-specific procedural knowledge, often leading to inefficient multi-turn probing, API misuse, or failed rendering outputs. The procedural depth, tool heterogeneity, and long-horizon sequence characteristics of SciVis tasks (spanning domains from volumetric rendering to topology and molecular visualization) further stress the limitations of generic agent architectures.

The work addressed in "SciVisAgentSkills: Design and Evaluation of Agent Skills for Scientific Data Analysis and Visualization" (2606.05525) systematizes a solution grounded in the concept of agent skills: reusable, modular packages of domain and tool-specific procedural knowledge that can be dynamically loaded by coding agents at inference time, providing critical scaffolding for robust, efficient, and portable scientific workflows.

Design and Implementation of SciVisAgentSkills

The SciVisAgentSkills suite targets four representative SciVis tools—ParaView, napari, VMD, and TTK—selected to span a broad range of scientific environments, data modalities, and workflow complexities. Each skill is operationalized as a self-contained, version-stable module, constructed by:

• Fixing environment parameters, including precise tool versions, dependencies, and execution contexts, to mitigate spurious configuration failures.
• Distilling official documentation and encoding execution patterns, including scripting idioms, recommended pipelines, and runtime constraints (e.g., headless operation and viewport selection for image capture).
• Incorporating function exemplars, error-handling logic, and empirical refinements derived from multistage agent-agent and human-in-the-loop debugging cycles.
• Ensuring tool-generalization by strictly excluding benchmark-specific or solution-leaking content; all procedural guidance derives from public documentation or canonical agent implementations.

The skills employ a YAML frontmatter for metadata (version, tags, tool), concise Markdown with structured usage rules, code templates, API summaries, troubleshooting guides, and where necessary, auxiliary reference files for tools with large API surfaces.

Evaluation Protocol and Benchmark Suite

Effectiveness of SciVisAgentSkills is validated on SciVisAgentBench [ai2026scivisagentbench], a comprehensive benchmark comprising 108 expert-crafted, multi-step SciVis tasks, classified into five suites: ParaView visualization, molecular visualization (VMD), bioimage visualization (napari), topology visualization (TTK), and object identification from volumetric evidence. Each suite is directly aligned with a corresponding agent skill.

Figure 1: Representative scientific data analysis and visualization tasks from SciVisAgentBench highlight the breadth and complexity modeled.

Evaluation employs two widely deployed coding agents—OpenAI Codex and Anthropic Claude Code—tested both with and without the skills applied. Task completion and correctness are independently adjudicated via multimodal LLM judges, image similarity metrics (PSNR, SSIM, LPIPS), code validators, and deterministic outcome rules, ensuring a robust, outcome-centric assessment that avoids unreliability associated with trajectory-level (intermediate step) scoring.

Performance is reported as mean task scores (primary aggregate), completion rates, and consistency measures (pass@k and pass^k over multiple executions), alongside token efficiency analysis to probe agent-skill-harness interplay.

Figure 2: pass@$k$ and pass$^k$ metrics reveal the improvement in run-to-run consistency and success rates with agent skills across task suites.

Empirical Results and Analysis

Performance Gains: Across all five SciVisAgentBench task suites, inclusion of agent skills increases mean task scores for both Claude Code and Codex. The magnitude varies: the largest relative gain is observed in topology visualization for Claude Code (~60%), affirming the hypothesis that tasks involving niche or poorly documented APIs benefit most from explicit procedural scaffolding. Scores for molecular visualization (VMD) see minimal boost for Claude Code, suggesting that strong LLMs may have already internalized mainstream tool workflows.

Completion Rate and Consistency: Notably, while mean scores invariably improve, changes in completion rates are sometimes non-monotonic—e.g., Codex on object identification achieves higher scores but lower completion rates with skills, indicating that while skills can unlock more sophisticated plans or outputs, they might introduce complex branches that occasionally fail to execute end-to-end.

Token Efficiency: Skill effects on token usage are model- and tool-dependent. Claude Code benefits from reduced output tokens and, in most settings, fewer input tokens; Codex displays less predictable trends, even using more tokens in some cases. There is no strong correlation between token usage and achieved benchmark scores; the efficiency gains appear to derive from improved context management and targeted skill application within the agent harness, rather than skill verbosity per se.

Discussion and Theoretical Implications

Agent Skills as Procedural Knowledge Distillation: The construction methodology—environment pinning, documentation distillation, empirical correction—can be interpreted as a domain-agnostic protocol for capturing best practices in scientific software, externalizing them from ephemeral agent trajectories to reusable, shareable artifacts. This amplifies both reliability and community reuse.

Limitations and Decoupling Factors: Skills provide greatest leverage where foundation models’ training distributions are skewed or where actor harnesses lack fine-grained context retention. However, there are notable diminishing returns for widely known, well-documented tools. Additionally, the token cost of skill integration is emergent from the interplay of agent, harness, and skill design, not predictable from skill design alone.

Harness-Oriented Co-Design: The study underscores that durable agent performance on long-horizon scientific tasks demands co-optimization of skills (knowledge/content layer) and harnesses (execution/context management layer). Persistent tracking, orchestrator submodules, and memory—studied independently in recent AI VIS co-scientist systems—must be jointly designed with procedural knowledge artifacts to realize stable and reproducible agentic workflows for scientific analysis [miao2026toward].

Portability and Collaborative Potential: That the four tested skills generalize across ParaView, napari, VMD, and TTK—tools with disparate user bases and workflows—highlights the portability of the agent skill protocol. The authors explicitly call for community-driven expansion of both skills and benchmarks, opening a pathway for broader standardization and evaluation.

Practical and Future Directions

The findings validate that agent skills are a practical, modular means to improve coding-agent reliability and efficiency in complex, real-world scientific environments—a low-cost extensibility avenue that does not require retraining or fine-tuning LLMs. However, further research is warranted to:

• Systematically benchmark skill effectiveness with alternate interaction paradigms (CLI, API, MCP), isolating the impact of skill format versus interface structure.
• Expand the skill ecosystem to more tools, data modalities, and less-charted domains (e.g., high-dimensional tensor visualization, climate informatics).
• Develop dynamic harnesses capable of adaptive skill loading/unloading, progressive context curation, and shared long-term memory for persistent agents.
• Investigate the security and trust implications of skill-based extensibility, as provenance and provenance leakage become pertinent for scientific reproducibility and reliability.

Conclusion

SciVisAgentSkills demonstrates that judiciously designed agent skills—encapsulating domain-specific procedural knowledge in modular retainers—significantly enhance the capability of general-purpose coding agents to execute, explain, and troubleshoot long-horizon scientific data analysis and visualization workflows. These improvements are robust across task types and agent architectures, provided the underlying execution harness judiciously manages skill invocation and context. The methodology, results, and released skills lay a foundation for a standardized, collaborative approach to procedural knowledge management for agentic scientific computing (2606.05525).