- The paper introduces SciDSK, a reusable package that combines dataset metadata, scientific context, file structure, operational guidance, quality checks, provenance, and version-specific identifiers while leaving data in source repositories.
- The evaluation shows registered SciDSKs improved discovery over agentic retrieval of conventional records, raising Hit@1 from 71.15% to 80.77%, Recall@5 from 90.38% to 94.23%, and MRR from 79.04% to 86.41%.
- The paper finds that SciDSK-based interpretation produced more actionable file-level guidance, including checks for continuity, readability, missing files, invalid labels, orphan records, and sampled content, although broader validation is needed across models and repositories.
Motivation and problem statement
AI agents built on LLMs are increasingly expected to locate, understand, and apply scientific datasets autonomously. The authors of "Scientific Data Skills: Enabling Agent-Ready Scientific Data Services at Scale" (2608.19625) argue that current dataset representations are inadequate for this purpose for three reasons: scientific data are fragmented across heterogeneous repositories with incompatible metadata schemas and access mechanisms; dataset-level metadata omits file organization, file-level semantics, and inter-file relationships; and documentation rarely specifies task-specific procedures for preparation, validation, and constrained use. Existing AI-readiness efforts—FAIR principles, RO-Crate, Croissant, data cards, and data-readiness frameworks—are largely declarative: they describe what a dataset contains and how it was produced, but not when it should be selected, how its fields map to scientific concepts, or which checks must precede use. A dataset can therefore be FAIR-compliant and programmatically accessible while remaining unusable by a general-purpose agent without substantial inference from scattered documentation.
The paper's central question is how to represent scientific datasets so that agents can reliably discover, interpret, and invoke them.
The SciDSK representation
The proposed answer is the Scientific Data Skill (SciDSK), which extends the Agent Skill paradigm—from task-oriented capabilities to individual datasets. A SciDSK is a reusable skill package that organizes dataset descriptions, scientific context, file organization, operational guidance, usage constraints, quality checks, and provenance, while the underlying data remain in their original repository. The design follows progressive disclosure: a concise YAML frontmatter serves as the routing signal for discovery, and a Markdown instruction body is loaded only after selection.
The specification has four notable elements:
- Agent-readable frontmatter: standard skill fields (name, description, license, compatibility) plus a nested metadata block recording dataset identity, source location, persistent identifiers, scope, object structure, and snapshot information.
- Instruction body with six components: dataset overview, data access, data fields and structure, usage guidance, quality checks, and provenance/citation.
- Dataset association and versioning: each SciDSK represents exactly one dataset snapshot, identified via
snapshot_id, landing-page URL, and persistent identifier, keeping the skill–dataset relationship explicit and traceable.
- Validation requirements: structural conformance, consistency with source materials, and package integrity checks before publication.
Construction proceeds through a three-stage pipeline—dataset grounding (building an evidence base from authoritative records), specification-guided authoring (mapping evidence into the schema, omitting unsupported claims), and iterative validation/revision. The paper positions SciDSK relative to dataset metadata, dataset cards, agent skills, and tool/MCP interfaces as the only concept covering all of description, scientific context, task knowledge, operational guidance, agent discovery, and dataset invocation.
The Scientific Data Skill Bank
The authors operate an online platform (scidsk.cn) publishing a curated collection spanning six disciplines: physics, chemistry, earth sciences, biology, materials science, and computer science. Each resource passes pre-publication review along four dimensions—source authenticity, representation fidelity, skill safety, and agent compatibility—and receives an independent CSTR persistent identifier, with the associated dataset's CSTR (or DOI) recorded in the frontmatter so that skill and dataset can be cited and traced independently. The platform distributes skill packages only; data access remains at the source repository.
Evaluation
Two evaluations are reported, both using qwen3.6-plus as the underlying model in a controlled MCP-based harness with a strict tool allowlist.
Dataset discovery. A retrieval benchmark uses 72 target datasets across six disciplines, 104 human-verified test queries (after excluding 16 ambiguous ones), and a corpus roughly four times larger than the public collection, where each candidate appears both as a conventional record and as a SciDSK. Results:
| Method |
Hit@1 |
Recall@5 |
MRR |
nDCG@5 |
| BM25-Raw |
47.12 |
69.23 |
57.59 |
59.12 |
| Agent-Raw |
71.15 |
90.38 |
79.04 |
81.90 |
| Agent-SciDSK-Text |
70.19 |
90.38 |
79.01 |
81.92 |
| Agent-SciDSK |
80.77 |
94.23 |
86.41 |
88.40 |
The ablation structure yields two findings. First, most of the gain over BM25 comes from agentic retrieval itself rather than the representation. Second—and more pointedly—treating complete SKILL.md documents as ordinary searchable text (Agent-SciDSK-Text) provides essentially no benefit over conventional records; the improvement arises specifically from registering SciDSKs as routable skills with discipline-level routing and structured search. This attributes the gain to the end-to-end skill mechanism rather than to document content alone, though the authors concede that the contributions of routing, registration, and skill content cannot be fully separated.
Dataset interpretation. Four controlled cases (CT image sequences, GIS rasters with sidecars, image-based tables, cross-file event mapping) were scored against 24 atomic criteria under three evidence conditions. The SciDSK condition satisfied 23/24 criteria versus 22/24 for the ScienceDB landing page. The qualitative differences are sharper than the aggregate suggests: on the CT case, the landing-page condition incorrectly described the visible sequence as "200+ slices," whereas the SciDSK condition correctly identified 196 consecutive slices, distinguished them from the reported total of 1,576 files, and specified concrete continuity, readability, parameter-file, and directory-completeness checks. On the Weibo case, the SciDSK condition identified 4,664 labeled events and required count validation, binary-label checks, orphan detection, and sampled content verification, where the baseline offered only general consistency checks. These cases indicate that packaged operational guidance reduces unsupported inference and produces more actionable pre-use validation—an implication directly relevant to reproducibility of agent-driven analyses.
Limitations and open questions
Several constraints temper these results. The interpretation evaluation covers only four cases with six criteria each, scored partly by deterministic term matching with a single blinded reviewer; discrepancies are reported separately but the quantitative comparison rests on small numbers. The discovery benchmark's corpus is drawn from the same ecosystem as the published skills, raising a distributional-match concern the paper does not test against out-of-domain repositories. The evaluation uses a single underlying model with thinking disabled, leaving model dependence unexamined. Finally, the authors themselves note that the retrieval gains cannot be decomposed into routing, registration, and content effects—a decomposition that would require further ablations. Whether manually curated, evidence-grounded authoring scales to the long tail of scientific datasets without automated construction also remains open.
Conclusion
This paper contributes a concrete specification, construction pipeline, and publication infrastructure for representing scientific datasets as reusable agent skills, together with evidence that registered skill-based workflows improve dataset discovery (Hit@1 rising from 71.15% to 80.77% over agentic retrieval on conventional records) and yield measurably more precise file-level interpretation. The work's main unresolved questions concern attribution of the discovery gains among mechanism components, generalization beyond the curated collection, and scalable authoring.
