Datasheets for Datasets

• Datasheets for Datasets are structured documents that detail dataset provenance, curation methods, design choices, and known limitations.
• They enhance transparency, reproducibility, and accountability by clearly documenting dataset collection, processing, and ethical considerations.
• Integration with ML pipelines and automated tools ensures that datasheets support regulatory compliance and end-to-end auditability.

Datasheets for Datasets are standardized, structured documentation artifacts that systematically describe the properties, provenance, design choices, known limitations, ethical considerations, and recommended uses of datasets. Concretely paralleling Model Cards for ML models, datasheets aim to increase dataset transparency, reproducibility, and downstream safety, enabling stakeholders—engineers, auditors, regulators, and end-users—to critically assess the suitability and risks associated with dataset deployment in ML pipelines. The formalization of datasheets responds to the recognition that responsible model development and deployment is data-centric, with upstream data collection, transformation, and validation having cascading impacts on model reliability and fairness. While the foundational Model Card paradigm is centered on model artifacts, recent extensions such as DAG Cards and regulatory frameworks position dataset documentation as an equally critical pillar for comprehensive AI system governance (Tagliabue et al., 2021).

1. Rationale and Motivation for Datasheets

The foundational motivation for datasheets stems from the observation that model-centric documentation is insufficient for end-to-end system characterization, particularly as ML pipelines become increasingly data-driven and modular. Traditional Model Cards treat the train–predict loop atomically, ignoring the provenance, curation logic, and versioning of datasets that fundamentally influence learned model behavior (Tagliabue et al., 2021). This omission creates critical documentation blind spots:

• Traceability gaps: Without explicit dataset documentation, the lineage of input data—collection methods, transformations, postprocessing—is often irretrievable, impeding reproducibility and error diagnosis.
• Accountability deficits: Data-centric risks, such as sampling bias, privacy leakage, or incomplete curation protocols, are obscured unless surfaced in structured forms.
• Regulatory compliance: Modern AI regulation (e.g., EU AI Act, NIST AI RMF) requires explicit documentation of both model and data artifacts; model-centric cards alone are not compliant (Kennedy-Mayo et al., 2024).

Datasheets for datasets evolved in response, extending the model card concept to provide a canonical, version-controlled, machine-readable summary of dataset origin, intended uses, lifecycle events, curation and annotation protocols, and known or suspected limitations (Tagliabue et al., 2021). In contemporary best practice, both models and datasets are documented via parallel “cards”—with datasheets anchoring the data side of the lineage graph and enabling comprehensive evaluation and governance.

2. Canonical Structure and Section Schema

A modern dataset datasheet comprises well-defined sections, each targeting a distinct dimension of dataset transparency and risk management. The following schema is typical:

Section	Key Fields	Example/Instruction
General Information	dataset_name, version, curators, release_date	"technetium-i-ecg", v1.3, owner emails, 2026-03-01
Provenance	collection_protocol, source_institution, license, IRB	"clinical notes extracted, IRB-approved, CC BY-NC-SA 4.0"
Design & Construction	sampling_strategy, timespan, curation filters	"consecutive admissions, Jun–Aug 2019, restrict age 18–99"
Preprocessing	cleaning steps, feature transformations, normalization	"drop notes <50 tokens, anonymize entities, spell-checker v2.1"
Annotation	labeling protocol, annotator expertise, QA steps	"3 RN coders, consensus review, 95% interrater agreement"
Usage Guidance	intended uses, contraindications, risk notes	"for clinical NLP benchmarking, not for direct diagnosis use"
Known Issues	biases, missingness, drift, reidentification risk	"geographic bias to New England; missing pediatric notes"
Versioning	version id, changelog, hash/checksum, release notes	"v1.3 adds cardiac event labels; SHA256: abc...; released 2026-03"
Ethical & Legal	consent, privacy mitigations, data subject rights	"consent waived; fully de-identified; GDPR-compliant"

Datasheets are increasingly instantiated as JSON/YAML artifacts with fixed mandatory fields and rigorous completeness checks (Imanov et al., 27 Jan 2026). A completeness metric—$C = (1/|F|)\sum_{f\in F} I_f$—is often enforced, with release or deployment gated on $C=1$, ensuring no required field is left undocumented.

3. Role in End-to-End ML Documentation and DAG Cards

In data-centric ML pipelines, dataset datasheets are integral to holistic system documentation and provenance graphs. Within the DAG Card framework, each pipeline node’s input artifact (such as a dataset or table) explicitly links to a datasheet, anchoring the node’s lineage (Tagliabue et al., 2021). Formal DAG notation $G=(V,E)$ attaches artifact identifiers $A_\mathrm{in}(v)$, $A_\mathrm{out}(v)$, with each dataset referenced by versioned checksum and datasheet URL.

This linkage enables:

• Automatic lineage aggregation: Downstream DAG Card sections can enumerate all raw and derived datasets used, resolving to their datasheet documentation.
• Cross-stage quality analysis: Data quality checks ($Q(v)$) and drift metrics can be traced to their dataset source and protocol.
• Immutable documentation of pipeline runs: Each pipeline run tracks exact dataset versions and associated datasheets in the versioning tuple $$\mathbf{V}_r = (\mathrm{git\_sha}, \{(v,p)\}, \{A_\mathrm{in}(v),A_\mathrm{out}(v)\})$$.

Without datasheet anchors, reproducibility, governance, and compliance checks (e.g., GDPR, HIPAA) across entire ML lifecycles are rendered impractical.

4. Formalization, Automation, and Best Practice

Datasheet generation and maintenance are increasingly automated, facilitated through –

• Pipeline introspection: Tools such as Metaflow, Airflow, and Kubeflow DSL can extract dataset dependencies, populate datasheet fields from manifest files, and link these to DAG Cards (step parameters, artifact versions, metadata services) (Tagliabue et al., 2021).
• Validation and completeness: Release of datasets is gated by completeness checks on datasheet fields. Missing mandatory entries (e.g., annotation protocol, IRB status) are flagged until satisfied.
• Versioning: Each datasheet version is cryptographically linked (e.g., SHA256 checksum) to dataset artifacts, ensuring immutability and precise traceability in audit contexts.

Best practice recommendations include:

• Embedding dataset datasheets in the same version-controlled repository as ML artifacts.
• Systematic update of datasheets with each change in data collection, curation, or preprocessing protocol.
• Automation of datasheet generation as a byproduct of CI/CD or data ingestion pipelines, preventing drift between deployed datasets and documentation (Tagliabue et al., 2021).

5. Interplay with Model Cards, Regulatory, and Industry Standards

Datasheets for Datasets operate in concert with Model Cards, but with distinct roles:

• Model Cards: Focus on trained model artifacts, benchmarked evaluation, intended use, subgroup performance, and context-driven risk (Mitchell et al., 2018).
• Datasheets: Center on dataset properties, collection protocols, curation processes, and data-centric risks.
• DAG/System Cards: Tie together model and dataset artifacts, encoding the full pipeline as a DAG; dataset datasheet URIs serve as node data anchors (Tagliabue et al., 2021).

Regulatory frameworks (EU AI Act, NIST AI RMF) now require explicit documentation of both models and datasets (Kennedy-Mayo et al., 2024). Machine-readable datasheets are referenced in model cards, risk management templates, and compliance checklists to ensure that data governance and provenance standards are met throughout the development and deployment lifecycle.

6. Extensions: Domain-Specific and Multi-Artifact Documentation

Recent work demonstrates adaptation of the datasheet concept for domain-specific contexts and multi-artifact environments:

• TeMLM-Datasheet: In clinical NLP, TeMLM-Datasheet pairs with TeMLM-Card and TeMLM-Provenance to provide audit-ready, machine-readable documentation of synthetic and real clinical datasets. Each model’s performance claim, calibration test, and evaluation must be linked to a specific datasheet version with documented de-identification and annotation strategies (Imanov et al., 27 Jan 2026).
• Digital/Web Forensics: Domain-specialized extensions include dataset datasheets integrating chain-of-custody, bias, and error taxonomies driven by controlled vocabularies (Maio, 19 Dec 2025).

Datasheets are being further integrated with system-level documentation artifacts (system cards, DAG cards), automated provenance graphs, and sustainability metrics, supporting modular, extensible, and standardized audit trails across disparate domains.

7. Open Issues and Future Directions

Open research challenges for Datasheets for Datasets include:

• Standardization of schemas: No single schema is universally adopted; consensus is emerging, but divergence remains in field requirements across domains (Imanov et al., 27 Jan 2026).
• Automated, trustworthy generation: Ensuring datasheet entries are not only complete but verifiable remains an open technical problem.
• Integration with ML property card attestation: Hardware-assisted attestation techniques, as in the Laminator framework, enable binding of card fields (including datasheet attributes) to cryptographically verifiable measurements, supporting trusted documentation and audit in regulatory contexts (Duddu et al., 2024).
• Streaming and dynamic datasets: For online, evolving datasets used in continual learning or real-time pipelines, best practice for datasheet versioning, update frequency, and immutability remains underdeveloped.

The trajectory is toward machine-checkable, immutable, and pipeline-integrated datasheets as first-class artifacts, critical for traceable, reproducible, and compliant data-centric AI deployments.