Danish Dynaword Corpus
- Danish Dynaword is a Danish-language corpus built under the Dynaword framework, featuring traceable licensing, reproducibility, and community contributions.
- It aggregates 4.8 billion tokens from over 40 diverse, traceably licensed sub-datasets including legal, literary, and web sources.
- The corpus employs a versioned release mechanism with automated CI tests to ensure reproducibility and quality, setting a new standard for evolving datasets.
Searching arXiv for the specified paper and closely related context. Danish Dynaword is a Danish-language corpus developed as the first concrete implementation of the Dynaword framework, a framework for creating large-scale, open datasets that can be continuously updated through community collaboration. It was introduced to address three identified limitations of existing large-scale natural-language datasets: reliance on ambiguously licensed sources, static one-shot releases, and quality-assurance processes restricted to publishing teams rather than community expertise. In this formulation, Danish Dynaword is both a production-scale Danish pre-training corpus and a living testbed for a continuously developed dataset model (Enevoldsen et al., 4 Aug 2025).
1. Definition and conceptual basis
“Dynaword” denotes any dataset following four core principles: traceable and openly licensed; fully reproducible from source code and metadata; well documented via datasheets; and extensible through a formal, low-barrier contribution process (Enevoldsen et al., 4 Aug 2025). Danish Dynaword is identified as the first concrete implementation, designated “v1.2.7,” and serves as a validation of the approach in a Danish setting.
The project is motivated by specific deficiencies in prior Danish corpora. Existing resources, including Danish Gigaword, are described as either restrictively licensed, delivered as one-shot releases, or lacking community-driven updates and transparent quality checks. Danish Dynaword was therefore created to demonstrate that a large-scale, openly licensed, well-documented Danish corpus can be developed and then continuously extended by a broad community of researchers and practitioners (Enevoldsen et al., 4 Aug 2025).
This positioning makes Danish Dynaword more than a static collection of texts. A plausible implication is that the corpus should be understood as an evolving infrastructure artifact whose defining properties include provenance, reproducibility, and governed extensibility, rather than merely aggregate size.
2. Licensing, provenance, and access model
A central property of Danish Dynaword is that every sub-corpus is under a clear, permissive license, with examples including CC-0, CC-BY-SA 4.0, and Apache 2.0, and with explicit provenance recorded for each source (Enevoldsen et al., 4 Aug 2025). The corpus is constructed by aggregating only traceably licensed sources, including public-domain books, EU legal texts, and CC-BY or CC-0 web texts. The stated purpose of this restriction is to avoid ambiguous copyright claims.
The licensing model has direct downstream implications. The data may be downloaded, redistributed, adapted, and combined in research or commercial projects, subject to the obligations of the underlying source license, such as attribution requirements for CC-BY. This design sharply distinguishes Danish Dynaword from datasets whose reuse conditions are uncertain or contested.
The dataset is published on Hugging Face Datasets at https://huggingface.co/datasets/danish-foundation-models/danish-dynaword, and the repository contains the data-collection and preprocessing scripts together with datasheets and tests under /scripts/, /datasheets/, and /tests/ (Enevoldsen et al., 4 Aug 2025). The repository also includes a LICENSE file specifying Apache 2.0 for code and CC-BY-SA 4.0 for documentation, while the data remain under their own source-specific licenses.
3. Scale, composition, and domain coverage
Danish Dynaword has a total size of 4.80 billion tokens, measured in Llama 3 tokenization (Enevoldsen et al., 4 Aug 2025). It is described as containing over four times as many tokens as comparable releases and specifically as being over four times the size of the previous largest openly licensed Danish release, Danish Gigaword at approximately 1 billion tokens.
The corpus is drawn from more than 40 sub-datasets spanning multiple genres and institutional domains. The source categories explicitly listed include legal texts such as EU law, Danish court decisions, and tax guidelines; news and journalism such as TV2 Nord and municipal websites; books and novels including Project Gutenberg and Modern Breakthrough literature; web forums and debate sites; parliamentary and spoken-language transcripts; dialectal corpora such as the Bornholmsk dictionary and Sønderjysk text; and encyclopedic or reference materials including Wikipedia, Wikisource, and Europarl (Enevoldsen et al., 4 Aug 2025).
| Aspect | Description |
|---|---|
| Total size | 4.80 billion tokens |
| Tokenization basis | Llama 3 tokenization |
| Source inventory | 40+ sub-datasets |
The breadth of this composition suggests that Danish Dynaword is intended to function not only as a generic language-model pretraining corpus but also as a heterogeneous record of Danish textual production across legal, literary, dialectal, parliamentary, and web-native registers. The paper directs readers to Appendix Table A1 for a full inventory with sizes and licenses (Enevoldsen et al., 4 Aug 2025).
4. Continuous versioning and reproducible update mechanics
The Dynaword approach is explicitly contrasted with the traditional one-shot dataset model, in which data are scraped, cleaned, packaged, and then left unmodified. In place of that model, Dynaword separates dataset production into three components: a reproducible ingestion pipeline consisting of scripts and datasheets; a lightweight CI test suite covering format, quality, and documentation; and a versioned release mechanism that allows incremental point releases such as v1.2.7 → v1.2.8 → … (Enevoldsen et al., 4 Aug 2025).
The paper does not present a formal equation for the update mechanism, but it refers to the notion of a “substantially equivalent dataset” from the Open-Source AI Definition. The overview offers the paraphrased representation
with the qualification that each contribution must pass the CI tests. All changes are captured in CHANGELOG.md and in Git tags (Enevoldsen et al., 4 Aug 2025).
This update model makes reproducibility a first-class property. Because the dataset is reconstructed from source code and metadata rather than treated as an opaque artifact, version transitions are auditable. A plausible implication is that scholarly use can distinguish between corpus identity at a named release and corpus evolution across releases, which is important for reproducible benchmarking and longitudinal model training.
5. Contribution workflow and governance structure
The formal contribution workflow begins when a contributor identifies a potential new data source or an improvement to an existing one. The contributor then forks the repository, adds a new datasheet under /datasheets/, and writes or updates a download or preprocessing script in /scripts/. A pull request must describe the source, license evidence, estimated size, domain coverage, and minimal quality checks. Maintainers then review license traceability, script reproducibility, and domain fit. Once the tests pass and governance reviewers approve, the pull request is merged and the CI publishes a snapshot release (Enevoldsen et al., 4 Aug 2025).
Governance is overseen by core maintainers at Aarhus University, the University of Copenhagen, Alexandra Institute, and SDU. Community discussion and lightweight triage are conducted through an open Discord and an issue tracker on Hugging Face (Enevoldsen et al., 4 Aug 2025).
The extension guidelines further specify operational norms. Contributors are instructed to begin with a minimal datasheet using /datasheets/template.md, gather legal evidence such as an author death date, a specific CC license page, or direct permission statements, keep preprocessing filters light so that downstream users may apply stricter cleaning, and write one test per new source in /tests/test_new_source.py (Enevoldsen et al., 4 Aug 2025).
This governance model places legal traceability and reproducibility ahead of aggressive filtering. The stated preference for light preprocessing indicates that the project treats source preservation and explicit documentation as more fundamental than imposing a single global normalization policy.
6. Quality assurance, repository organization, and data format
Quality assurance is implemented through a lightweight test suite. Formatting checks require every document to be valid UTF-8, non-empty, and free of control characters such as zero-width characters and nulls. Language and coherence tests include a lightweight language-identification filter ensuring more than 95% Danish tokens and rejecting junk, exemplified by an alpha-ratio < 0.7 threshold. Datasheet validation requires each new source to provide metadata fields including license, URL, description, and token_count (Enevoldsen et al., 4 Aug 2025).
The overview reproduces representative test logic, including functions that assert string typing and non-emptiness and that require langid.classify(record["text"])[0] == "da" (Enevoldsen et al., 4 Aug 2025). CI runs on every pull request, and any test failure raises a blocking error. After merge, the workflow automatically bumps the version in dataset_infos.json and triggers a new Hugging Face release.
The repository structure is organized around reproducibility and inspection:
| Path | Role |
|---|---|
scripts/ |
Ingestion and normalization pipelines |
datasheets/ |
Markdown documentation of provenance, license, and domain |
tests/ |
Pytest suites for formatting, language ID, and metadata completeness |
Additional repository components include dataset_builder.py, described as a Hugging Face-style DatasetBuilder class that ties scripts and tests into a versioned dataset, and CHANGELOG.md, which tracks transitions such as v1.2.7 → v1.2.8 → … (Enevoldsen et al., 4 Aug 2025).
Final published data are served as JSON-lines records, with examples of fields id, source, license, and text. All intermediate steps use plain text or UTF-8 CSV for metadata, and no proprietary binaries are used (Enevoldsen et al., 4 Aug 2025). This file-format choice reinforces the project’s emphasis on transparency and low-friction reuse.
7. Use, significance, and relation to broader dataset practice
The dataset can be loaded with the Hugging Face datasets library either as the latest version or pinned to a specific release such as 1.2.7 (Enevoldsen et al., 4 Aug 2025). The overview also shows examples of basic exploration, including retrieval of document counts, feature inspection, and inspection of the source and text fields, as well as an update workflow involving git pull, checkout of a newer tag, and local rerunning of pytest tests/.
In significance terms, Danish Dynaword is described as a 4.8B-token, completely open-licensed, continuously versioned Danish corpus whose combination of transparent sourcing, community governance, automated testing, and clear extension pathways makes it a blueprint for future “dynawords” in other languages or specialized domains (Enevoldsen et al., 4 Aug 2025). That formulation indicates the intended generality of the framework while grounding it in a concrete Danish implementation.
The principal misconception the project directly counters is that large-scale language resources must either depend on ambiguously licensed web-scale scraping or remain fixed as one-shot releases. Danish Dynaword is presented as evidence that a corpus can instead be openly licensed, reproducible from code and metadata, and incrementally extensible through a maintained contribution process. This suggests a different institutional model for dataset stewardship: one in which corpus publication is not the terminal step of a project, but part of an ongoing, versioned, community-reviewed research infrastructure.