Wiki STEM Corpus Overview

• Wiki STEM Corpus is a curated collection of wiki texts dedicated to STEM topics, offering structured content with rich metadata and cross-linking.
• The corpus construction uses automated extraction, markup cleaning, and TF-IDF-based indexing to facilitate robust linguistic normalization and retrieval.
• Advanced entity extraction methods, including SciBERT-based sequence labeling and symbolic techniques, enable effective annotation and knowledge graph construction.

A Wiki STEM Corpus is a curated or structured collection of texts from collaborative wiki platforms (such as Wikipedia or domain-specific wikis) that consist predominantly of content relating to science, technology, engineering, and mathematics (STEM). Recent advances in corpus construction, entity extraction, information retrieval, and semantic annotation have made Wiki STEM corpora essential resources for linguistic analysis, knowledge graph construction, entity linking, and evaluation of information extraction methodologies across disciplines and languages.

1. Foundations and Scope of the Wiki STEM Corpus

The Wiki STEM Corpus concept draws upon the unique properties of wiki-based resources—namely, their semi-structured markup, community-based content curation, rich metadata, and coverage of diverse STEM topics. Wikipedia, as the canonical foundation, offers a vast and continually expanding set of articles, characterized by categorization, consistent template usage, and extensive cross-linking mechanisms. These features support automated text extraction, linguistic normalization, and semantic enrichment.

Corpora may be monolingual or multilingual, as demonstrated by the construction and analysis of Russian and Simple English Wikipedia subcorpora for linguistic indexing and investigation (0808.1753). In recent developments, semi-automatically or fully automatically annotated corpora have been generated that focus on domain-independent representation of scientific entities, as in the STEM-ECR (D'Souza et al., 2020) and STEM-NER-60k (D'Souza, 2022) efforts.

2. Corpus Construction and Automated Indexing

Corpus creation for Wiki STEM resources centers on automated extraction of article text, transformation of markup to plain language, and subsequent linguistic processing. The typical processing pipeline, as detailed by (0808.1753), is modular and composed of the following main components:

• Markup Extraction and Cleaning: Specialized regular expressions or parsing scripts remove or transform wiki markup (templates, links, HTML tags) while preserving visible content, such as link labels.
• Linguistic Normalization: Natural language processing toolkits (e.g., GATE for tokenization and tagging, language-specific lemmatizers integrated via wrappers such as RussianPOSTagger) produce canonical forms of words (lemmas), which underpin indexing and downstream retrieval tasks.
• Index Database Construction: Outputs are stored in inverted file indices with relational schema—tables link document IDs (page IDs), lemmas, frequency counts, and document-lemma associations. Term weights (TF-IDF) are computed as:

$$w(t_i) = TF(t_i) \cdot \text{idf}(t_i), \quad \text{idf}(t_i) = \log \frac{D}{DF(t_i)}$$

where $TF(t_i)$ is the term frequency of $t_i$ in the document, $DF(t_i)$ is the number of documents containing the term, and $D$ is the total document count.

• Language and Edition Comparisons: Corpus statistics and growth rates of different language editions (e.g., Russian Wikipedia (RW) vs. Simple English Wikipedia (SEW)) highlight trade-offs between vocabulary richness and growth dynamics: RW is larger in volume and lexicon but SEW has a higher growth rate and lexicon acquisition (0808.1753).

3. Entity Annotation: Generic and Domain-Specific Models

Recent work has advanced the generic annotation of scientific entities across disciplines. The STEM-ECR v1.0 corpus (D'Souza et al., 2020) and STEM-NER-60k corpus (D'Souza, 2022) formalize scientific entities with four domain-independent categories:

• Process: Natural phenomena or activities (e.g., “growing,” “transfer”)
• Method: Experimental protocols, computational techniques (e.g., “finite-element modelling”)
• Material: Physical or digital objects/entities (e.g., “soil,” “electron”)
• Data: Quantitative/qualitative measurements or derived parameters (e.g., “tensile strength”)

Such a formalism enables domain-independent annotation, supports cross-domain machine learning, and promotes harmonized named entity recognition (NER) architectures. For instance, the STEM-NER-60k corpus utilizes a SciBERT-based sequence labeling system, trained on the expert-annotated STEM-ECR data, to automatically annotate 1M+ entities over 60k abstracts spanning 10 STEM fields (D'Souza, 2022).

A summary of the entity type scheme is provided in Table 1 below.

Entity Type	Description	Example
Process	Phenomena, activities, operations	“polymerization,” “migration”
Method	Techniques, algorithms, procedures	“finite-element simulation”
Material	Objects, species, elements	“basilisk lizard,” “water”
Data	Measurement, result, parameter	“tensile strength,” “pH 6.8”

4. Entity Extraction and Resolution Methodologies

Automated scientific entity extraction employs a range of neural and symbolic methods:

• Neural Sequence Labeling: The benchmark approach leverages domain-specific BERT variants (e.g., SciBERT) to generate contextual token embeddings, further processed by BiLSTM layers, and decoded by CRF sequence taggers (D'Souza et al., 2020, D'Souza, 2022). The AllenNLP platform is commonly used for such implementations.
  • Cross-validation experiments demonstrate macro-F1 scores above 65% for automatic extraction on multidisciplinary datasets, with “Material” as the most reliably identified class.
• Resolution and Disambiguation: Extracted entities undergo a three-step human-guided resolution:
  1. Linkability assessment (deciding eligibility for linking to external knowledge bases)
  2. Multi-word expression splitting (normalization to robust collocations)
  3. Joint encyclopedic linking and lexicographic disambiguation: Each entity is mapped to both Wikipedia (concept-level) and Wiktionary (word sense), yielding the mapping:

  $R = \{(p_i, s_i) \mid e_i \in E,\; 1\leq i\leq N\}$

  with $p_i$ the encyclopedic link and $s_i$ the dictionary sense or “Nil” if not assignable (D'Souza et al., 2020).

Specialized methods for mathematical entity extraction, as illustrated by (Collard et al., 2022), leverage a combination of:

• String matching (OpenTapioca) for high-precision, entity-linkable concepts already present in the knowledge base;
• Neural span-based extractors (DyGIE++) for language-model-based generalization;
• Graph-based keyword ranking (TextRank) for salient phrase identification in noisy text;
• Linguistic candidate generation (Parmenides) for high recall, syntactic normalization via spaCy.

Aggregate evaluation against multiple “silver standards” (author keywords, page titles, automatically extracted noun phrases) provides robustness to reference set incompleteness.

5. Statistical and Linguistic Analysis

Statistical properties of Wiki STEM corpora have been rigorously analyzed:

• Zipf’s Law: Wiki corpora in both Russian and Simple English editions conform to Zipf’s law by producing a linear log-log relation between word frequency and rank, consistent with a $1/x^{a}$ scaling, where the exponent differs slightly between languages and corpus sizes (0808.1753).
• Lexical Entropy: Disciplinary entropy analyses (e.g., Medicine and Chemistry at ~4.58 bits) situate technical vocabulary within broader English, allowing quantitative comparison across fields (D'Souza, 2022).
• Growth Dynamics: Observations such as the Simple English Wikipedia’s 14% higher page growth rate and 7% higher new lexeme acquisition rate (relative to Russian) over a five-month period indicate pace and scale of corpus evolution (0808.1753).

In addition, domain phenomena such as interdisciplinary “word-sense switching” are captured, with terms assuming different meanings across disciplines (for example, “cloud” in Computer Science vs. Astronomy) (D'Souza, 2022).

6. Applications and Open Access Resources

The availability and utility of Wiki STEM corpora are substantially enhanced by open access to both data and tooling:

• Corpora and Tools: Full source code for text extraction, normalization, lemmatization, and index construction is available under GPL for the Wikipedia-based system (0808.1753); modern annotated datasets such as STEM-NER-60k are released under CC-BY compatible terms on public repositories (D'Souza, 2022).
• Use Cases: Wiki STEM corpora enable:
  • Construction and enrichment of scientific knowledge graphs for machine-interpretable publishing (D'Souza, 2022)
  • Enhanced search engines and digital library features using fine-grained, structured indices (0808.1753)
  • Benchmarking and training for domain-independent scientific NER and entity resolution (D'Souza et al., 2020)
  • Evaluation and optimization of mathematical entity extraction across reference corpora (Collard et al., 2022)
• Visualization and Exploration: Multidisciplinary word cloud visualizations and entropy measurements illuminate cross-disciplinary concept distributions and field-specific terminology (D'Souza, 2022).

7. Challenges and Methodological Considerations

Corpus construction and annotation for Wiki STEM resources remain subject to challenges including:

• Markup and Formatting Variability: Differences in wiki syntax and document structure across languages and communities require tailored parsing and cleaning routines (0808.1753).
• Ambiguity and Word Sense Variation: Shared terms with field-specific senses necessitate systematic resolution via encyclopedic and lexicographic linking (D'Souza et al., 2020, D'Souza, 2022).
• Noisy Domain Text: Extraction of mathematical entities is complicated by context-dependent abbreviations, LaTeX interspersed tokens, and inconsistent referentiality (Collard et al., 2022).
• Evaluation: The adoption of multiple overlapping “silver standards” for evaluation increases robustness, reflecting the incomplete gold-standard annotation in many subfields (Collard et al., 2022).

Recent corpora and tools address these limitations by combining expert-annotated gold standards, open extraction algorithms, and modular designs for rapid iteration and cross-disciplinary deployment.

---

The systematic development of Wiki STEM corpora—anchored in open, multilingual wiki resources, generic entity annotation formalisms, neural extraction architectures, and rigorous statistical analysis—underpins advances in scientific information retrieval, entity resolution, and scholarly knowledge graph construction across STEM domains.