Chinese Minority Headline Generation

Updated 10 July 2026

CMHG is a headline generation task for Chinese minority languages characterized by severe corpus scarcity and unique script preprocessing needs.
It leverages a quality-centric corpus (MC²) built from web-scale data with rigorous cleaning, deduplication, and language purity filtering to address contamination issues.
Benchmark evaluations compare encoder-decoder and few-shot models using ROUGE-L F1 scores, highlighting the trade-offs in handling low-resource, script-specific headline generation.

Chinese Minority Headline Generation (CMHG) denotes headline generation for minority languages in China, particularly Tibetan, Uyghur, and Mongolian, under conditions of severe corpus scarcity, script-specific preprocessing requirements, and high risk of language contamination in web-scale data. In the literature provided here, CMHG appears both as a downstream generative use case enabled by document-level minority-language corpora and as a named dataset and benchmark for supervised headline generation. The topic sits at the intersection of low-resource summarization, multilingual pre-training, script-aware corpus construction, and reliability assessment for generated headlines (Zhang et al., 2023).

1. Definition and scope

CMHG is introduced explicitly as a headline generation task for minority languages in China in the context of the CMHG dataset and benchmark, which includes Tibetan, Uyghur, and Mongolian and is designed specifically for headline generation tasks (Xu et al., 12 Sep 2025). The underlying problem formulation is the standard document-to-headline mapping: the source is the main text of a webpage or news article, and the target is the title or headline associated with that content.

Within the broader research trajectory, CMHG is also framed as a generative task that requires document-level understanding and script-sensitive processing. MC $^2$ states that the corpus was constructed with a focus on enabling headline/text generation tasks such as CMHG, but it also states that no direct text generation or headline generation experiments with MC $^2$ are described in that paper (Zhang et al., 2023). This distinction is central: MC $^2$ is foundational infrastructure, whereas CMHG, as a benchmarked task, emerges in later work.

A recurring theme is that Chinese minority headline generation differs materially from high-resource headline generation because the relevant languages use writing systems that have often been neglected in corpus construction. The cited work highlights Kazakh Arabic script and Traditional Mongolian script as examples of scripts long neglected in favor of higher-resource variants, and the CMHG benchmark further emphasizes that minority languages in China face challenges due to unique writing systems that differ from international standards (Zhang et al., 2023).

2. Corpus foundations and data pipeline

The immediate corpus substrate for CMHG is MC $^2$ , described as the largest open-source corpus of its kind so far for four underrepresented Chinese minority languages: Tibetan, Uyghur, Kazakh, and Mongolian (Zhang et al., 2023). Its post-processing sizes are 2.22 GB for Tibetan, 736 MB for Uyghur in Arabic script, 937 MB for Kazakh in Arabic script, and 874 MB for Traditional Mongolian. The corpus has a document-level focus, with average document length reported as 807 tokens, and a domain distribution of News (53.1%), Culture (17.9%), Gov (16.3%), Wiki (8.1%), and Other (4.7%).

The collection methodology is explicitly quality-centric. MC $^2$ uses a website-centric approach built on manually curated lists of high-quality websites for each target language, with the aim of ensuring that the site’s entire content is in the target language. This design is motivated by contamination problems in existing corpora; one cited example is that in CulturaX, 16% of “Uyghur” texts were actually Kazakh Arabic. Extraction is AI-assisted and rule-based: AI such as GitHub Copilot analyzes HTML structure, and site-specific Python scripts extract only the main content and title while discarding headers, footers, and sidebars. Deduplication is multi-level, using duplicate URL removal, exact deduplication via SHA-256 hashing, and fuzzy deduplication with MinHash using $n=5$ for 5-grams, $b=450$ , and $r=20$ . Filtering removes documents with high repetition, too short length, or too few target-script characters, with custom thresholds per language and script. Private information such as emails, phones, and ID numbers is replaced by special tokens such as [email] (Zhang et al., 2023).

These properties are directly relevant to CMHG because headline generation is unusually sensitive to article-title alignment and to contamination in surrounding webpage material. A plausible implication is that the removal of headers, footers, and sidebars is not only a corpus-cleaning step but also a safeguard against spurious title-content associations in downstream headline generation.

3. Benchmark dataset and linguistic coverage

The CMHG dataset is presented as a dataset and benchmark for headline generation of minority languages in China, with three supported languages: Tibetan, Uyghur, and Mongolian (Xu et al., 12 Sep 2025). The raw dataset sizes are 100,000 entries for Tibetan and 50,000 entries each for Uyghur and Mongolian. The annotated high-quality benchmark contains 2,901 Tibetan samples, 2,950 Uyghur samples, and 2,931 Mongolian samples.

Language	Raw	Annotated
Tibetan	100,000	2,901
Mongolian	50,000	2,931
Uyghur	50,000	2,950

The domain composition is uneven across languages. Tibetan contains 66% government documents and 34% news; Mongolian contains 100% government documents; Uyghur contains 85% government documents and 15% news (Xu et al., 12 Sep 2025). This matters for interpreting benchmark results because headline style and lexical compression differ across administrative and journalistic registers. A plausible implication is that cross-language comparability within CMHG is constrained not only by script and morphology but also by domain imbalance.

Collection uses web crawling, where webpage titles serve as headlines and main text serves as source content. Cleaning includes removal of ads, navigation, and media, along with deduplication and text normalization over character encoding, whitespace, and format. For annotation sets, language purity filtering is regex-based, while the raw training data tolerates light noise for robustness (Xu et al., 12 Sep 2025). This split between strict benchmark purity and noisier training material defines the benchmark’s methodological stance: evaluation data are quality-controlled by humans, whereas training data preserve some web realism.

4. Modeling approaches

The methodological baseline for headline generation in the supplied literature begins with neural headline generation under encoder-decoder architectures. "Neural Headline Generation with Sentence-wise Optimization" models the conditional probability of a headline $\mathbf{y}$ given a document $\mathbf{x}$ as

$^2$ 0

Its core contribution is minimum risk training (MRT), which replaces word-level maximum likelihood optimization with sentence-level optimization against evaluation metrics. The approximated MRT objective is

$^2$ 1

with $^2$ 2 set to $^2$ 3. For Chinese headline generation on LCSTS, the paper uses ROUGE-1, ROUGE-2, and ROUGE-L F1 and reports that NHG (MRT) reaches 38.2, 25.2, and 35.4 respectively, compared with 34.9, 23.3, and 32.7 for NHG (MLE) (Ayana et al., 2016). Although this work is not about minority languages, it establishes a sentence-level optimization paradigm directly relevant to CMHG.

For Chinese minority languages specifically, CINO is a multilingual encoder-only pre-trained LLM covering Standard Chinese, Yue Chinese, Tibetan, Mongolian, Uyghur, Kazakh, Zhuang, and Korean (Yang et al., 2022). It uses a vocabulary extended with tokens for Tibetan and Mongolian and prunes unused tokens, resulting in a final vocabulary of about 135k. Its fast MLM objective restricts masked prediction to language-specific tokens:

$^2$ 4

Because CINO is encoder-only, the cited material describes its use for headline generation through encoder-decoder adaptation or extractive formulations rather than as a native generator.

A later generation-oriented framework, XLM-SWCM, directly targets extremely low-resource text generation by reusing encoder weights in the decoder (Su et al., 15 Feb 2025). It uses CINO as the encoder and constructs a decoder from two layer types: NormalDecoderLayer, which is randomly initialized, and CustomDecoderLayer, which inherits weights from the encoder’s self-attention and feed-forward blocks. After every $^2$ 5 CustomDecoderLayers, a NormalDecoderLayer is inserted; the best performance is reported with $^2$ 6. Pretraining uses denoising auto-encoding and machine translation, and language balancing follows

$^2$ 7

with $^2$ 8 set to 0.3. This framework is applied to Tibetan, Uyghur, Kazakh, and Mongolian.

5. Evaluation protocol and benchmark results

CMHG evaluates systems with ROUGE-L F1 Score as the standard metric for headline or summary overlap (Xu et al., 12 Sep 2025). The benchmark compares fine-tuned smaller models and few-shot LLMs. The fine-tuned models are cino-cum, described as an encoder-decoder based on XLM-R tailored for minority languages, and swcm, described as a shared-weights encoder-decoder. Both are fine-tuned on CMHG’s raw, unannotated data and evaluated on the annotated benchmark set. The few-shot models are Qwen2.5-72B and LLaMA3.1-70B, both using 2-shot prompting.

Model	Size	ROUGE-L F1
cino-cum	411M	bo 0.20 / mn 0.12 / ug 0.09
swcm	457M	bo 0.23 / mn 0.18 / ug 0.15
Qwen2.5	72B	bo 0.24 / mn 0.32 / ug 0.29
LLaMA3.1	70B	bo 0.34 / mn 0.30 / ug 0.35

On the high-quality 500-sample subset, the corresponding ROUGE-L F1 values are 0.21, 0.13, and 0.10 for cino-cum; 0.23, 0.17, and 0.14 for swcm; 0.24, 0.29, and 0.34 for Qwen2.5; and 0.34, 0.31, and 0.34 for LLaMA3.1, in Tibetan, Mongolian, and Uyghur respectively (Xu et al., 12 Sep 2025). Fine-tuning uses 50 epochs with AdamW, $^2$ 9, $^2$ 0, and a learning rate from $^2$ 1 during warm-up to $^2$ 2.

The related XLM-SWCM study reports a different headline-generation evaluation setting centered on Tibetan and cross-lingual transfer. For Tibetan headline generation, ROUGE-L F1 is 25.7 for XLM-SWCM, 16.1 for MC2-LLaMA-13B, and 8.6 for mBART-CM (Su et al., 15 Feb 2025). The same study states that XLM-SWCM outperforms strong baselines on headline generation across Tibetan, Uyghur, Mongolian, and Kazakh in few-shot cross-lingual transfer, and that unlike mBART-CM and MC2-LLaMA, it correctly produces output in the target minority language rather than Chinese. Because these results come from a different dataset configuration than CMHG, they are best interpreted as evidence about modeling strategy rather than directly comparable benchmark scores.

6. Annotation, reliability, and error analysis

The CMHG benchmark’s high-quality test set is produced through native-speaker annotation. For each language, 3,000 entries are randomly sampled, and two annotators assess alignment between the article title and content (Xu et al., 12 Sep 2025). The criteria are whether the article is complete, whether the headline has spelling, grammar, or context errors, and whether the headline is irrelevant to the content. If no major issue is found, the sample is labeled “Normal.” Matching is scored on a 1–7 scale, where 1 is “Completely Mismatched” and 7 is “Fully Matched.”

Agreement control is explicit. Scores differing by more than 2 points from the majority are discarded as invalid. The incentive mechanism rewards annotations that match majority tendency, using a split at score 4 for alignment versus non-alignment; close agreement within 1.5 points receives extra reward. The final statistics are: for Tibetan, Cohen’s $^2$ 3, ICC $^2$ 4, and Same Tendency $^2$ 5; for Mongolian, $^2$ 6, ICC $^2$ 7, and Same Tendency $^2$ 8; for Uyghur, $^2$ 9, ICC $^2$ 0, and Same Tendency $^2$ 1. Most retained samples scored 7, with average headline quality reported as 6.9/7. Average title/content lengths are 12.3/376.7 tokens and 74.0/1,884.1 characters for Tibetan; 27.2/429.8 tokens and 136.1/2,149.0 characters for Mongolian; and 30.2/815.7 tokens and 151.0/4,078.5 characters for Uyghur (Xu et al., 12 Sep 2025).

Hallucination detection work in multilingual headline generation provides an adjacent evaluation perspective. MFHHD introduces a multilingual fine-grained hallucination detection dataset with 11,469 article-headline pairs in English, Spanish, German, French, and Portuguese, annotated with coarse-grained labels Support, Neutral, and Contradict and seven fine-grained hallucination types (Shen et al., 2024). The paper uses Example-F1 for multi-label fine-grained evaluation:

$^2$ 2

Although MFHHD does not target Chinese minority languages, its relevance to CMHG is methodological: it offers a structured way to distinguish mismatch types beyond simple overlap metrics. This suggests that future CMHG evaluation may benefit from separating lexical overlap from factual support.

7. Challenges, misconceptions, and research directions

A common misconception is that CMHG is simply a specialization of Chinese headline generation and can therefore inherit high-resource assumptions. The cited literature does not support that view. It emphasizes unique writing systems, non-standard encodings, contamination in multilingual corpora, and the scarcity of supervised resources for these languages (Xu et al., 12 Sep 2025). MC $^2$ 3 further argues that script multiplicity is closely related to cultural awareness in the resulting models, and its inclusion of Kazakh Arabic script and Traditional Mongolian script is presented as a corrective to prior neglect (Zhang et al., 2023).

Another misconception is that larger generic models are automatically sufficient. The available evidence is mixed. On the CMHG benchmark, few-shot 70B–72B models achieve the strongest ROUGE-L F1 scores among the reported baselines (Xu et al., 12 Sep 2025). However, XLM-SWCM reports that a 492M shared-weight model surpasses both a 611M mBART-CM and a 13B MC2-LLaMA-13B model on Tibetan headline generation in its own evaluation setting, and ablation shows that removing weight sharing reduces ROUGE-L from 25.7 to 17.1 (Su et al., 15 Feb 2025). Within the supplied evidence, this indicates that architecture and adaptation strategy remain decisive under extreme low-resource conditions.

The most immediate research direction is resource development. MC $^2$ 4 is described as being structurally poised to facilitate CMHG but does not itself provide direct generation experiments, while the CMHG benchmark supplies the supervised pairs and native-speaker evaluation that were previously absent (Zhang et al., 2023). The CMHG paper states that there was previously no supervised headline generation dataset for these languages and positions its release as the first large-scale, open-source headline-generation corpus for Chinese minority languages (Xu et al., 12 Sep 2025). A plausible implication is that the field is now moving from corpus formation to benchmarked generation, with the next technical questions likely to concern transfer learning, hallucination control, and evaluation protocols that better capture title-content faithfulness than overlap alone.