K-News-Stance: Korean News Stance Benchmark

• K-News-Stance is a Korean dataset that benchmarks article-level stance detection by annotating 2,000 news articles across 47 societal issues.
• It employs journalism-guided segmentation with labels for headlines, leads, conclusions, and quotations to capture editorial stances.
• The benchmark uses an issue-disjoint split and integrates advanced methods like JoA-ICL to enhance stance prediction in long-form news.

Searching arXiv for recent and foundational work on K-News-Stance and closely related news stance detection. K-News-Stance is a Korean dataset and benchmark for article-level news stance detection centered on target issues rather than generic sentiment or topic classification. It was introduced as “the first Korean dataset for article-level stance detection,” comprising 2,000 news articles, 19,650 segment-level stance annotations, and 47 societal issues, with both article-level and journalistically motivated segment-level labels (Lee et al., 15 Jul 2025). In its formal task definition, given a news article $A$ and a target issue $T$, a model predicts one of three article-level stance labels—supportive, neutral, or oppositional—thereby framing stance as an issue-conditioned property of long-form journalism rather than a purely lexical or sentence-local phenomenon (Lee et al., 15 Jul 2025).

1. Conceptualization of stance in long-form news

K-News-Stance defines stance as the overall position of a news article toward a predefined target issue. The formulation is explicitly target-dependent: given article $A$ covering issue $T$, the objective is to predict stance label $$L\in\{\text{supportive},\text{neutral},\text{oppositional}\}$$ (Lee et al., 15 Jul 2025). This distinguishes the task from generic sentiment analysis, which concerns affective polarity rather than a text’s position toward a specified proposition or issue. The distinction is consistent with prior stance literature, which repeatedly treats stance as a relation between text and target rather than as standalone sentiment or topic labeling (Krejzl et al., 2017, Alam et al., 2022).

The benchmark is particularly notable for locating stance in article-level journalistic discourse rather than in short social texts. Earlier stance work often focused on rumor replies, tweets, or headline–body pairs, including the SDQC rumor taxonomy of Support, Deny, Query, and Comment (Lozhnikov et al., 2018), and the Fake News Challenge label space of agree, disagree, discuss, and unrelated (Riedel et al., 2017, Dulhanty et al., 2019). K-News-Stance instead adopts a three-way article-level taxonomy—supportive, neutral, oppositional—because its target object is a societal issue and its input is a full news article rather than a response in a conversation tree (Lee et al., 15 Jul 2025).

This design suggests a shift from interactional stance to editorial or discursive stance. Prior article-level work such as “360° Stance Detection” modeled article stance toward topics like political figures or controversial issues with labels such as in favour, against, neutral, and sometimes unrelated (Ruder et al., 2018). K-News-Stance narrows the formulation to issue-conditioned Korean news articles and removes the unrelated category at the article level, thereby making all instances issue-relevant by construction (Lee et al., 15 Jul 2025).

2. Corpus construction and issue-disjoint benchmark design

K-News-Stance was collected from BigKinds, a Korean news platform operated by the Korea Press Foundation, and from the Naver News search API. The collection period spans June 2022 to June 2024 (Lee et al., 15 Jul 2025). The pipeline first used BigKinds to identify nationwide weekly social issues across domains such as labor, gender, domestic politics, and international affairs, then randomly sampled 47 issues while maintaining temporal balance, retrieved full article contents through the Naver News search API, gathered 2,989 raw articles from 31 news outlets, and finally retained a stance-annotated subset of 2,000 articles (Lee et al., 15 Jul 2025).

A defining methodological choice is genre filtering. Articles were first assigned one of four journalism genres—straight news, analysis, opinion, or other—and only analysis and opinion pieces were retained for stance annotation because these genres are “more likely to contain opinionated content” (Lee et al., 15 Jul 2025). This distinguishes the dataset from broader news corpora that mix descriptive reporting with overtly interpretive genres.

The benchmark uses an issue-disjoint split rather than a random article split. The train partition contains 999 articles over 24 issues, and the test partition contains 1,001 articles over 23 issues (Lee et al., 15 Jul 2025). Article-level class counts are balanced overall: 638 supportive, 674 neutral, and 688 oppositional articles (Lee et al., 15 Jul 2025). The issue-disjoint split follows a general principle seen in earlier stance work: evaluation should test generalization to unseen targets rather than memorization of issue-specific lexical patterns. Related news stance work similarly enforced target separation, for example by ensuring that entities do not overlap across train, validation, and test splits in a news topic stance dataset (Ruder et al., 2018).

The benchmark’s article lengths also matter. K-News-Stance articles have mean length 1,483.58 characters, median 1,335, minimum 376, and maximum 8,185, while quotation density is substantial, with mean 7.78 quotations per article, median 8, minimum 0, and maximum 45 (Lee et al., 15 Jul 2025). This scale places the dataset firmly in the long-form news regime, a setting known to be difficult for flat encoders and truncation-based architectures (Mohtarami et al., 2018, Dulhanty et al., 2019).

3. Annotation schema and journalism-guided segment structure

A central feature of K-News-Stance is that it does not annotate only whole-article stance. It also annotates four journalistically meaningful segment types with the same three-way stance taxonomy: headline, lead, conclusion, and direct quotations (Lee et al., 15 Jul 2025). The lead is operationalized as the first paragraph, and the conclusion as the final paragraph. Quotations are defined as direct speech enclosed in double quotation marks (Lee et al., 15 Jul 2025).

The dataset’s journalism rationale is explicit. Headline, lead, conclusion, and quotations are not arbitrary spans but structural components through which journalists foreground, frame, reinforce, and source positions (Lee et al., 15 Jul 2025). This contrasts with excerpt-based stance annotation strategies that select windows around topic mentions primarily to reduce annotation cost and sequence length (Ruder et al., 2018). K-News-Stance instead treats structure as part of the theory of stance expression in news.

The full segment-level label distribution is as follows (Lee et al., 15 Jul 2025):

Segment type	Supportive	Neutral	Oppositional
Headline	21.3%	49.6%	29.1%
Lead	20.6%	52.7%	26.8%
Conclusion	27.1%	41.1%	31.9%
Quotations	26.1%	40.5%	33.4%
Article	31.9%	33.7%	34.4%

These statistics show that neutral is more frequent at the segment level, especially for headlines and leads, even though article-level classes are relatively balanced (Lee et al., 15 Jul 2025). This pattern is consistent with the idea that individual segments in journalistic writing often maintain outward neutrality while the article-level position emerges from cross-segment orchestration.

Annotation quality control is stronger than in several earlier stance datasets. Two trained annotators from the authors’ institution labeled all 2,000 articles and 19,650 segments, disagreements were resolved through discussion and consensus, and Krippendorff’s alpha ranged from 0.68 to 0.84 across article-level and segment-level annotations (Lee et al., 15 Jul 2025). This stands in contrast to older datasets where inter-annotator agreement was omitted or only majority retention rates were reported (Lozhnikov et al., 2018, Ruder et al., 2018).

The guidelines were developed under the lead of a mass communication specialist and were informed by narrative stance cues including information selection, direct quotation patterns, lexical choices, and cues implying preferred interpretations or actions (Lee et al., 15 Jul 2025). When ambiguity arose, annotators were instructed to consult additional articles on the same issue to improve consistency (Lee et al., 15 Jul 2025). This suggests a domain-aware annotation protocol closer to journalism studies than to purely platform-centered stance tagging.

4. Structural properties and empirical associations within the dataset

K-News-Stance encodes an internal theory of article stance as distributed across structured discourse units. The paper reports strong associations between article stance and headline, lead, and conclusion stance, with Cramer's $V$ around 0.7, while quotation stance has weaker association, around 0.3 (Lee et al., 15 Jul 2025). Headline and lead are also strongly correlated (Lee et al., 15 Jul 2025). These associations are important because they empirically justify the segmentation scheme and indicate that some structural components carry more direct information about the article’s overall stance than others.

The weaker association of quotations is particularly informative. Quotations are abundant and journalistically central, but they may express mixed or conflicting voices even in a single article (Lee et al., 15 Jul 2025). This makes quotation stance useful but less directly predictive of article stance than lead or headline. A similar difficulty appears in prior work on stance toward long documents: relevant evidence is distributed unevenly across paragraphs, and only some spans are directly informative (Mohtarami et al., 2018). K-News-Stance refines this by identifying discourse units that are structurally privileged rather than relying on generic paragraph segmentation.

The dataset also supports genre analysis. JoA-ICL performs better on opinion than analysis articles, with 0.780 accuracy and 0.785 macro F1 on opinion articles ($N=100$) versus 0.667 accuracy and 0.663 macro F1 on analysis articles ($N=901$) (Lee et al., 15 Jul 2025). This result aligns with the annotation premise that opinion writing expresses stance more explicitly, whereas analysis writing often embeds stance in framing, source choice, and selective emphasis rather than direct declaration.

Error analysis further indicates that supportive articles are the hardest class. For the 323 supportive test articles, the strongest baseline misclassifies many as neutral (128) and oppositional (48), and JoA-ICL still preserves the general pattern despite improving all classes (Lee et al., 15 Jul 2025). The paper attributes failures mainly to difficulty interpreting positive descriptions as supportive and to difficulty orchestrating mixed segment signals, particularly when quotations express divergent viewpoints (Lee et al., 15 Jul 2025). This is broadly consistent with earlier stance literature in which minority or subtle stance classes are frequently confused with neutral or discuss-type labels (Hanselowski et al., 2018, Mohtarami et al., 2018).

5. JoA-ICL: journalism-guided agentic in-context learning

The benchmark is introduced together with JoA-ICL, a “Journalism-Guided Agentic In-Context Learning” framework that uses segment-level stance predictions to improve article-level stance prediction (Lee et al., 15 Jul 2025). The method’s premise is that long-form news stance is difficult to infer from raw full-text prompting alone because relevant cues are distributed and may be diluted in context (Lee et al., 15 Jul 2025). Earlier work on long-document stance has likewise shown that retaining or prioritizing relevant context is crucial, whether through paragraph-level memory networks (Mohtarami et al., 2018), hierarchical neural encoders (Borges et al., 2018), or target-centered excerpts (Ruder et al., 2018).

JoA-ICL decomposes the problem into two roles. First, a segment-level agent predicts stance for the headline, lead, conclusion, and quotations relative to the issue. Second, an article-level LLM receives the full article augmented with XML-like markup encoding those predicted segment labels and outputs the final article stance (Lee et al., 15 Jul 2025). The article-level prompt explicitly instructs the model to determine final stance by considering the detailed stance labels of each part (Lee et al., 15 Jul 2025).

The framework is called journalism-guided because the segments correspond to meaningful journalistic structures, and agentic because segment-level stance prediction is delegated to a separate LM or classifier before article-level reasoning (Lee et al., 15 Jul 2025). No explicit aggregation equation is defined; instead, the article-level LLM itself performs the aggregation through prompt-conditioned reasoning over the structured annotation scaffold (Lee et al., 15 Jul 2025).

Two broad segment-agent families are evaluated: LLM-based in-context segment prediction and fine-tuned masked LLMs, specifically RoBERTa (Lee et al., 15 Jul 2025). Although Gemini-2.0-flash with 6-shot prompting is best on segment classification alone, RoBERTa-based segment prediction yields better downstream article-level JoA-ICL performance because it handles neutral segments more reliably (Lee et al., 15 Jul 2025). This indicates that the segment agent’s utility depends not only on isolated segment accuracy but also on compatibility with the article-level reasoning pipeline.

The journalism prior is validated by ablation. Using journalism-guided segments yields 0.678 accuracy and 0.672 macro F1, whereas randomly selected segments matched for length yield 0.649 accuracy and 0.645 macro F1 (Lee et al., 15 Jul 2025). The lead is the most important segment type in both oracle and predicted settings: removing lead information drops the predicted-label configuration from 0.678/0.672 to 0.610/0.630, while removing quotations has the smallest effect (Lee et al., 15 Jul 2025). This suggests that the lead functions as the most concentrated local summary of article stance, matching its definition as the first paragraph following the inverted pyramid structure (Lee et al., 15 Jul 2025).

6. Baselines, results, and comparative context

K-News-Stance evaluates both fine-tuned pretrained models and prompt-based LLM baselines. Among fine-tuned baselines, PT-HCL is strongest with 0.617 accuracy and 0.618 macro F1, compared with RoBERTa at 0.594/0.577, CoT Embeddings at 0.582/0.562, and LKI-BART at 0.545/0.538 (Lee et al., 15 Jul 2025). Among pure LLM prompting baselines, the best result is Gemini-2.0-flash with chain-of-thought prompting at 0.661 accuracy and 0.657 macro F1 (Lee et al., 15 Jul 2025).

The practical JoA-ICL setting—predicted segment labels from a RoBERTa agent plus article-level Gemini-2.0-flash with CoT and 6-shot prompting—achieves the best reported real-world result on the dataset: 0.678 accuracy and 0.672 macro F1 (Lee et al., 15 Jul 2025). This surpasses both the best fine-tuned baseline and the best non-agentic prompt baseline (Lee et al., 15 Jul 2025). With oracle segment labels, the ceiling is much higher: Exaone-2.4b without CoT or few-shot reaches 0.837 accuracy and 0.837 macro F1 (Lee et al., 15 Jul 2025). The size of the oracle–predicted gap shows that segment prediction quality is the main bottleneck.

These results place K-News-Stance in a broader pattern within news stance detection. Earlier article-level topic stance work using target-conditioned BiLSTMs reached 61.7 accuracy and 56.9 macro F1 on a 32,227 article–topic dataset (Ruder et al., 2018). Pair-encoded transformer models for English claim–article stance achieved 90.01 weighted accuracy on FNC-1, but under a label set and metric dominated by unrelated detection (Dulhanty et al., 2019). Retrospective analyses of FNC-1 showed that weighted metrics can overestimate real stance discrimination and that macro F1 provides a more faithful picture, especially for difficult minority classes (Hanselowski et al., 2018). K-News-Stance follows the latter principle by reporting macro F1 and accuracy as standard multi-class metrics (Lee et al., 15 Jul 2025).

The benchmark also tests generalization to German CheeSE, where JoA-ICL improves over zero-shot LLM baselines across GPT-4o-mini, Gemini-2.0-flash, and Claude-3-haiku (Lee et al., 15 Jul 2025). Since CheeSE lacks segment annotations, the experiment uses distant supervision by copying article labels to all segments (Lee et al., 15 Jul 2025). This does not establish full cross-lingual equivalence, but it suggests that the journalism-guided decomposition is not inherently Korean-specific.

7. Applications, limitations, and place within the literature

K-News-Stance is presented not only as a benchmark but also as an enabling resource for viewpoint-aware recommendation and media bias analysis. In a recommendation case study, JoA-ICL-enhanced re-ranking improves ideological diversity relative to both plain Contriever retrieval and standard maximal marginal relevance while maintaining similar precision (Lee et al., 15 Jul 2025). In a media-bias case study on 2025 presidential election issues, supportive versus oppositional stance distributions cluster major Korean outlets in ways aligned with known editorial leanings (Lee et al., 15 Jul 2025). These applications connect the dataset to a line of work that uses stance and sentiment annotations to audit recommender-system bias and viewpoint diversity (Alam et al., 2022).

At the same time, the benchmark has explicit limitations. It is Korean-only, covers 47 issues, retains only analysis and opinion pieces, and annotates only four segment types (Lee et al., 15 Jul 2025). The methodology also depends on segment prediction quality and on article-level LLM inference, which increases latency and cost: the appendix reports 1.242 seconds inference time per sample and approximately \$0.0008 API cost per sample for JoA-ICL, compared with 0.007 seconds for RoBERTa and 0.599 seconds for a zero-shot LLM baseline (Lee et al., 15 Jul 2025).

Within the broader stance literature, K-News-Stance occupies a distinct niche. It differs from rumor-reply datasets such as RuStance, which use Support, Deny, Query, and Comment labels on Russian tweets and news comments (Lozhnikov et al., 2018); from headline–body benchmarks such as FNC-1 (Riedel et al., 2017, Hanselowski et al., 2018); and from ideological article-level datasets that infer stance from outlet labels or political slant (Ko et al., 2023, Zhang et al., 2022). Its distinctive contribution is to treat stance in long-form news as a structured journalistic phenomenon and to ground both annotation and modeling in headline, lead, conclusion, and quotation structure (Lee et al., 15 Jul 2025).

This suggests a broader methodological implication. Prior work has repeatedly shown that long-document stance detection benefits from target conditioning, selective evidence extraction, and document structure (Ruder et al., 2018, Mohtarami et al., 2018, Borges et al., 2018). K-News-Stance extends that trajectory by making journalistic structure itself a supervised object. A plausible implication is that future news stance research may move away from monolithic article encoders toward architectures that explicitly model editorial discourse functions, source quotations, and lead framing as first-class stance carriers.