---
title: 'GOAL Dataset: Diverse Benchmarks'
url: https://www.emergentmind.com/topics/goal-dataset
type: topic
---

# GOAL Dataset: Diverse Benchmarks

The expression **GOAL Dataset** is used in the literature for several distinct resources rather than a single canonical corpus. The name appears most prominently in three benchmark families: a multilingual wikiHow-derived dataset for **Goal-Oriented Script Construction**, an English benchmark for **sports game summarization**, and a soccer benchmark for **knowledge-grounded video captioning**. In automated planning, closely related usage appears in **GOAL-style** goal-recognition benchmarks, where recent work has revised dataset construction to reduce planner-induced bias [2107.13189] [2207.08635] [2303.14655] [2602.14691].

## 1. Nomenclature and principal uses

The ambiguity of the term is best understood as a consequence of independent research communities reusing the word “goal” for different formal objects: procedural intentions, task-conditioned summaries, commentary generation, and goal hypotheses in planning.

| Usage | Domain | Core characterization |
|---|---|---|
| **GOAL** [2107.13189] | Script learning | Multilingual wikiHow benchmark for generating ordered steps from a goal |
| **GOAL** [2207.08635] | NLP summarization | First English sports game summarization dataset from live commentary to news |
| **GOAL** [2303.14655] | Video-language | Knowledge-grounded soccer video captioning benchmark for real-time commentary |
| **GOAL-style benchmarks** [2602.14691] | Automated planning | Goal-recognition datasets built from planning traces, recently extended to multi-plan versions |
| **GOALS** [2207.14447] | Medical imaging | Different acronym: Glaucoma OCT Analysis and Layer Segmentation |

This plurality matters because the term “GOAL Dataset” can otherwise be misread as referring to a single established benchmark. In practice, one must identify the relevant domain and paper before interpreting scale, annotations, and evaluation protocol.

## 2. GOAL as a multilingual script-learning dataset

In "Goal-Oriented Script Construction" [2107.13189], **GOAL** is a multilingual benchmark for constructing full procedural scripts from goals. The dataset is derived from **wikiHow**, and the paper describes it as the **first multilingual script learning dataset** supporting **18 languages**. The extracted schema is explicit: each article becomes a goal-oriented script with **Goal**, **Section**, **Steps**, and **Category**. The **Goal** is the article title with “How to” stripped; **Steps** are the ordered event sequence; and **Sections** are retained in extraction but flattened for modeling into a single ordered script.

The scale is substantial. Appendix counts sum to approximately **444,000 articles/scripts** across 18 languages, including **112,111** English articles, **64,725** Spanish, **34,194** Portuguese, **31,541** German, and smaller collections down to **1,434** Turkish. Only a subset is marked as ordered scripts; for example, English has **54,852 ordered articles**, Spanish **26,620**, and Turkish **520**. The split protocol holds out **10%** of articles in each language as test, uses **90%** for training and development, and then holds out **5%** of the training portion for development [2107.13189].

The task formulation is goal-conditioned. Given a goal $g$, the system constructs a complete script as an ordered list of steps $S$, with ground-truth reference $T$, and receives the expected number of steps $l$ as input. In the retrieval setting, the model must predict a subset of steps from a candidate pool $C$ and provide their ordering. The paper defines script-construction accuracy and Kendall-style ordering metrics, including
$$
\textrm{acc} = (\sum_i^l [s_i \in T])/l
$$
and
$$
\tau = \frac{NC(S\cap T,T\cap S)-ND(S\cap T,T\cap S)}{\binom{l}{2}}.
$$

The benchmark evaluates both generation and retrieval. The generation baseline fine-tunes **mT5-Large**; the retrieval pipeline performs **Step Inference** and **Step Ordering** with **mBERT** or **XLM-RoBERTa**. Reported average retrieval performance remains modest, with **mBERT** at **Accuracy .286, Kendall’s $\tau=.198$** and **XLM-R** at **Accuracy .317, Kendall’s $\tau=.075$**. Human evaluation shows a trade-off: retrieval outputs contain more correct steps, whereas generation outputs are better ordered but hallucinate more. The dataset also transfers zero-shot to **SMILE**, **OMICS**, **DeScript**, and **KAIROS Schema Learning Corpus**, which the paper uses to argue that goal-conditioned procedural knowledge learned from wikiHow is not merely an artifact of one source format [2107.13189].

## 3. GOAL as an English sports game summarization benchmark

In "GOAL: Towards Benchmarking Few-Shot Sports Game Summarization" [2207.08635], **GOAL** denotes the **first English sports game summarization dataset**. The task is to generate a sports news article from a document of real-time soccer commentary. The corpus is crawled from **Goal.com** and covers four competitions: **UEFA Champions League**, **UEFA Europa League**, **Premier League**, and **Serie A**, over the period **2016 to 2020**.

The paper collected **2,263 football games total**. All **2,263** have commentary documents, but only **103** have paired sports news articles. Those **103 commentary-news pairs** constitute the supervised dataset, while the remaining **2,160 commentary-only documents** are released as unlabeled data for semi-supervised research. The labeled split is **63 training**, **20 validation**, and **20 test** examples. The average commentary length is **2724.9 words** and the average news length is **476.3 words**; the corresponding 95th percentiles are **3699** and **614** [2207.08635].

The task is formalized as mapping a timestamped commentary sequence
$$
C=\{(t_1,c_1), (t_2,c_2),...,(t_n,c_n)\}
$$
to a news article
$$
R=\{r_1,r_2,...,r_m\}.
$$
A central difficulty is that, unlike prior Chinese sports summarization datasets, GOAL commentary omits the explicit score field $s$ found in tuples of the form $(t,c,s)$. The model must therefore infer game state implicitly from commentary text. The paper also emphasizes a strong source-target style gap: commentary is short, event-driven, and colloquial, whereas news is compact and journalistically written.

The benchmark includes extractive and abstractive baselines. **Longest**, **TextRank**, and **PacSum** are used as extractive systems; **PGN** and **LED** are used as abstractive systems. The best reported model is **LED**, with test **ROUGE-1 34.7**, **ROUGE-2 7.8**, and **ROUGE-L 24.3**. The paper interprets these numbers as evidence that the task remains difficult even for long-document summarization architectures, and reports frequent repetition and missed important events in generated outputs [2207.08635].

## 4. GOAL as a knowledge-grounded soccer commentary benchmark

In "GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary Generation" [2303.14655], **GOAL** stands for **knowledge-Grounded videO cAptioning benchmark for reaL-time soccer commentary generation**. Here the target is not summarization from text, but generation of soccer commentary from video plus external knowledge. The paper introduces the task of **Knowledge-grounded Video Captioning (KGVC)**, formalized as
$$
S_{v,k} = \mathbf{g}(V,K_v) = \mathbf{g}(V,\mathbf{f}(V)).
$$
The function $\mathbf{f}$ selects relevant knowledge for video $V$, and $\mathbf{g}$ generates the final commentary from video and selected knowledge.

The benchmark is built from **SoccerNet-v2**. The authors start with **80 full-game videos narrated in English**, run **Azure ASR toolkit** on commentary speech, filter out videos with **low resolution** or **too sparse narration**, and retain **20 games** as final annotation candidates. The final resource contains **over 8.9k soccer video clips**, **22k sentences**, and **42k knowledge triples**. The average clip length is **10.31 seconds**, each clip has **2.46 labeled sentences** on average, and each video is relevant to **193.1 knowledge triples on average** [2303.14655].

Annotation is extensive. The team consists of **10 English native speakers** who are also experienced soccer fans. The **20 preserved videos** are distributed among these **10 annotators**; each annotator completes **4 videos**, so every video is annotated by **two people**. Their tasks are commentary proofreading, video-text alignment, and sentence-level knowledge annotation. Each sentence is labeled as **scene description**, **background introduction**, or **comment**, and linked to entities such as players, teams, and soccer terms. Knowledge is collected from **WhoScored** and **Wikidata**, with entity linking performed using **BLINK** and **2-hop related entity pages** crawled from Wikidata [2303.14655].

The paper positions this GOAL benchmark as harder than standard video captioning because commentary must identify fine-grained entities, infer event semantics, and incorporate knowledge not directly visible in the clip. Three adapted baselines are reported: **HMN**, **SwinBERT**, and **ALPRO**, each with and without knowledge. The strongest result is **ALPRO + Knowledge**, with **BLEU 15.6**, **METEOR 6.4**, **ROUGE-L 11.5**, and **CIDEr 6.6**. The gap to standard captioning benchmarks is large: the paper notes that SwinBERT averages **95.5 CIDEr** on several standard datasets but only **3.7** on GOAL, which the authors interpret as evidence that KGVC remains substantially unresolved [2303.14655].

## 5. GOAL-style datasets in automated planning and goal recognition

A different usage appears in "Removing Planner Bias in Goal Recognition Through Multi-Plan Dataset Generation" [2602.14691]. This work does not introduce a branded dataset called **GOAL**, but it explicitly addresses the design of **GOAL-style goal-recognition benchmarks** in automated planning. Its central claim is that existing datasets are systematically biased because they are usually generated by heuristic-based forward-search planners and therefore encode a narrow planner-specific behavioral signature.

The paper formalizes a planning task as
$$
\planningtask=\langle S, A, \gamma, \operatorname{cost}, I, G\rangle
$$
and a goal-recognition task as
$$
\Xi^\Omega_\mathcal{G} = \langle \planningdomain, \goalconditions, \observations \rangle.
$$
Its methodological contribution is to generate multiple plans for the same goal hypothesis using top-$k$ planning rather than a single trace. Each plan yields a different observation sequence, and each observation sequence becomes a different **version** of the same underlying goal-recognition instance. The generation pipeline, denoted $\operatorname{TaskGenerator}$, takes a planning task $\Xi$, planner, goal hypothesis $g$, required number of variants $k$, observability percentage $O$, and noise percentage $N$, then returns clean and noisy task collections. The paper states that common dataset settings use
$$
O \in \{10, 30, 50, 70, 100\}
$$
and
$$
N \in \{0,10,20,30\}.
$$

To evaluate grouped versions, the paper introduces **Version Coverage Score (VCS)**. For a set of goal-recognition tasks sharing the same hypothesis,
$$
\Gamma=\{\Xi^{\Omega_1}_\mathcal{G},\dots,\Xi^{\Omega_k}_\mathcal{G}\},
$$
and a recognizer $\mathcal{R}$,
$$
\operatorname{VCS}:\Gamma\times\{\mathcal{R}\}\rightarrow\mathbb{R}
$$
computes the ratio of correctly solved tasks to the number of tasks. A recognizer is said to be resilient if
$$
\operatorname{VCS}(\Gamma,\mathcal{R}) \ge T
$$
for threshold $T\in[0,1]$ [2602.14691].

Empirically, the paper regenerates the Ramírez and Geffner benchmark with **$k=5$** using **SymK** and evaluates a landmark-based recognizer. The reported result is that accuracy and positive predictive value decline as the resilience threshold increases, especially at low observability. At **10% observability**, mean accuracy falls from **95.0** at threshold **0.0** to **76.0** at threshold **1.0**, while PPV falls from **81.0** to **13.0**. The paper’s broader implication is that GOAL-style evaluation should measure robustness across multiple valid plans for the same goal, not only success on a single planner-preferred trace [2602.14691].

## 6. Disambiguation with GOALS and other goal-oriented resources

The ambiguity surrounding GOAL is compounded by nearby acronyms and by the generic use of the adjective **goal-oriented**. **GOALS**, for example, is a different term altogether: in medical imaging it denotes the **Glaucoma OCT Analysis and Layer Segmentation** challenge, which released **300 circumpapillary OCT images** for retinal layer segmentation and glaucoma classification [2207.14447]. Despite the visual similarity of the names, GOALS is unrelated to the GOAL datasets in script learning, sports summarization, soccer commentary, or planning.

A second source of confusion is that many datasets are goal-oriented without being named GOAL. Examples include **doc2dial**, a goal-oriented document-grounded dialogue dataset with **4,793 conversations** over **487 documents** [2011.06623], and the code-mixed DSTC2 derivatives introduced in "A Dataset for Building Code-Mixed Goal Oriented Conversation Systems" [1806.05997]. These resources are relevant to the broader study of goals in language and planning, but they are not instances of a proper noun “GOAL Dataset.”

Taken together, these usages suggest that **GOAL Dataset** is best treated as a disambiguation term. In script learning, it denotes a large multilingual procedural benchmark; in NLP summarization, it denotes a small but influential English soccer-news resource; in video-language research, it denotes a knowledge-grounded commentary benchmark; and in automated planning, it denotes a family of goal-recognition benchmarks whose construction has recently shifted from single-plan to multi-plan evaluation.

Source: https://www.emergentmind.com/topics/goal-dataset