---
title: PIE-English Dataset Overview
url: https://www.emergentmind.com/topics/potential-idiomatic-expression-pie-english-dataset
type: topic
---

# PIE-English Dataset Overview

A Potential Idiomatic Expression (PIE)–English Dataset is a rigorously constructed resource that captures multiword expressions in English which can bear idiomatic as well as literal interpretations, serving as a foundation for classification, detection, and understanding of non-compositional language phenomena in NLP tasks. From its inception in "Casting a Wide Net" [1911.08829] through later large-scale and fine-grained corpora, PIE-English datasets provide both annotated and automatically extracted examples to enable state-of-the-art research on idiomaticity in machine translation, understanding, generation, and representation learning.

## 1. Definition and Conceptual Scope

A Potential Idiomatic Expression (PIE) is defined as a multiword string in English whose meaning can be, but is not necessarily, non-compositional—i.e., the meaning may depart from the sum of its parts, potentially bearing a figurative or fixed meaning in context [1911.08829]. Datasets label each PIE instance in sentence context (token- or span-level), often with a dichotomous (idiomatic/literal) or multi-class (e.g., metaphor, euphemism, literal, etc.) sense assignment. The PIE scope includes fixed idioms, syntactically flexible expressions, and figurative multiword units.

## 2. Dataset Construction Methodologies

PIE-English datasets are curated via a pipeline that combines dictionary mining, corpus-driven extraction, and structured human annotation:

- **Dictionary Aggregation**: Resources such as Wiktionary, Oxford Dictionary of English Idioms, and UsingEnglish.com are scraped and normalized. The set intersection yields high-confidence PIE types (e.g., 591 entries in [1911.08829]).
- **Corpus Harvesting**: Candidate sentences are drawn from well-established corpora such as the British National Corpus (BNC) and Web-as-Corpus (UKWaC), targeting broad genre coverage [1911.08829, 2105.03280].
- **Span and Sense Annotation**: Sentences identified via dependency parses or pattern-matching are annotated for PIE presence and sense, utilizing guidelines for core word alignment, permissible inflection or insertion, and idiom sense taxonomy (idiomatic, literal, other) [1911.08829, 2112.02994, 2105.03280].
- **Statistical Sampling and Extension**: For extensibility, sampled sentences for each idiom type (15–22 per type), with automatic or manual balancing for literal/idiomatic senses [2105.03280].
- **Automated Extraction**: Parse-based subtree alignment and inflection-aware string matchers yield robust span identification, with system F₁ scores up to 92% [1911.08829].

## 3. Annotation Schemes and Reliability

Annotation protocols marry precise linguistic heuristics with standardized evaluation of inter-annotator agreement (IAA). Key schemes and metrics:

- **BIO Tagging**: Pie spans are labeled in the token sequence with “B-IDIOM”/“I-IDIOM”/“O” tags [2112.02994, 2511.16345].
- **Sense Labels**: PIEs are assigned to fine-grained classes—beyond literal/non-literal, PIE-English supports metaphor, simile, euphemism, personification, oxymoron, paradox, hyperbole, irony, parallelism, and literal [2105.03280].
- **Agreement Metrics**: IAA is computed using Fleiss’ κ or raw percentage agreement. Reported English PIE datasets achieve κ = 0.74–0.91 (PIE detection) and κ = 0.63–0.83 (sense labeling) [1911.08829]; overall IAA of 88.89% is reported on the ten-class variant [2105.03280].
- **Adjudication**: Cases of disagreement are resolved through expert consensus or application of a label hierarchy.

## 4. Dataset Composition and Format

PIE-English resources span varied corpus sizes, granularity, and annotation richness:

| Dataset                       | Sentences   | PIE Types | Sense Labels                   | Format Details                         |
|-------------------------------|-------------|-----------|-------------------------------|----------------------------------------|
| [1911.08829] PIE corpus       | 1,646,208*  | 591       | idiomatic/literal/other        | Sent + span + lemma/PoS + label        |
| [2112.02994] EPIE Static      | 21,890      | 359       | idiomatic (only)               | Cloze-style, BIO, tokenized, context   |
| [2105.03280][2205.03666] PIE  | 20,174      | 1,200     | 10-class (metaphor, etc.)      | Token-in-context, PoS, class, gloss    |
| [2511.16345] PIFL-OSCAR       | 5.8M        | 3,152     | unlabeled/human-labeled subset | Full sentence, BIO, PoS, metrics       |

*Corpus tokens; 2,239 candidate sentences, 1,050 gold PIEs annotated.

Most datasets release tabular (csv/tsv), JSON, or HuggingFace-compatible formats, each record including sentence, idiom span or type, sense label, linguistic tags, and (where relevant) disambiguation scores or metrics.

## 5. Evaluation Protocols and Metrics

Standard protocol distinguishes between identification (span or sentence-level), disambiguation (idiomatic vs. literal), and classification (multi-class):

- **Precision/Recall/F₁**: F₁ computed as $2\frac{P \times R}{P + R}$ where $P = \frac{TP}{TP + FP}$ and $R = \frac{TP}{TP + FN}$.
- **Inter-Annotator Agreement**: Fleiss’ κ, raw agreement, Cohen’s κ (where pairwise), and classwise statistics [2105.03280, 1911.08829, 2511.16345].
- **Baseline Models**: Sequence labeling (BiLSTM-CRF), transformer classifiers (BERT, T5), and dialogue generation models (DialoGPT) [2205.03666, 2511.16345].
- **Slot/Sentence Labeling**: Sequence accuracy (mean exact match), True Positive Consistency for idiom detection [2405.10579].

Reported performance includes F₁ > 0.92 for parser-based extraction [1911.08829], F₁ ~0.95 for BERT on multi-class sense [2105.03280], and macro-F₁ = 0.98 for T5 classification [2205.03666]. PIFL-OSCAR achieves F₁ = 0.77/0.89 on human-annotated test splits [2511.16345].

## 6. Use Cases and Applications

PIE-English datasets power a spectrum of research and applied NLP tasks:

- **Detection and Disambiguation**: Binary/multi-class labeling of idiom vs. literal usage, figurative-sense classification, and cross-lingual transfer [2105.03280, 2205.03666].
- **Machine Translation**: Alignment-aware idiom detection boosters; transfer of non-compositionality to target languages [2105.03280, 1911.08829, 2511.16345].
- **Text Generation**: Controlled style transfer—literal-to-idiomatic paraphrasing and bench-marking text generation models for idiom plasticity [2104.06541].
- **Representation Learning**: Probing contextual and static word/sentence embeddings for sensitivity to idiomaticity (Affinity and Scaled Similarity metrics) [2411.02610].
- **Dialogue Systems**: Fine-tuning conversational models for idiom-rich input domains, improving responsiveness and naturalness [2205.03666].
- **Figurative Language Analysis**: Richly annotated suites for metaphor, simile, and other figures-of-speech enable sophisticated NLU and WSD research [2105.03280].

## 7. Limitations and Extensibility

Notwithstanding their scale and detail, PIE-English resources exhibit certain limitations:

- **Coverage**: Even at millions of instances, idiom inventories may omit emergent, domain-specific, or regionally circumscribed expressions [2511.16345].
- **Balance**: Class imbalance reflects the natural distribution of figurative types (e.g., metaphors vastly outnumber irony) [2105.03280].
- **Annotation Depth**: Full gold sense annotation is unavailable for large-scale automatically extracted corpora; verified subsets provide reliable gold standards [2511.16345].
- **Contextual Boundaries**: Single-sentence focus may fail to capture discourse-level idiom cues or multi-sentence idiom instantiations [2104.06541].
- **Adaptability**: Extensions to new idioms, genres, or research tasks are encouraged in released guidelines—addition of new idiom types, cross-lingual, and multimodal analogs (e.g., XMPIE) are now emerging [2601.08645].

## References

- "Casting a Wide Net: Robust Extraction of Potentially Idiomatic Expressions" [1911.08829]
- "IBERT: Idiom Cloze-style reading comprehension with Attention" [2112.02994]
- "Potential Idiomatic Expression (PIE)-English: Corpus for Classes of Idioms" [2105.03280]
- "Vector Representations of Idioms in Conversational Systems" [2205.03666]
- "NLP Datasets for Idiom and Figurative Language Tasks" [2511.16345]
- "A Parallel Cross-Lingual Benchmark for Multimodal Idiomaticity Understanding" [2601.08645]

Source: https://www.emergentmind.com/topics/potential-idiomatic-expression-pie-english-dataset