---
title: 'FEST Competition: Low-Supervision Benchmarks'
url: https://www.emergentmind.com/topics/fest-competition
type: topic
---

# FEST Competition: Low-Supervision Benchmarks

Searching arXiv for papers on FEST-related competitions to ground the article in the cited literature.
In current arXiv usage, **FEST Competition** does not denote a single canonical benchmark. The label appears most explicitly in two document-analysis settings: the **ICDAR 2025 Competition on FEw-Shot Text line segmentation of ancient handwritten documents (FEST)** and the **Baidu-FEST** track of the **ICDAR 2023 Competition on Structured text extraction from Visually-Rich Document images (SVRD)**. A related but distinct use of **Fest** appears in **BUET CSE Fest DL Sprint 4.0**, which hosted long-form Bengali speech tasks rather than a document-analysis benchmark [2509.12965][2306.03287][2602.21183].

## 1. Scope and nomenclature

The most precise use of **FEST** in the competition literature is the acronym **“FEw-Shot Text line Segmentation of ancient handwritten documents”**, introduced for ICDAR 2025. That competition is described as the **first competition specifically focused on text line segmentation of handwritten historical documents in a few-shot setting** [2509.12965].

A second use is **Baidu-FEST**, a track within the broader ICDAR 2023 SVRD competition. There, FEST refers to **Zero-shot / Few-shot Structured Text extraction** from visually rich document images, evaluated from an end-to-end perspective rather than as isolated OCR or labeling subproblems [2306.03287].

A third, non-acronymic usage appears in speech technology. The system paper **“823-OLT @ BUET DL Sprint 4.0”** states that its official competition dataset was provided as part of the **DL Sprint 4.0 competition organized under BUET CSE Fest**, with tasks in **Automatic Speech Recognition (ASR)** and **speaker diarization** for long-form Bengali audio [2602.21183].

| Setting | Official designation | Primary task |
|---|---|---|
| ICDAR 2025 | FEw-Shot Text line Segmentation of ancient handwritten documents (FEST) | Pixel-level text line segmentation |
| ICDAR 2023 SVRD | Baidu-FEST | Zero-shot / few-shot structured text extraction |
| BUET CSE Fest DL Sprint 4.0 | BUET DL Sprint 4.0 organized under BUET CSE Fest | Long-form Bengali ASR and speaker diarization |

This distribution of meanings suggests that **FEST** functions less as a single benchmark identity than as a recurring label for low-supervision or practically constrained evaluation settings.

## 2. ICDAR 2025 FEST: task definition and dataset design

The ICDAR 2025 FEST competition addresses **text line segmentation in ancient handwritten manuscript pages** under an explicitly **few-shot learning setting**. Participants must develop systems capable of segmenting text lines in the **U-DIADS-TL** dataset using **only three annotated images per manuscript for training**. The stated motivation is that historical handwritten documents exhibit **irregular handwriting, faded or degraded ink, multi-column and non-linear layouts, overlapping lines, and heterogeneous scripts**, while large pixel-accurate annotations are expensive and rare [2509.12965].

The dataset, **U-DIADS-TL**—expanded as **Uniud - Document Image Analysis DataSet - Text Line**—contains **84 color page images** drawn from **three historical manuscripts**: **Latin 2**, **Latin 14396**, and **Syriaque 341**, with **28 pages from each manuscript**. Each image has a **pixel-accurate GT mask in PNG format**, at the same resolution as the source image, and the annotation classes are **background** and **text lines**. The ground truth is described as **fine-grained, noise-free, and non-ambiguous** [2509.12965].

The few-shot split is manuscript-specific. For each manuscript, the organizers provided **13 annotated images**, of which **3 images constituted the training set** and **10 could be used for validation**. In addition, **15 private test images per manuscript** were withheld and used only for final evaluation. The benchmark is therefore designed to mimic a humanities workflow in which a scholar has only a few manually labeled pages for a new manuscript [2509.12965].

A central distinguishing claim is that FEST differs from the earlier **ICDAR 2024 SAM competition**, which addressed **layout segmentation** rather than **text line segmentation** and used **U-DIADS-Bib** rather than **U-DIADS-TL**. This distinction matters because FEST evaluates pixel-level delineation of text lines rather than broader region-level layout parsing [2509.12965].

## 3. Evaluation protocol, methods, and results in ICDAR 2025 FEST

Evaluation in ICDAR 2025 FEST used **five metrics**: **Pixel Intersection over Union (PIU)**, **Line Intersection over Union (LIU)**, **Detection Rate (DR)**, **Recognition Accuracy (RA)**, and **F-measure (FM)**. A predicted line and a ground-truth line are matched by component-wise overlap, and a match is confirmed if both pixel-level precision and recall exceed **75%**. The competition computes all metrics **individually for each manuscript**, and the **final ranking** is based on the **average Line IU score across the three manuscripts** [2509.12965].

The organizers reported three semantic-segmentation baselines—**FCN**, **PSPNet**, and **DeepLabV3+**—to contextualize submissions. The paper states that **most competition entries outperform these baselines**, indicating that the few-shot historical-document setting benefits from task-specific architectures, post-processing, or layout-aware decomposition rather than naïve transfer of conventional supervised segmentation models [2509.12965].

A total of **11 teams** submitted results. The submissions covered a broad methodological spectrum. **CV-Group** used **HookFormer**, a multiscale Transformer with a **context branch**, a **target branch**, a **feature hooking mechanism**, and **Cross-Attention Swin-Transformer blocks**, followed by a **positional refinement stage** based on **Mask R-CNN pretrained on READ-BAD**. **TAU-CH** used a two-stage classical pipeline built from **Otsu thresholding**, morphological processing, **anisotropic Gaussian filtering** with elongation factor $\eta = 3$, and **watershed segmentation**. **SRCB** used **SegFormer** with post-processing and explicitly compared **Dice loss** with **Lovász-Softmax loss**. **VAI-OCR** used a **DBNet**-style two-stage pipeline, with a first stage for **shrunk text line extraction** and a second stage for **full text line segmentation**. **GPI** used a fully deterministic classical pipeline with only one learned parameter: the **maximum connected-component area** [2509.12965].

The winning team was **PERO**. Its system used a two-stage pipeline consisting of **layout analysis** and **post-processing**. The first stage used **ParseNet**, trained on the **PERO layout dataset**, to predict **baselines**, **distances from baseline to top/bottom of bounding polygon**, and **text region borders**. The second stage applied **adaptive Gaussian thresholding**, baseline rendering to bridge fragments, morphological dilation, and connected-component filtering. On the final ranking, **PERO** achieved **PIU 0.803**, **LIU 0.961**, **DR 0.815**, **RA 0.849**, and **FM 0.830**. The paper states that this **LIU = 0.961** outperformed **SRCB** by **3.7%** and **CV-Group** by **6.4%** [2509.12965].

The remaining ranking was: **SRCB** second with **LIU 0.924**; **CV-Group** third with **LIU 0.897**; **GPI** fourth with **LIU 0.856**; **CV-Lab** fifth with **LIU 0.792**; **TAU-CH** sixth with **LIU 0.790**; **VAI-OCR** seventh with **LIU 0.747**; **DIA-Group** eighth with **LIU 0.572**; **BBA** ninth with **LIU 0.219**; **Codecrackers** tenth with **LIU 0.203**; and **LDLD** eleventh with **LIU 0.000**. The paper also highlights that **CV-Group** had strong **LIU** but comparatively low **RA** and **FM**, implying that line-count consistency and one-to-one matching remained difficult even when overlap metrics were strong. It further highlights **GPI** as evidence that carefully designed classical vision methods can remain competitive in this setting [2509.12965].

## 4. Baidu-FEST in ICDAR 2023 SVRD

**Baidu-FEST** was one of two tracks in the **ICDAR 2023 Competition on Structured text extraction from Visually-Rich Document images (SVRD)**. Its purpose was to evaluate the performance and generalization of **Zero-shot / Few-shot Structured Text extraction from an end-to-end perspective**. The benchmark targets enterprise-like scenarios, especially documents from **finance**, **insurance**, **logistics**, and **customs inspection**, with formats including **cards**, **receipts**, and **forms** [2306.03287].

The benchmark contains **about 11 kinds of synthetic business documents for training** and **10 types of real visually-rich document images for testing**, with **about 60 images per document type**. Each image is annotated with **text-field bounding boxes (bboxes)**, **transcript**, **entity caption**, and **entity id** for each text bbox. The geometry of each box is represented by **four vertices in clockwise order starting from the top** [2306.03287].

Baidu-FEST defined two tasks. **Task 3**, **E2E Zero-shot Structured Text Extraction**, required extraction of texts for predefined key fields from document images where the **training and testing scenarios are disjoint**. **Task 4**, **E2E Few-shot Structured Text Extraction**, used the same output format but additionally provided **localization information and transcripts**, while limiting each test scenario to **no more than five images per scenario** for adaptation. In both tasks, the ground-truth captions **`caption_en`** and **`caption_ch`** could be used as prompts but **cannot be modified** [2306.03287].

Evaluation combined **Normalized Edit Distance (NED)** and **F-score** into a final ranking score,
$$
score = 0.8 \cdot score1 + 0.2 \cdot score2
$$
where `score1` is the NED-based score and `score2` is the F-score. The paper emphasizes that the benchmark evaluates the **full extraction pipeline** rather than isolated OCR or field-labeling modules [2306.03287].

Participation was moderate but nontrivial. For the full competition, the paper reports **50 participants** and **117 submissions**. For Baidu-FEST specifically, **Task 3** received **7 participants** and **15 submissions**, and **Task 4** received **8 participants** and **11 submissions** [2306.03287].

## 5. Systems, outcomes, and limits in Baidu-FEST

For **Task 3: Zero-shot**, the top three teams were **USTC + iFLYTEK AI Research** with **78.71%**, **TencentOCR** with **77.35%**, and **Ant Group** with **71.44%**. The champion system used **DBNet-det + SVTR-rec** for OCR, manually ordered boxes, concatenated OCR text into a sequence, and fed that sequence into a seq2seq model trained with multiple **BERT-family variants** under **UniLM**; the method also used augmentations such as **random text replacement/erasing** and **box scaling/shifting**, and for English documents incorporated **DocPrompt outputs** [2306.03287].

For **Task 4: Few-shot**, the top three teams were **TencentOCR** with **84.43%**, **USTC + iFLYTEK AI Research** with **82.13%**, and **CMSS** with **74.90%**. The champion system used a **LayoutLMv3 + StrucTextv1 backbone**, pretrained-model fine-tuning, OCR preprocessing to merge or split badly detected boxes, self-supervised training for segment-based classification on some folders, rule-based postprocessing, and model ensembling [2306.03287].

The paper stresses that strong aggregate scores should not be mistaken for solved generalization. It explicitly states that there is still a **large performance gap** for complex and zero-shot structured extraction. The reported per-scenario results of the Task 3 champion dropped to **53.25%** on **Letter/Email** and **46.2%** on **Technical Report**; in Task 4, performance dropped to **59.23%** on **Car Ticket**. The authors interpret these results as evidence that current systems may overfit easier document patterns and remain weak on harder unseen document types [2306.03287].

This establishes Baidu-FEST as a benchmark oriented toward **scenario-level robustness**, **generalization to unseen document schemas**, and **end-to-end extraction quality**. A plausible implication is that the track exposed limitations not visible in traditional OCR-centric benchmarks, particularly when training and testing document types are intentionally separated.

## 6. Related “Fest” usages and broader significance

A separate competition context appears in **BUET CSE Fest DL Sprint 4.0**, where the tasks were **long-form Bengali ASR** and **speaker diarization** rather than document analysis. The system description paper states that the competition focused on **long-form Bengali speech understanding**, with realistic recordings containing **background noise**, **music contamination**, **silence intervals**, **overlapping or alternating speakers**, and **conversational variability**. The described ASR pipeline combined **Demucs (htdemucs)** vocal separation, **Silero VAD**, a **gap aware windowing strategy** with about **20-second** windows and a **5-second** silence threshold, **Context Padding ($\pm 1s$)**, **Whisper-Medium decoding**, and text normalization. For diarization, the final system fine-tuned **`pyannote/segmentation-3.0`** on the official competition dataset and reported a best **Private DER: 0.28816** [2602.21183].

This speech benchmark is not the same as ICDAR 2025 FEST or Baidu-FEST, but it shows that **Fest** can designate a broader event umbrella under which a technical competition is organized. That naming overlap is a recurrent source of ambiguity.

A second source of ambiguity is terminological rather than competitive: **FEST** is also the acronym for **Feature Engineering with Self-evolving Trees**, an interpretable feature-engineering framework for raw text and images, not a competition. That work reports results in **17 of 20 classifier-task combinations**, a **mean gain of 4.2 pp** over the strongest baseline across five classifiers, and introduces the **BrandGuide** dataset with **1M+ assets across 2,683 brands**, but it is methodologically unrelated to the competition benchmarks that use the same label [2606.08800].

Taken together, the competition literature indicates that the most established meaning of **FEST Competition** is the ICDAR 2025 benchmark on **few-shot text line segmentation of ancient handwritten documents**, while **Baidu-FEST** denotes a distinct ICDAR 2023 track on **zero-shot and few-shot structured text extraction**, and **BUET CSE Fest** denotes an event context for a speech competition rather than a FEST acronym. This suggests that any technical discussion of “FEST Competition” requires immediate disambiguation by domain, host event, and task definition.

Source: https://www.emergentmind.com/topics/fest-competition