FEST Competition: Low-Supervision Benchmarks

• FEST Competition is a suite of benchmarks focused on low-supervision challenges in document image analysis and speech recognition.
• It encompasses tasks such as few-shot text line segmentation for ancient manuscripts and zero-shot structured text extraction from visually-rich documents, using datasets like U-DIADS-TL.
• Evaluations employ diverse metrics (PIU, LIU, DR, RA, FM) and a range of methodologies, highlighting both advances and persistent challenges in generalization.

Searching arXiv for papers on FEST-related competitions to ground the article in the cited literature. In current arXiv usage, FEST Competition does not denote a single canonical benchmark. The label appears most explicitly in two document-analysis settings: the ICDAR 2025 Competition on FEw-Shot Text line segmentation of ancient handwritten documents (FEST) and the Baidu-FEST track of the ICDAR 2023 Competition on Structured text extraction from Visually-Rich Document images (SVRD). A related but distinct use of Fest appears in BUET CSE Fest DL Sprint 4.0, which hosted long-form Bengali speech tasks rather than a document-analysis benchmark (Zottin et al., 16 Sep 2025, Yu et al., 2023, Dhar et al., 24 Feb 2026).

1. Scope and nomenclature

The most precise use of FEST in the competition literature is the acronym “FEw-Shot Text line Segmentation of ancient handwritten documents”, introduced for ICDAR 2025. That competition is described as the first competition specifically focused on text line segmentation of handwritten historical documents in a few-shot setting (Zottin et al., 16 Sep 2025).

A second use is Baidu-FEST, a track within the broader ICDAR 2023 SVRD competition. There, FEST refers to Zero-shot / Few-shot Structured Text extraction from visually rich document images, evaluated from an end-to-end perspective rather than as isolated OCR or labeling subproblems (Yu et al., 2023).

A third, non-acronymic usage appears in speech technology. The system paper “823-OLT @ BUET DL Sprint 4.0” states that its official competition dataset was provided as part of the DL Sprint 4.0 competition organized under BUET CSE Fest, with tasks in Automatic Speech Recognition (ASR) and speaker diarization for long-form Bengali audio (Dhar et al., 24 Feb 2026).

Setting	Official designation	Primary task
ICDAR 2025	FEw-Shot Text line Segmentation of ancient handwritten documents (FEST)	Pixel-level text line segmentation
ICDAR 2023 SVRD	Baidu-FEST	Zero-shot / few-shot structured text extraction
BUET CSE Fest DL Sprint 4.0	BUET DL Sprint 4.0 organized under BUET CSE Fest	Long-form Bengali ASR and speaker diarization

This distribution of meanings suggests that FEST functions less as a single benchmark identity than as a recurring label for low-supervision or practically constrained evaluation settings.

2. ICDAR 2025 FEST: task definition and dataset design

The ICDAR 2025 FEST competition addresses text line segmentation in ancient handwritten manuscript pages under an explicitly few-shot learning setting. Participants must develop systems capable of segmenting text lines in the U-DIADS-TL dataset using only three annotated images per manuscript for training. The stated motivation is that historical handwritten documents exhibit irregular handwriting, faded or degraded ink, multi-column and non-linear layouts, overlapping lines, and heterogeneous scripts, while large pixel-accurate annotations are expensive and rare (Zottin et al., 16 Sep 2025).

The dataset, U-DIADS-TL—expanded as Uniud - Document Image Analysis DataSet - Text Line—contains 84 color page images drawn from three historical manuscripts: Latin 2, Latin 14396, and Syriaque 341, with 28 pages from each manuscript. Each image has a pixel-accurate GT mask in PNG format, at the same resolution as the source image, and the annotation classes are background and text lines. The ground truth is described as fine-grained, noise-free, and non-ambiguous (Zottin et al., 16 Sep 2025).

The few-shot split is manuscript-specific. For each manuscript, the organizers provided 13 annotated images, of which 3 images constituted the training set and 10 could be used for validation. In addition, 15 private test images per manuscript were withheld and used only for final evaluation. The benchmark is therefore designed to mimic a humanities workflow in which a scholar has only a few manually labeled pages for a new manuscript (Zottin et al., 16 Sep 2025).

A central distinguishing claim is that FEST differs from the earlier ICDAR 2024 SAM competition, which addressed layout segmentation rather than text line segmentation and used U-DIADS-Bib rather than U-DIADS-TL. This distinction matters because FEST evaluates pixel-level delineation of text lines rather than broader region-level layout parsing (Zottin et al., 16 Sep 2025).

3. Evaluation protocol, methods, and results in ICDAR 2025 FEST

Evaluation in ICDAR 2025 FEST used five metrics: Pixel Intersection over Union (PIU), Line Intersection over Union (LIU), Detection Rate (DR), Recognition Accuracy (RA), and F-measure (FM). A predicted line and a ground-truth line are matched by component-wise overlap, and a match is confirmed if both pixel-level precision and recall exceed 75%. The competition computes all metrics individually for each manuscript, and the final ranking is based on the average Line IU score across the three manuscripts (Zottin et al., 16 Sep 2025).

The organizers reported three semantic-segmentation baselines—FCN, PSPNet, and DeepLabV3+—to contextualize submissions. The paper states that most competition entries outperform these baselines, indicating that the few-shot historical-document setting benefits from task-specific architectures, post-processing, or layout-aware decomposition rather than naïve transfer of conventional supervised segmentation models (Zottin et al., 16 Sep 2025).

A total of 11 teams submitted results. The submissions covered a broad methodological spectrum. CV-Group used HookFormer, a multiscale Transformer with a context branch, a target branch, a feature hooking mechanism, and Cross-Attention Swin-Transformer blocks, followed by a positional refinement stage based on Mask R-CNN pretrained on READ-BAD. TAU-CH used a two-stage classical pipeline built from Otsu thresholding, morphological processing, anisotropic Gaussian filtering with elongation factor $\eta = 3$, and watershed segmentation. SRCB used SegFormer with post-processing and explicitly compared Dice loss with Lovász-Softmax loss. VAI-OCR used a DBNet-style two-stage pipeline, with a first stage for shrunk text line extraction and a second stage for full text line segmentation. GPI used a fully deterministic classical pipeline with only one learned parameter: the maximum connected-component area (Zottin et al., 16 Sep 2025).

The winning team was PERO. Its system used a two-stage pipeline consisting of layout analysis and post-processing. The first stage used ParseNet, trained on the PERO layout dataset, to predict baselines, distances from baseline to top/bottom of bounding polygon, and text region borders. The second stage applied adaptive Gaussian thresholding, baseline rendering to bridge fragments, morphological dilation, and connected-component filtering. On the final ranking, PERO achieved PIU 0.803, LIU 0.961, DR 0.815, RA 0.849, and FM 0.830. The paper states that this LIU = 0.961 outperformed SRCB by 3.7% and CV-Group by 6.4% (Zottin et al., 16 Sep 2025).

The remaining ranking was: SRCB second with LIU 0.924; CV-Group third with LIU 0.897; GPI fourth with LIU 0.856; CV-Lab fifth with LIU 0.792; TAU-CH sixth with LIU 0.790; VAI-OCR seventh with LIU 0.747; DIA-Group eighth with LIU 0.572; BBA ninth with LIU 0.219; Codecrackers tenth with LIU 0.203; and LDLD eleventh with LIU 0.000. The paper also highlights that CV-Group had strong LIU but comparatively low RA and FM, implying that line-count consistency and one-to-one matching remained difficult even when overlap metrics were strong. It further highlights GPI as evidence that carefully designed classical vision methods can remain competitive in this setting (Zottin et al., 16 Sep 2025).

4. Baidu-FEST in ICDAR 2023 SVRD

Baidu-FEST was one of two tracks in the ICDAR 2023 Competition on Structured text extraction from Visually-Rich Document images (SVRD). Its purpose was to evaluate the performance and generalization of Zero-shot / Few-shot Structured Text extraction from an end-to-end perspective. The benchmark targets enterprise-like scenarios, especially documents from finance, insurance, logistics, and customs inspection, with formats including cards, receipts, and forms (Yu et al., 2023).

The benchmark contains about 11 kinds of synthetic business documents for training and 10 types of real visually-rich document images for testing, with about 60 images per document type. Each image is annotated with text-field bounding boxes (bboxes), transcript, entity caption, and entity id for each text bbox. The geometry of each box is represented by four vertices in clockwise order starting from the top (Yu et al., 2023).

Baidu-FEST defined two tasks. Task 3, E2E Zero-shot Structured Text Extraction, required extraction of texts for predefined key fields from document images where the training and testing scenarios are disjoint. Task 4, E2E Few-shot Structured Text Extraction, used the same output format but additionally provided localization information and transcripts, while limiting each test scenario to no more than five images per scenario for adaptation. In both tasks, the ground-truth captions caption_en and caption_ch could be used as prompts but cannot be modified (Yu et al., 2023).

Evaluation combined Normalized Edit Distance (NED) and F-score into a final ranking score,

$score = 0.8 \cdot score1 + 0.2 \cdot score2$

where score1 is the NED-based score and score2 is the F-score. The paper emphasizes that the benchmark evaluates the full extraction pipeline rather than isolated OCR or field-labeling modules (Yu et al., 2023).

Participation was moderate but nontrivial. For the full competition, the paper reports 50 participants and 117 submissions. For Baidu-FEST specifically, Task 3 received 7 participants and 15 submissions, and Task 4 received 8 participants and 11 submissions (Yu et al., 2023).

5. Systems, outcomes, and limits in Baidu-FEST

For Task 3: Zero-shot, the top three teams were USTC + iFLYTEK AI Research with 78.71%, TencentOCR with 77.35%, and Ant Group with 71.44%. The champion system used DBNet-det + SVTR-rec for OCR, manually ordered boxes, concatenated OCR text into a sequence, and fed that sequence into a seq2seq model trained with multiple BERT-family variants under UniLM; the method also used augmentations such as random text replacement/erasing and box scaling/shifting, and for English documents incorporated DocPrompt outputs (Yu et al., 2023).

For Task 4: Few-shot, the top three teams were TencentOCR with 84.43%, USTC + iFLYTEK AI Research with 82.13%, and CMSS with 74.90%. The champion system used a LayoutLMv3 + StrucTextv1 backbone, pretrained-model fine-tuning, OCR preprocessing to merge or split badly detected boxes, self-supervised training for segment-based classification on some folders, rule-based postprocessing, and model ensembling (Yu et al., 2023).

The paper stresses that strong aggregate scores should not be mistaken for solved generalization. It explicitly states that there is still a large performance gap for complex and zero-shot structured extraction. The reported per-scenario results of the Task 3 champion dropped to 53.25% on Letter/Email and 46.2% on Technical Report; in Task 4, performance dropped to 59.23% on Car Ticket. The authors interpret these results as evidence that current systems may overfit easier document patterns and remain weak on harder unseen document types (Yu et al., 2023).

This establishes Baidu-FEST as a benchmark oriented toward scenario-level robustness, generalization to unseen document schemas, and end-to-end extraction quality. A plausible implication is that the track exposed limitations not visible in traditional OCR-centric benchmarks, particularly when training and testing document types are intentionally separated.

6. Related “Fest” usages and broader significance

A separate competition context appears in BUET CSE Fest DL Sprint 4.0, where the tasks were long-form Bengali ASR and speaker diarization rather than document analysis. The system description paper states that the competition focused on long-form Bengali speech understanding, with realistic recordings containing background noise, music contamination, silence intervals, overlapping or alternating speakers, and conversational variability. The described ASR pipeline combined Demucs (htdemucs) vocal separation, Silero VAD, a gap aware windowing strategy with about 20-second windows and a 5-second silence threshold, Context Padding ($\pm 1s$), Whisper-Medium decoding, and text normalization. For diarization, the final system fine-tuned pyannote/segmentation-3.0 on the official competition dataset and reported a best Private DER: 0.28816 (Dhar et al., 24 Feb 2026).

This speech benchmark is not the same as ICDAR 2025 FEST or Baidu-FEST, but it shows that Fest can designate a broader event umbrella under which a technical competition is organized. That naming overlap is a recurrent source of ambiguity.

A second source of ambiguity is terminological rather than competitive: FEST is also the acronym for Feature Engineering with Self-evolving Trees, an interpretable feature-engineering framework for raw text and images, not a competition. That work reports results in 17 of 20 classifier-task combinations, a mean gain of 4.2 pp over the strongest baseline across five classifiers, and introduces the BrandGuide dataset with 1M+ assets across 2,683 brands, but it is methodologically unrelated to the competition benchmarks that use the same label (Khurana et al., 7 Jun 2026).

Taken together, the competition literature indicates that the most established meaning of FEST Competition is the ICDAR 2025 benchmark on few-shot text line segmentation of ancient handwritten documents, while Baidu-FEST denotes a distinct ICDAR 2023 track on zero-shot and few-shot structured text extraction, and BUET CSE Fest denotes an event context for a speech competition rather than a FEST acronym. This suggests that any technical discussion of “FEST Competition” requires immediate disambiguation by domain, host event, and task definition.