IDSEM Dataset Overview

• IDSEM is an ambiguous term referring to distinct datasets: one for pixel-level industrial smoke segmentation and another for synthetic Spanish electricity invoice extraction.
• The IJmond Industrial Smoke Segmentation Dataset supports environmental monitoring with precise pixel-level annotations distinguishing low and high-opacity smoke using rule-based cropping from fixed camera views.
• The Invoices Database of the Spanish Electricity Market offers a synthetic corpus of 75,000 invoices, enabling document information extraction research with fine-grained semantic labeling and robust few-shot prompting evaluations.

In recent arXiv usage, “IDSEM Dataset” is an ambiguous designation rather than a single universally fixed resource. One report explicitly identifies the IJmond Industrial Smoke Segmentation Dataset, a real-world environmental-monitoring dataset for pixel-level industrial smoke segmentation from fixed cameras in the Netherlands, and notes that this dataset is referred to as IDSEM (Hsu et al., 24 Mar 2026). A separate document-information-extraction paper uses IDSEM to denote the Invoices Database of the Spanish Electricity Market, a public synthetic corpus of Spanish electricity invoices (Gómez et al., 1 Apr 2026). Consequently, precise citation requires the full dataset name and domain, not the acronym alone.

1. Terminological scope and disambiguation

The acronym IDSEM currently appears in at least two unrelated dataset contexts. In the environmental-vision literature, it denotes the IJmond Industrial Smoke Segmentation Dataset. In business-document extraction, it denotes the Invoices Database of the Spanish Electricity Market. This suggests that the acronym is not stable across domains and should be disambiguated by full title, task, and provenance (Hsu et al., 24 Mar 2026, Gómez et al., 1 Apr 2026).

Usage of “IDSEM”	Full name	Domain
Environmental monitoring	IJmond Industrial Smoke Segmentation Dataset	Industrial smoke segmentation
Document information extraction	Invoices Database of the Spanish Electricity Market	Spanish electricity invoice extraction

A further naming complication appears in work on Easy-to-Read civic text simplification: that paper states that the dataset is the iDEM Corpus, not “IDSEM,” and treats “IDSEM” as a likely mistaken variant or confusion with iDEM rather than an official dataset name (Bott et al., 5 Mar 2026).

2. IJmond Industrial Smoke Segmentation Dataset

The IJmond Industrial Smoke Segmentation Dataset is a dataset for pixel-level industrial smoke segmentation in real-world environmental monitoring imagery. Its purpose is to support models that identify which pixels belong to industrial smoke emissions, rather than only predicting whether smoke is present in an image. The dataset is designed for computer-vision research on industrial smoke detection and segmentation, especially for stationary camera monitoring of heavy industrial facilities and local air-pollution observation. It is published on Figshare with DOI 10.21942/uva.31847188 and released under CC BY 4.0 (Hsu et al., 24 Mar 2026).

The images come from three stationary cameras operated by FrisseWind.nu in the IJmond region in the Netherlands. The camera views are named kooks_1, kooks_2, and hoogovens_6_7; these names also appear as filename prefixes. The authors state that the images were scraped directly from the cameras with consent from FrisseWind.nu. The collection setting is therefore a real outdoor industrial-surveillance and environmental-monitoring setup rather than a synthetic or laboratory corpus.

The dataset contains both raw images and cropped images. The reported composition is:

• 900 raw images
• 1209 polygons across the raw images
• 7 raw images have no smoke
• 2074 cropped images
• 1109 cropped images have smoke and therefore have meaningful pixel-level masks

Annotation was performed in Roboflow with the smart polygon tool. The workflow used points as prompts to the Segment Anything Model (SAM) to obtain an initial mask, followed by manual refinement. The first and second authors labeled the data jointly, and the first author manually checked and edited all segmentation masks. The report further notes that the first author had prior experience in industrial smoke detection and training in EPA Method 9 smoke reading.

A distinctive feature is the opacity-aware labeling scheme. The annotations distinguish:

1. low opacity smoke, corresponding to less than 50% opacity according to EPA Method 9
2. high opacity smoke, corresponding to more than 50% opacity

This allows treatment as a three-label pixel classification problem: background, low-opacity smoke, and high-opacity smoke. Masks are stored as uint8 images with the encoding

$$M(x,y)= \begin{cases} 0 & \text{background} \ 155 & \text{low-opacity smoke} \ 255 & \text{high-opacity smoke} \end{cases}$$

where black denotes background, gray denotes low-opacity smoke, and white denotes high-opacity smoke.

3. Data organization, cropping, and split protocols for the smoke dataset

The smoke dataset preserves both polygonal and rasterized segmentation structure. Although the principal task is pixel-level segmentation, the original annotations are polygons, so the data can also be used for polygon-based image segmentation. After annotation, the raw images were cropped into smaller images using a rule-based strategy derived from Project RISE, based on knowledge of where smoke emissions frequently occur. The metadata example in the report shows a crop with width = 900 and height = 900; crop coordinates are defined from the top-left origin, which is also the coordinate origin for the original images (Hsu et al., 24 Mar 2026).

The metadata includes fields such as original filename, cropped filename, corresponding masks, crop location, dimensions, view_id, and camera_name. The paper states that the metadata fields cropped_npy_name and cropped_mask_npy_name should be ignored for normal use.

Two official split protocols are provided:

• Split by timestamp: train is the first 70%, validation the next 10%, and test the final 20% of cropped images ordered by timestamp.
• Split by camera: train is the first 80% of kooks_2 sorted by timestamp, validation is the remaining 20% of kooks_2, and test is all images from hoogovens_6_7 and kooks_1.

The second protocol is explicitly a cross-camera generalization setup and is intended for benchmarking domain shift across viewpoints and camera locations.

Within each split, the data are further separated into with mask and without mask subsets. The report emphasizes that this does not mean labeled versus unlabeled: even images without smoke have mask files, but those masks are all-black PNGs. The organization is intended to support experiments using only positive examples or mixtures of positive and negative examples.

The training set also has five reduced-data variants: 100%, 80%, 60%, 40%, and 20%. These are defined as the last portions of the training set after timestamp sorting, so that the time line remains continuous when combined with validation and test.

The file structure centers on ijmond_seg. Under the data directory are:

• raw images in JPG
• raw masks in PNG
• polygon annotations in MS COCO JSON
• a cropped directory with cropped images, cropped masks, split-definition text files, metadata.json, and text files listing image-mask pairs

Accordingly, the principal formats are JPG for images, PNG for masks, JSON for metadata and COCO polygons, and TXT for pair listings.

4. Operational framing, intended uses, and limitations of the smoke dataset

The smoke dataset is framed as addressing a need that differs from generic smoke or fire datasets: the imagery comes from fixed outdoor cameras observing industrial emission sources. The cropping strategy is motivated by earlier work, especially Project RISE, and uses domain knowledge about where emissions typically occur. This positions the dataset for operationally relevant monitoring scenarios in which emissions arise from known plant structures and fixed viewpoints (Hsu et al., 24 Mar 2026).

The intended tasks are clear even though the report does not define an official leaderboard or provide numerical baselines. These tasks include:

• pixel-level industrial smoke segmentation
• polygon-based segmentation
• binary smoke/background segmentation
• multi-class segmentation by opacity
• robustness under temporal split
• robustness under cross-camera split
• experiments under limited training data

The report does not specify official evaluation metrics and provides no baseline model results.

Several limitations are explicit. First, the opacity labels are uncertain under unfavorable lighting and sun-position geometry. The annotator attempted to apply the EPA Method 9 rule, but the report cautions that the assigned low/high opacity class may differ from real-world opacity, especially near the 50% boundary. Second, the dataset is restricted to three fixed cameras in one geographic region, so transfer to other sites, climates, or camera setups may be limited. Third, the cropping policy uses domain knowledge about likely emission regions, which is operationally useful but may bias models toward expected locations. Fourth, only 7 raw images have no smoke, whereas the cropped set contains many more negative crops, so class balance differs substantially between raw and cropped representations. A plausible implication is that raw-image and crop-level experiments probe somewhat different class-prior regimes.

On legal and ethical aspects, the report states that collection occurred with consent from FrisseWind.nu and that the release is under CC BY 4.0. The dataset is tied to environmental monitoring and public-interest air-quality observation, and the paper does not discuss human-subject or privacy issues, presumably because the imagery focuses on industrial facilities and emissions.

5. IDSEM as the Invoices Database of the Spanish Electricity Market

In a different literature, IDSEM denotes the Invoices Database of the Spanish Electricity Market, a publicly available dataset of 75,000 synthetically generated invoices in PDF format designed to resemble real commercial electricity bills from the Spanish market. The dataset was created to support structured information extraction while avoiding the privacy barriers associated with real invoices. Its generation pipeline comprises a simulation module that generates invoice field values from statistical distributions derived from official Spanish regulatory bodies, a population system that fills DOCX invoice templates using placeholder codes such as {A1}, and conversion of the populated documents to PDF. The templates were based on real invoices from Iberdrola, Endesa, Naturgy, EDP, and Repsol (Gómez et al., 1 Apr 2026).

The full dataset is organized as:

• 30,000 documents in a training directory across 6 templates
• 45,000 documents in a test directory across 9 templates
• 3 of the test templates are unseen relative to training

The LLM extraction study using this dataset does not use the full corpus. Instead, it selects 400 random invoices from each of the 6 training templates, for a total of 2,400 documents, because the test set has no publicly available ground truth labels.

The extraction schema is unusually fine-grained. The paper distinguishes 86 distinct semantic concepts and 107 extractable labels, because some concepts have multiple representational variants, especially dates and monetary or rate values. The 86 concepts are grouped into 12 thematic categories: customer receiving the invoice, contract-stated customer data, marketer information, distributor information, contract details, general invoice information, customer financial information, energy consumption data, invoice breakdown summary, detailed invoice breakdown, equipment rental charges, and taxes and fees. Concrete examples include A1 for customer name, B1 for customer name in the contract, J5 for total invoice amount, F4l/F4s/F4p for billing-start date variants, and K2/K2d for annual versus daily rate representations.

The study processes invoices through a text-only pipeline. PDFs are converted to Markdown, motivated by the claim that Markdown better preserves headers, tables, formatting markers, and hierarchical relations than plain text. The authors explicitly note several losses in this conversion: text in margins, embedded images, and graphical elements may not be extracted. They therefore construct an explicit template-to-label mapping so that models are not penalized for omitting labels absent or unextractable in a given template.

Prompting, rather than fine-tuning, is the primary experimental variable. Two general-purpose LLMs are evaluated: Gemini 1.5 Pro and Mistral-small. Across six prompting strategies, the best results come from cross-template few-shot prompting. The reported best F1 values are 97.61% for Gemini and 96.11% for Mistral-small under Cross-valid_v1. The paper further reports that, for Gemini, the maximum F1 variation across 19 decoding settings is only 0.58 points, whereas the difference between zero-shot and the best few-shot strategy is 19.22 percentage points. The study therefore uses IDSEM to argue that prompt design dominates hyperparameter tuning in this extraction setting.

The paper also identifies the main source of difficulty as document template structure. In zero-shot evaluation, T1 is the easiest template and T5 the hardest; the explanation given is that T1’s more tabular layout survives PDF-to-Markdown conversion better, whereas T5’s dense multi-column layout degrades more severely. Hard labels include C9, CD, E8, K2/K2d, K4/K4d, M3/M3d, and DD, due to low frequency, semantic ambiguity, and poor preservation under conversion.

6. Related methodological and naming issues

Not all papers mentioning “IDSEM” are actually about an IDSEM dataset. The paper on dataset-level membership inference via Semantic Correlation Descriptors (SCDs) does not study a dataset named IDSEM and contains no experiments on IDSEM. Its relevance is methodological: given white-box model access, a target dataset, and a standalone reference model trained on that dataset, it proposes a way to test whether the target dataset contributed to a model’s training mixture (Gobeaja et al., 28 May 2026). This is therefore a procedure for asking whether a dataset such as an IDSEM candidate was used in training, not evidence about any IDSEM dataset itself.

A separate simplification-corpus paper makes the terminological problem explicit from the opposite direction. It describes a multilingual resource created within the iDEM project and states that the dataset is the iDEM Corpus, not “IDSEM,” treating “IDSEM” as a likely mistaken variant, alternate reference, or confusion with iDEM rather than an official dataset name (Bott et al., 5 Mar 2026). Taken together with the smoke-segmentation and invoice-extraction usages, this indicates that “IDSEM Dataset” is not a self-disambiguating phrase in current literature.

The principal encyclopedia-level consequence is that any reference to IDSEM should specify at least the full dataset name, domain, and ideally the associated paper or repository. Without that disambiguation, the term may refer either to a real-world industrial smoke segmentation dataset from the IJmond region or to a synthetic Spanish electricity-invoice extraction dataset, and some searches may additionally conflate it with the unrelated iDEM Corpus.