IOBES Library for Span-Level Processing

• IOBES Library is a Python tool that standardizes parsing of token-level labels into coherent spans using schemes like IOB, BIO, IOBES/BILOU, and BMEWO.
• The library uses a single-pass, linear-time algorithm to convert labels to spans and enumerate legal transitions, ensuring fair evaluation of NLP tasks.
• It enforces explicit policies for handling malformed sequences and supports reproducible span-level metrics such as precision, recall, and F1.

Searching arXiv for the specified paper to ground the article. iobes is a small, dependency-free Python library for span-level processing when spans are represented as token-level labels. It was introduced to standardize parsing sequences of tags into spans, converting among common labeling schemes, and enumerating legal tag transitions in tasks such as named entity recognition and slot-filling, where token tags are post-processed into labeled text spans. The library addresses a specific reproducibility problem: inconsistent handling of schemes such as IOB, BIO, IOBES/BILOU, and BMEWO, together with divergent policies for malformed sequences, can yield different predicted entities and therefore different precision, recall, and $F1$ scores on the same tag sequences. iobes makes these policies explicit, reusable, and testable so that span-level metrics are fair and comparable (Lester, 2020).

1. Motivation and problem setting

Many NLP tasks require identifying and labeling contiguous spans of text rather than classifying tokens in isolation. In common practice, these tasks are recast as sequence labeling problems in which each token receives a label prefixed by markers such as B- or I-, and the final spans are recovered by post-processing the token-level outputs. Properly parsing these annotations is therefore not an auxiliary implementation detail; it directly determines the entities that are counted during evaluation (Lester, 2020).

The problem that iobes addresses is the absence of an easy-to-use, standardized, programmatically integratable library for this span reconstruction layer. Datasets and codebases often mix labeling schemes or implement different entity-resolution policies for malformed sequences. The data specifically contrasts the behavior of conlleval.pl, which implicitly splits spans when the type changes, with NCRF++, which ignores illegal tags inside a span and only honors B and E. Because these choices can produce different entities from the same predicted tags, they can also produce incomparable metrics across papers.

Standardization matters most when model improvements are small or when datasets are converted across schemes. A plausible implication is that discrepancies attributed to architecture or training can in some cases originate from evaluation tooling instead. iobes is designed to reduce this source of silent metric drift by providing a tested implementation of parsing, conversion, validation, and legality checks.

2. Labeling schemes and span formalization

iobes supports IO(B), BIO, IOBES, BILOU, and BMEWO. It unifies their handling through a scheme descriptor, SpanFormat, which specifies strings for BEGIN, INSIDE, [END](https://www.emergentmind.com/topics/eccentric-nuclear-disk-end), and [SINGLE](https://www.emergentmind.com/topics/smooth-incremental-graphical-lasso-estimator-single), allowing one code path to operate across multiple schemes (Lester, 2020).

The tag semantics are scheme-agnostic. O denotes a token outside any span. B-X begins a span of type X. I-X marks an interior token of a span of type X, excluding first and last positions in schemes that also use explicit end markers. E-X marks the last token of a span in IOBES/BILOU, while S-X marks a single-token span in IOBES/BILOU. In BILOU, L- corresponds to E- and U- corresponds to S-; in BMEWO, M- replaces I- and W- replaces S-. The data notes that IOB1 uses I-X by default and reserves B-X for separating adjacent touching spans of the same class, whereas BIO requires every span to begin with B-X.

The library formalizes a span as

$S = (i, j, c),$

where $i$ is the inclusive start token index, $j$ is the exclusive end token index, and $c$ is the class label. This matches the Span NamedTuple with fields type, start, end, and tokens. The use of exclusive end indices enables simple slicing. The mapping from labels to spans is defined as a single left-to-right pass

$f(T) \to S,$

where $T$ is a sequence of length $n$ of token-level tags. The output is a set of non-overlapping, contiguous spans determined by scheme-specific rules and an explicit policy for malformed sequences.

Nested or overlapping spans are not represented by these schemes and are therefore disallowed. iobes outputs flat, non-overlapping spans only. This restriction is a property of the supported tagging formalisms rather than an incidental limitation of the implementation.

3. Parsing semantics and malformed-sequence policy

The core parsing procedure is scheme-aware and operates in one pass. It initializes current_span = None and spans = [], then processes each token at index $k$ by decomposing tag[k] into a prefix in O/B/I/E/S and a type label when present. Valid and invalid transitions depend on the selected scheme, and the library both enforces valid transitions and records errors when they are violated (Lester, 2020).

A defining design choice is its treatment of malformed sequences. For parsing, iobes follows the conlleval.pl policy for entity resolution. The rule of thumb is that a change in type triggers span termination and a new span start. If a span of type C is currently open and the next tag is I-X, B-X, E-X, or S-X with $X \neq C$, the parser closes the span of type C at the previous token and begins a span of type X when appropriate for the observed prefix, while logging the transition as erroneous. This produces spans even from illegal sequences, allowing evaluation to proceed without silently discarding predictions.

The data enumerates common error conditions: I-X after O in BIO; I-X after a different type; E-X without a preceding B-X or I-X of the same type; spans that end without an E-X in IOBES/BILOU; and starting a span with I-X in BIO, IOBES, or BILOU. At sequence end, any open span is closed, and the parser logs an “unterminated span” error if the scheme expects an E- marker.

The pseudocode given for an IOBES parser makes these rules concrete. O closes any open span; S closes any open span and yields a singleton span; B closes any current span and opens a new one; I starts a span if none is active, or terminates and restarts on a type switch; E emits a singleton if there is no matching open span, or closes the active span if the type matches. The computational notes specify single-pass execution with $S = (i, j, c),$0 time and $S = (i, j, c),$1 additional space aside from outputs.

This permissive behavior applies to malformed predictions, not to gold data conversion. The distinction is central to the library’s semantics: evaluation should remain possible on ill-formed model outputs, but gold annotations should not be mutated through hidden repair rules.

4. Conversion, validation, and legality constraints

iobes supports both directions of transformation: spans to labels and labels to spans. For spans to labels, the output depends on the target scheme. In IOB, tokens inside $S = (i, j, c),$2 receive I-c by default, except that if two spans touch and share class $S = (i, j, c),$3, the second begins with B-c. In BIO, each span begins with B-c and continues with I-c. In IOBES/BILOU, length-one spans use S-c, and longer spans use B-c at the start, E-c at $S = (i, j, c),$4, and I-c in the interior. BMEWO follows the same pattern but uses M- instead of I- and W- instead of S- (Lester, 2020).

Scheme conversion is defined by parsing a valid source sequence into spans and re-emitting labels in the target scheme. For valid sequences, this mapping is deterministic. For malformed sequences, however, the library adopts a strict policy when converting gold labels: it raises an exception and returns the collected errors rather than silently imposing a repair policy. The stated rationale is that humans can fix gold data, and silently deciding a conversion policy would be overly prescriptive and could mutate the dataset in hidden ways.

The library also enumerates legal tag-to-tag transitions for each scheme and can return a CRF-ready transition mask. This functionality serves decoding-time constraint enforcement: illegal transitions can be masked so that a structured decoder such as a CRF cannot emit malformed sequences. In this respect, iobes is not limited to post hoc evaluation; it also supports prevention of ill-formed outputs during inference.

Automatic repair is explicitly not performed. The library exposes errors and legality so that practitioners can enforce constraints or correct data, but it does not silently fix sequences. This is an important point of interpretation because “standardization” in iobes refers to explicit policies and interfaces, not automatic normalization of defective annotations.

5. Evaluation semantics and reproducibility

The recommended evaluation metric is exact span match $S = (i, j, c),$5 at the span level: both span boundaries and type must match exactly. Partial credit is not described or provided. If TP is the number of predicted spans that exactly match a gold span in both $S = (i, j, c),$6 and $S = (i, j, c),$7, FP is the number of predicted spans with no exact gold match, and FN is the number of gold spans with no exact predicted match, then

$S = (i, j, c),$8

These formulas are given directly in the source description (Lester, 2020).

The justification for span-level evaluation is that token-level accuracy can obscure structurally important errors. The data gives the example of predicting I-MISC within an ORG span: token labels may appear locally plausible while the recovered entity boundaries are wrong. Because different entity-resolution policies can produce different span sets from the same tag sequence, standardized span parsing is necessary for fair comparison.

The practical guidance emphasizes several reproducibility norms. Gold data should be validated and corrected rather than silently repaired. The same parser and scheme should be used across gold and predictions. Reports should specify exact span match $S = (i, j, c),$9 together with the scheme and policy, namely iobes parsing with conlleval-style entity resolution. Conversions should be performed once and cached, and round-trips from labels to spans and back should be checked for losslessness on the dataset. Versioning converted datasets and recording the scheme used are recommended because small differences, such as IOB versus BIO, can change evaluation.

This suggests a broader methodological role for iobes: it operationalizes evaluation policy as part of the experimental protocol, rather than leaving it implicit in task-specific scripts.

6. API, interoperability, comparisons, and limitations

The core API revolves around Span, SpanFormat, parsing functions such as parse_spans_<scheme>(tags) and parse_spans(tags, span_type=...), conversion functions such as convert_<schemeA>_to_<schemeB>(tags), and enumerate_legal_transitions(span_type), which returns a legality map or mask suitable for decoding constraints. Typical input is a list of string tags such as ["B-ORG", "I-ORG", "E-ORG", "O", ...], and typical output is List[Span] together with List[Error], where the error types described include type switches, illegal starts, illegal ends, and unterminated spans (Lester, 2020).

The engineering notes state that the library is dependency-free, uses property-based tests and fuzzing, and offers both scheme-specific interfaces and generic dispatchers. The paper does not report runtime benchmarks, but parsing and conversion are described as single left-to-right passes with $i$0 complexity, while legal-transition enumeration is $i$1 over the label set and is performed once per scheme. For interoperability, the paper emphasizes CoNLL-style histories and common datasets such as CoNLL-2003 and WNUT-2017, but it does not specify dataset import or export utilities; the intended use is to place the parser or converter around an existing data loader.

The comparison with existing tools is specific. conlleval.pl is treated as the de facto CoNLL evaluator, and its “type change = span shift” policy is adopted for parsing malformed sequences so that results remain comparable. NCRF++ is described as evaluating only legal spans using B/E boundaries and ignoring illegal I-type changes, which can yield different entities from the same tags. seqeval and spaCy/Flair utilities are acknowledged as common evaluation or I/O helpers, but the paper’s emphasis is that their policies often differ and are often implicit. iobes distinguishes itself by making policies explicit and testable, providing cross-scheme conversion with strict error signaling on malformed gold, enumerating legal transitions for constraint enforcement, and returning structured errors to help fix data.

The illustrative examples reinforce the library’s scope. One malformed IOBES prediction, B-ORG I-MISC E-ORG, is evaluated differently under conlleval-style and NCRF++-style policies; iobes follows the former, which yields multiple predicted spans that do not match gold. Another example concerns a data conversion bug from IOB1 to IOBES, where the sequence I-MISC B-MISC I-MISC was misconverted into three singletons in a third-party dataset. These examples motivate rigorous, standardized conversion and parsing.

The limitations are explicit. iobes supports flat spans only; nested and overlapping entities are outside the representational capacity of the supported schemes. It does not perform automatic repair. The paper does not claim partial-match metrics, nested entity handling, or dataset import/export utilities. It also states that specific installation commands and the open-source license are not provided in the paper, although the library is described as open-source and dependency-free.

Taken together, these characteristics define iobes as infrastructure for exact, scheme-aware span processing rather than a general-purpose annotation platform. Its central contribution is to make parsing, conversion, validation, and legality constraints explicit at the span level, thereby reducing discrepancies between model outputs, evaluation scripts, and published metrics (Lester, 2020).