---
title: Extended Burrows-Wheeler Transform (eBWT)
url: https://www.emergentmind.com/topics/extended-burrows-wheeler-transform-ebwt
type: topic
---

# Extended Burrows-Wheeler Transform (eBWT)

The extended Burrows–Wheeler Transform (eBWT) generalizes the classical Burrows–Wheeler Transform from single-string input to collections or multisets of strings, graphs, or even more general combinatorial structures. This extension preserves and amplifies many of the combinatorial, algebraic, and algorithmic properties that make the original transform central in text indexing, compression, and bioinformatics, especially in settings where data repetitiveness and structural complexity are paramount. The eBWT underlies a wide spectrum of advanced data structures, including those for reference-free genomic analysis, suffix and Wheeler graph-based indexes, and compressed representations of highly repetitive datasets.

## 1. Mathematical Foundations and Formal Definitions

The eBWT for a multiset of strings $M = \{S^1, S^2, ..., S^k\}$ over a totally ordered alphabet $\Sigma$ is constructed by considering all conjugates (cyclic rotations) of the strings in $M$, forming a collection $C$, and sorting $C$ in infinite periodic order $<_\infty$, whereby $U <_\infty V$ iff $U^\infty$ is lexicographically smaller than $V^\infty$. The transform $eBWT(M)$ is the sequence of characters that precede each sorted conjugate in this order, yielding a string of length $N = \sum_{S\in M} |S|$ [2504.19123, 1901.08392, 2506.04926].

For string collections, the framework can also be formulated via the generalized suffix array (GSA) and the multi-string BWT:
- GSA$[i] = (j, r)$ indicates that the $i$-th smallest suffix (over all augmented strings $S_r\$ _r$) starts at position $j$ in $S_r$.
- $eBWT[i]$ is $S_r[j-1]$ if $j>1$ and $\$_r$ if $j=1$ [1805.01876].

The construction extends smoothly to more general objects. On Wheeler graphs, the eBWT is the string of edge-labels in Wheeler order [1811.02457], and further generalizations (e.g., tree-structured input, see XBWT) are available for labeled trees [1809.07320].

## 2. Algorithmic Construction and Complexity

Multiple algorithmic approaches exist for eBWT construction across different settings:

- **GSA-based:** For a collection of $m$ strings, compute the GSA and corresponding LCP array in $O(P)$ time and space, with $P = \sum_i |S_i|$ [1805.01876].
- **Suffix array adaptation:** For primitive factors, one can canonicalize each input string to its minimal conjugate, sort them, concatenate, and compute the BBWT by constructing the circular suffix array (CSA) under $<_\infty$ allowing linear-time construction via an SAIS extension [1911.06985].
- **Lyndon grammar approach:** For large, repetitive datasets, grammar compression is leveraged by building Lyndon straight-line programs (SLPs) representing the input corpus. Construction involves three phases: grammar construction (Duval’s algorithm streaming right-to-left), lexicographic sorting of nonterminals (via first-symbol forests), and eBWT extraction by enumerating conjugates in infinite periodic order. Time and space can be brought to $O(N+g)$ when the grammar size $g$ is much smaller than input length $N$, and practical implementations can further run-length encode the BWT for additional compression [2504.19123].
- **Dollarless and tree-based:** For aligned readsets, eliminating separator symbols lowers run count. The XBWT model removes explicit symbols by leveraging the structure of a labeled tree that encodes reads and their alignments, with sorting by upward-path labels replacing suffix arrays. The overall cost becomes $O(m \log m)$, where $m$ is the number of edges [1809.07320].

Parallelization (e.g., thread-parallel Lyndon parsing), hash-consing of grammar rules, and hybrid in-memory/external-memory schemes enable scalability to datasets comprising tens of billions of characters [2504.19123].

## 3. Structural and Algebraic Properties

The eBWT is bijective on its combinatorial domain (e.g., primitive necklaces, multiset of strings) and supports invertibility akin to the original BWT. The inverse is performed by reconstructing the standard permutation associated to the output via order-preserving partial maps between the sorted and original output positions, followed by decomposing the permutation into its cycles, each corresponding to a primitive in the original input [1901.08392]. This cycle-based inversion generalizes classical BWT inversion and is practical in $O(N\log k)$ time ($N$ output length, $k$ alphabet size).

The algebraic perspective connects the induced partial maps to syntactic semigroups of cyclic languages. Notably, for primitive $u$, $S(u) \cong S_u$ links the eBWT’s induced semigroup to language-theoretic constructs [1901.08392]. Combination with Lyndon factorizations and bijections facilitates combinatorial enumeration of special objects (e.g., de Bruijn words).

For tree- and graph-based models, the eBWT generalizes to Wheeler graphs, where the ordering constraints on vertices and edges yield efficient rank/select-enabled navigation and ensure correctness of FM-index operations after tunneling and other nontrivial transformations [1811.02457].

## 4. Run Count, Decomposition, and RLE Compressibility

Run-length encoding (RLE) size is determined by the number of runs, defined as the number of maximal constant blocks in the eBWT output [2506.04926, 1809.07320]. The choice of decomposition into substrings or tree/graph paths is pivotal:

- The number of decompositions of a word $w$ into factors greater than $k$ is exponential in $|w|$ (generalized Fibonacci growth). There exists a universal upper bound for the minimum run count among $k$-constrained decompositions, and for certain words, the ratio between maximal and minimal run counts across all decompositions is unbounded [2506.04926].
- For genomic readsets, omitting end-markers from the eBWT (dollarless variant) reduces the number of runs by nearly 19%, with tree-based XBWT lowering the run count by a further 15% [1809.07320]. Lower run counts directly improve space usage in run-length FM-indexes and accelerate rank/select queries.

Tunneling techniques, originally for repetitive datasets, can be applied to Wheeler graphs after eBWT construction, further compressing repeated isomorphic blocks beyond what RLE can achieve [1811.02457].

## 5. Applications Across Bioinformatics, Indexing, and Pattern Matching

The eBWT paradigm is exploited across a spectrum of computational biology and text retrieval tasks:

- **Reference-free/Alignment-free genomics:** eBWT supports clustering of reads covering the same genomic locus, with the expected cluster size and boundaries governed by Poisson statistics and determined rapidly via LCP array monotonicity (single scan for cluster detection) [1805.01876].
- **SNP/INDEL detection:** By leveraging the positional clustering effect in eBWT and LCP arrays, small variants and sequencing errors can be detected without prior reference, and filtered via Poisson model expectations [1805.01876].
- **Haplotype phasing, alternative splicing, error correction, assembly overlap detection:** All these tasks benefit from consensus detection inside eBWT clusters, with the general workflow being eBWT+LCP construction, cluster delimitation by LCP local minima, filtering by cluster size/distribution, and local reconstruction [1805.01876].
- **Indexing repetitive databases:** State-of-the-art indexes (e.g., run-length FM-indexes, XBWTs) for highly repetitive sequence collections rely on the eBWT for compressibility and navigation efficiency [2504.19123, 1809.07320].
- **Tree and graph-based indexing:** The generalized eBWT naturally extends to labeled trees (XBWT), Wheeler graphs, and advanced objects supporting efficient counting/locating/extracting [1809.07320, 1811.02457].
- **Pattern matching on circular or structurally complex texts:** The eBWT and its extensions (cBWT, XBWT) can support pattern match queries under generalized equivalences (e.g., Cartesian-tree matching), with succinct data structures and dynamic update capabilities [2411.12241].

## 6. Practical Performance and Benchmarks

Empirical studies demonstrate that grammar-based or Lyndon-grammar-based eBWT construction is feasible at the scale of human chromosomes or entire viral pangenomes, with significant memory and time benefits over classical suffix array-based BWT construction [2504.19123]. For instance:

- For 1000 human Chromosome 19 haplotypes (total ≈$6 \times 10^{10}$ bp): Lyndon grammar eBWT completes in 1444s (2.0 GiB RAM), outperforming libsais, r-pfbwt, CMS-BWT, and ropebwt3.
- For $10^6$ SARS-CoV-2 genomes (≈$3 \times 10^{10}$ bp): eBWT in 782s, using 2.7 GiB.
- Multithreading further reduces runtime to seconds for these datasets while keeping memory <10 GiB [2504.19123].

Run-count reductions (e.g., dollarless eBWT and XBWT) translate to full-text indexes up to 30% smaller with no loss in indexing functionality [1809.07320].

## 7. Limitations, Open Problems, and Ongoing Developments

Fundamental limitations arise from the combinatorial explosion of possible decompositions: brute-force optimization of run counts is infeasible due to exponential growth in decomposition space [2506.04926]. Known decompositions such as the Lyndon factorization are not universally optimal with respect to RLE [2506.04926]. Practical heuristics to discover nearly minimal run decompositions are an open problem.

Tunneling and block-based compression techniques offer additional reductions in index size, especially on Wheeler graphs, but the design and implementation of optimal tunneling algorithms remain active areas of research [1811.02457].

Extensions to dynamic, online, or partial-indexing for eBWT-based data structures (e.g., incremental construction, dynamic insertions in cBWT) are studied and possess succinct space guarantees, but with polylogarithmic slowdowns on update/query [2411.12241].

The algebraic and combinatorial underpinnings of eBWT inversion, its connections to syntactic semigroups, and theoretical properties (e.g., de Bruijn word construction, factor enumeration) continue to enrich both the foundational and applied aspects of the eBWT framework [1901.08392].

Source: https://www.emergentmind.com/topics/extended-burrows-wheeler-transform-ebwt