---
title: Run-Length Compressed BWTs
url: https://www.emergentmind.com/topics/run-length-compressed-burrows-wheeler-transforms-rlbwts
type: topic
---

# Run-Length Compressed BWTs

A run-length compressed Burrows-Wheeler transform (RLBWT) is a succinct, highly repetitive-aware data structure that encodes the Burrows-Wheeler transform (BWT) of a string or a collection of strings via run-length encoding (RLE) of maximal blocks of identical symbols. RLBWTs have become a central mechanism in compressed indexing, genomic data analysis, dictionary compression, and as a bridge between BWT-based and LZ77-based representations. The efficiency of RLBWTs arises from the observation that in highly repetitive texts, the number of runs is orders of magnitude smaller than the input length, enabling near-optimal storage and facilitating compressed algorithms whose working memory, construction time, and query performance scale with the number of BWT runs rather than the raw text size.

## 1. Formal Definition and Key Properties

Given a string $S\in\Sigma^n$ terminated by a unique end-marker (e.g., $\$$), its suffix array $SA[1..n]$ orders all suffixes of $S$ lexicographically. The BWT, $L[1..n]$, is defined as $L[i] = S[SA[i]-1]$, with $S[0]=\$$. A “run” is any maximal interval $L[i..j]=a^e$ ($e=j-i+1$) of a single symbol, delimited so $L[i-1]\neq a$ and $L[j+1]\neq a$ (with sentinels at the boundaries). The number of runs is denoted $R$. The RLBWT is then the sequence $\langle(c_1,\ell_1),...,(c_R,\ell_R)\rangle$, where $c_k$ is the symbol of the $k$th run and $\ell_k$ its length, with $\sum_{k=1}^R \ell_k = n$ [1510.06257].

In highly repetitive data, $R \ll n$, and this compressiveness means that both the representation and downstream computation can often be effected in $O(R)$ or $O(R \log n)$ bits, exponentially smaller than naïve representations.

## 2. Construction Techniques and Algorithms

Efficient RLBWT construction must meet two main objectives: minimize working space (ideally scaling with $R$) and minimize, where possible, dependence on $n$, the text length.

**Dynamic RLBWT Data Structures:**  
The construction algorithm in [1510.06257] maintains a dynamic RLBWT for $\widetilde{S} =$ reverse(\#$T$), supporting rank, select, access, and insert in $O(\log n)$ time, using $O(R \log n)$ bits. It reads $S$ left-to-right, inserting each new character at position $\mathrm{LF}^j(0)$ (using LF-mapping), and maintains run boundaries and bit-vectors marking run starts and per-character run boundaries.

**Complexity:**  
- Time: $O(n \log R)$
- Space: $O(R \log n)$ bits (working space)
- In highly repetitive cases ($R = O(1)$), the space can be $O(\log n)$ bits—exponentially smaller than $n$.

Further improvements leverage static arrays and table abstractions to replace dynamic structures, achieving additional reductions in working memory in practical settings [2202.07885]. The $r$-comp algorithm achieves optimal $O(n + r \log r)$ time and $O(r \log n)$ bits, and supports construction for very large (terabyte-scale) genomes or pangenomic collections.

## 3. Combinatorial Bounds and Compressiveness

The compressiveness of RLBWT is governed by upper bounds relating $R$ to external measures of repetitiveness, in particular, the size $z$ of the LZ77 factorization.

**Core Theorems:**
- For all $T$ of length $n$ and LZ77 size $z$, $R = O(z(\log n)^2)$ [1910.10631].
- For $q$-th power-free $T$ of LZ77 size $z$, $R \leq 73\cdot(\log_2 n)\cdot(z+2)^2$ [2002.06265].
- $R$ and $z$ are always within an $O(\mathrm{polylog}\, n)$ factor of each other.
- For any string $w$ (with $\rho(w)$ the number of original runs), $\rho(\mathrm{BWT}(w)) \le 2 \rho(w)$—the RLBWT never creates more than twice as many runs as the original run-count [2411.11298].

These combinatorial results demonstrate that for any highly repetitive string (where $z \ll n$), RLBWT delivers a succinct, near-optimal compressed representation. This enables compressed indexes (e.g., the $r$-index) to store and query data using only $O(R\,\mathrm{polylog}\,n)$ space.

## 4. RLBWT in LZ77 Computation and Self-Indexing

A central application of RLBWTs is computing the LZ77 factorization in compressed space. The key insight from [1510.06257] is that, after constructing the RLBWT for reverse(\#$T$), one can compute the LZ77 parsing by:

- Maintaining the current phrase-prefix length and BWT interval of the reversed prefix.
- Using at most two SA samples per run (a “suffix-array-sample” structure), enabling extension and location of previous prefixes.
- Performing all necessary checks and updates in $O(\log n)$ time per step, with $O(R \log n)$ bits of workspace.

**Consequences:**  
- LZ77 parsing is available in $O(n \log R)$ time and $O(R \log n)$ bits, so both parsing and indexing are possible in compressed, repetition-aware space.
- Self-indexes that combine an RLBWT with LZ77 and $O(z)$ supplemental pointers can be built in $O(R+z)$ words, which is asymptotically optimal (outputs $z$ phrases and retains $R$ runs).
- For repetitive data, $R$ and $z$ remain small, and both indexing and parsing remain efficient.

## 5. Large-Scale Merging and Scalable Implementation

Handling aggregate datasets (e.g., terabase-scale collections) requires scalable merging of multiple RLBWTs:

- **High-throughput merging:** The algorithm in [1511.00898] partitions a collection into $p$ subcollections, builds the RLBWT of each independently, and then merges them using a succinct, bitvector-mediated merging process. The total time per merge is $O(n t_r)$ where $t_r$ is the time to answer a single rank query; overall the merging is $O((p+t_r) n)$.
- **Practical implementation:** Utilizing block alignment, two-level arrays, memory-mapped buffers, and multithreading allows for the merging of $600$ Gbp/day with only $30$ GB memory overhead, supporting terabase-scale FM-indexes on commodity hardware.
- **Adaptive merging:** More recent advances incorporate measures such as the sum of LCPs at block boundaries to achieve merge times of $\tilde{O}(L + \sigma + R)$, where $L$ reflects the true overlap between subcollections and can be small even for large input [2511.16953].

Table: Complexity Comparison of RLBWT Construction/Merging

| Algorithm              | Time Complexity           | Space Complexity     | Applicability                         |
|------------------------|--------------------------|----------------------|---------------------------------------|
| Dynamic online [1510.06257] | $O(n \log R)$                | $O(R \log n)$ bits   | Streaming input, repetitive texts     |
| r-comp [2202.07885]    | $O(n + r\log r)$         | $O(r \log n)$ bits   | Pan-genomic, large-scale inputs       |
| Sirén merging [1511.00898] | $O((p + t_r) n)$             | $O(r\log n)$ bits    | Terabase-scale collections            |
| Adaptive merge [2511.16953] | $\tilde{O}(L + \sigma + R)$  | $O(R)$               | Sets of circular/repetitive strings   |

## 6. Influence of Alphabet Ordering and Heuristics

The alphabet ordering used during BWT computation strongly affects the number of runs—and hence the compressibility—of the RLBWT. The minimal-run ordering problem is NP-complete and APX-hard [2401.16435].

**Key findings:**
- For small alphabets, exhaustive search is possible; for large $\sigma$, heuristic search is necessary.
- First-improvement local search (using Swap or Insert neighborhoods and a variety of initializations such as ASCII, frequency, or first-appearance order) rapidly improves compressibility, often reducing the number of runs by 1–3 percentage points compared to naive ASCII orderings.
- In practical pipelines, sampling $O(10^3)$ permutations on small text samples can provide near-optimal alphabet orderings, making a significant impact at scale for large datasets.

## 7. Practical Applications and Broader Impacts

RLBWTs underpin state-of-the-art compressed indexes for pan-genomics, large document versioning systems, and other massively repetitive corpora:

- **Reference-free genomics:** Store and index tens of billions of sequencing reads efficiently in-memory [1511.00898].
- **Compressed self-indexes:** Combine RLBWT and LZ77 parsing plus minimal auxiliary data to support efficient locate/extract queries in $O(R+z)$ space [1510.06257].
- **Streaming and online processing:** RLBWTs allow LZ77 parsing and other compressed computations in streaming settings, suitable for one-pass algorithms [1510.06257].
- **Integration with grammar-based indexes:** Hybrid approaches leveraging grammar compression (e.g., GCIS) followed by RLBWT significantly reduce run-count and improve query times, especially for long pattern matches on repetitive data [2110.01181].

The robust relationship between the RLBWT run-count $R$ and LZ77 size $z$ (and other repetitiveness measures) ensures that RLBWT-based methods are provably efficient on all compressible inputs. Theoretical advances (e.g., [1910.10631, 2411.11298]) provide strong guarantees: no more than a polylogarithmic overhead is incurred in the worst case when transforming between BWT and dictionary-based compressors.

---

**References:**
- [1510.06257], [1511.00898], [2202.07885], [1910.10631], [2002.06265], [2110.01181], [2511.16953], [2401.16435], [2411.11298]

Source: https://www.emergentmind.com/topics/run-length-compressed-burrows-wheeler-transforms-rlbwts