Compressed Approximate String Matching

• Compressed approximate string matching is a method for performing efficient error-tolerant searches on compressed data using models like LZ77, LZW, and SLPs.
• It leverages techniques such as split index, FM-index, and periodicity-based decompositions to reduce query time and index space relative to the compressed size.
• This field enables practical applications in genomics and massive document repositories by balancing storage efficiencies with robust approximate matching capabilities.

Compressed approximate string matching refers to the study and design of algorithms and data structures for efficient approximate pattern matching queries (under Hamming or edit distance), where the input text or dictionary is given in a compressed representation, such as LZ77, LZ78/LZW, straight-line programs (SLPs), or as compact indexes. The primary goal is to achieve query time, or index space, dependent on the compressed size rather than the full uncompressed length. This area is central to large-scale text search in genomics, document repositories, and any context where massive storage demands favor compression but real-time or batch search with errors remains crucial.

1. Compression Models and Formal Problem Variants

Compressed string matching arises in several models, most notably:

• Dictionary Setting: Given a collection of strings (dictionary) stored in compressed or compact index form, the goal is to match a query pattern approximately, e.g., dictionary lookups with ≤$k$ mismatches (Cisłak et al., 2015).
• Full-text Indexing: The static text $T$ is compressed (or indexed via space-efficient data structures), and queries are for all substrings at edit distance ≤$k$ from a pattern $P$ (Belazzougui, 2011).
• Grammar-based (SLP) Compression: The text is represented by a context-free grammar of small size, supporting random access, substring extraction, and matching primitives (Bille et al., 2010, Charalampopoulos et al., 2020).
• LZ77 and LZ78/LZW Compression: The text is compressed as a sequence of overlapping (LZ77) or non-overlapping substrings (LZ78/LZW) indexed by previously seen phrases, favoring redundancy-rich sources (Gawrychowski et al., 2013, Gagie et al., 2011).

The central algorithmic objective is to answer approximate pattern matching queries—finding factors or dictionary words within Hamming or edit distance $k$ of a query—while minimizing resources as measured with respect to the compressed representation (e.g., number of rules $n$, number of LZ phrases $z$, or index space $S$).

2. Key Algorithmic Techniques and Data Structures

Contemporary approaches to compressed approximate string matching exploit a combination of combinatorial insights, periodicity, and succinct indexing. Core methods include:

• Split Index with Dirichlet Principle: By partitioning dictionary words or patterns into $k+1$ pieces (for $k$ mismatches), any matching word must match at least one piece exactly. The split index is a hash-based structure mapping pattern pieces to buckets holding "missing" remainder segments, allowing fast verification with effective space-time tradeoffs. To further compact the index, $T$0-gram substitution codes the most frequent substrings, reducing storage by up to 50% in repetitive datasets (Cisłak et al., 2015).
• Compressed Full-Text Indices: FM-indexes or compressed suffix arrays allow efficient exact pattern matching and can be augmented for approximate matching. For $T$1, Belazzougui's scheme combines FM-indexes on $T$2 and $T$3, weak prefix search, compact substitution-list stores (via heavy-light decompositions), and colored range reporting. Parameterized decompositions bound space to $T$4 bits and yield $T$5 query time for constant alphabets (Belazzougui, 2011).
• Grammar-Based Approaches (SLP/Block Graphs): SLP-based representations leverage heavy-path decompositions, biased weighted ancestor structures, and block graphs to enable $T$6 random access and $T$7 substring extraction. Approximate matching is reduced to the matching of $T$8-length overlap regions at each nonterminal, with total runtime dependent on the grammar size $T$9, the uncompressed length $k$0, and the complexity $k$1 of the chosen uncompressed algorithm (Bille et al., 2010, Gagie et al., 2011, Charalampopoulos et al., 2020).
• Periodicity-Based Decomposition: For LZW/LZ78 and SLP compression, periodicity and break arguments partition the problem into "few-break" (non-periodic) and "high-periodicity" (periodic) cases. The number of candidate matches is greatly reduced, with further acceleration via marking, approximate period recovery, and careful pc-string (pattern-compressed) reduction (Gawrychowski et al., 2013, Charalampopoulos et al., 2020).

3. Algorithmic Results and Complexity Bounds

The following table summarizes key bounds for different compression models and metrics as derived from recent works.

Model/Compression	Distance	Time Complexity	Space Complexity	Reference
Dictionary (Split Index)	Hamming	$k$2 (avg.), $k$3 (worst)	$k$4	(Cisłak et al., 2015)
FM-index (full-text, $k$5)	Edit	$k$6 or $k$7	$k$8 bits	(Belazzougui, 2011)
SLP (size $k$9)	Hamming	$P$0	$P$1	(Charalampopoulos et al., 2020)
SLP (size $P$2)	Edit	$P$3	$P$4	(Charalampopoulos et al., 2020)
LZ78/LZW (size $P$5)	Hamming	$P$6	$P$7	(Gawrychowski et al., 2013)
LZ78/LZW (size $P$8)	Edit	$P$9	$k$0	(Gawrychowski et al., 2013)
LZ77-block graph ($k$1 phrases)	Edit	$k$2	$k$3	(Gagie et al., 2011)

Here, $k$4 denotes the compressed size (rules or codewords), $k$5 the number of LZ77 phrases, $k$6 the pattern length, $k$7 the error threshold, $k$8 the output size, and $k$9 the uncompressed length.

4. Structural Insights and Periodicity Arguments

Recent theoretical advances exploit detailed structure theorems. For the fully-compressed setting, it is established that for a pattern $n$0 and windowed text $n$1 (possibly as SLP or LZW), there exists a dichotomy:

• If $n$2 is not approximately periodic (i.e., cannot be closely matched by a short-period repetition), then the total number of $n$3-mismatch (or $n$4-error) matches is at most $n$5, which is asymptotically small compared to the total number of possible positions.
• In the periodic case, all $n$6-error occurrences are highly structured: they begin at defined arithmetic progressions modulo the period, and their total is bounded by $n$7 (Hamming) or $n$8 (edit distance), as formalized in (Charalampopoulos et al., 2020).

These results enable "mark-and-verify" strategies, where candidate matches are rapidly marked (via breaks or repetitive regions) and verified, with the periodic case handled by efficient arithmetic progression enumeration.

5. Implementation Primitives and Compatibility with Succinct Data Structures

Efficient compressed approximate matching is realized via a set of core operations implementable on the compressed representations:

• Extract: Retrieve arbitrary substrings from the compressed text in $n$9 time.
• Longest Common Extension (LCE) and Longest Common Prefix/Suffix (LCP/LCS): Advanced implementations for SLPs and tries facilitate rapid string fragment comparison.
• Internal Pattern Matching (IPM): Return arithmetic progressions of occurrences of $z$0 inside small substrings.
• Compressed Hashing and Marking: Karp-Rabin or special-purpose hashing is used to compare and verify candidates against compressed data.
• Succinct Index Compatibility: Both split index q-gram codes and FM-index-based substitution stores dovetail with succinct structures like wavelet trees or permuterm tries, facilitating integration into compressed search pipelines (Cisłak et al., 2015, Belazzougui, 2011).

6. Experimental Performance and Practical Aspects

Empirical evaluations demonstrate that specialized compressed indexes often outperform generic compressed structures and uncompressed approximations:

• The split index achieves sub-microsecond queries for Hamming $z$1 on multi-megabyte dictionaries, with index size proportional to $z$2 (e.g., $z$3s query time, $z$4 MB index on a $z$5 MB dictionary) (Cisłak et al., 2015).
• With q-gram substitution, index size is reduced by up to 50% on repetitive DNA, with less than 10% increase in query time.
• In the LZW/LZ78 model, the $z$6 (Hamming) and $z$7 (edit) algorithms provide the first practical improvement over $z$8-time solutions for small $z$9 and moderate $S$0 (Gawrychowski et al., 2013).
• FM-index-based approaches offer time-optimal $S$1 performance for $S$2, with tunable space overhead according to the trade-off parameter $S$3 (Belazzougui, 2011).
• Grammar-based random access and approximate searching algorithms demonstrate $S$4 overhead over the compressed size; practical for large datasets with very small SLPs and for batch queries (Bille et al., 2010, Gagie et al., 2011).

7. Extensions, Open Questions, and Limitations

While current techniques offer significant space and time advantages, several directions remain open:

• For higher error thresholds $S$5, the dependence on $S$6 or $S$7 can be prohibitive; improving these exponents or finding parameterized lower bounds is an active topic.
• The methods for compressed regular expression matching and more general automata-based queries are less developed; some progress exists for Ziv-Lempel compressed text [0609085].
• Most frameworks rely on random access primitives (enabled by data structures like block graphs or skip trees); achieving constant-time or external memory-efficient versions is challenging.
• The efficacy of periodicity-based decompositions falls if the input lacks redundancy; in worst-case (non-repetitive) texts, the compressed advantages diminish, and bounds approach those of uncompressed algorithms.
• Compatibility of succinct index techniques with general compressed representations (arbitrary context-free grammars, LZ-End, RLZ) is a promising field for future advancement.

Compressed approximate string matching thus occupies a nexus between combinatorial stringology, data compression, and index data structures, with robust theoretical underpinnings and demonstrable practical effectiveness. Major algorithmic milestones delineate the landscape, with unified approaches now extending periodicity-based reasoning to both uncompressed and fully-compressed string models (Charalampopoulos et al., 2020, Gawrychowski et al., 2013, Bille et al., 2010, Gagie et al., 2011, Belazzougui, 2011, Cisłak et al., 2015).