Faster Algorithms for Shortest Unique or Absent Substrings

Abstract: We revisit two well-known algorithmic problems on strings: computing a shortest unique substring (SUS) and a shortest absent substring (SAS) of a string $S$ of length $n$. Both problems admit folklore $\mathcal{O}(n)$-time solutions using the suffix tree of $S$. However, for small alphabets, this complexity is not necessarily optimal in the word RAM model, where a string of length $n$ over alphabet $[0,σ)$ can be stored in $\mathcal{O}(n \log σ/\log n)$ space and read in $\mathcal{O}(n \log σ/\log n)$ time. We present an $\mathcal{O}(n \log σ/\sqrt{\log n})$-time algorithm for computing a SUS of $S$. This algorithm decomposes the problem according to the length and the period of the sought substring and uses several tools and techniques, such as synchronizing sets, the analysis of runs, and wavelet trees, to reduce the computation of a SUS to a simple geometric problem. Further, we adapt this algorithm and combine it with an efficient construction of de Bruijn sequences in order to obtain an $\mathcal{O}(n \log σ/\sqrt{\log n})$-time algorithm for computing a SAS of $S$.

• The paper introduces sublinear-time algorithms for computing a single SUS/SAS in packed strings with a complexity of O(n log σ/√(log n)), advancing beyond traditional O(n) solutions.
• It leverages techniques such as synchronizing sets, wavelet trees, and run-based period analysis to efficiently manage substring uniqueness and absence.
• The proposed methods enable faster querying in bioinformatics and text mining, offering practical improvements in handling large-scale string datasets.

Faster Algorithms for Shortest Unique or Absent Substrings

Problem Statement and Context

This paper addresses the classical string algorithmic problems of identifying a Shortest Unique Substring (SUS) and a Shortest Absent Substring (SAS) for a given string $S$ of length $n$ over an integer alphabet $[0, \sigma)$. A SUS is the shortest substring occurring exactly once in $S$, while a SAS is the shortest substring not occurring in $S$. Although $O(n)$-time solutions exist via suffix tree construction, the authors focus on the word RAM model with packed string representation, where the traditional $O(n)$ complexity is not necessarily optimal, especially for small alphabets.

The relevance of SUS and SAS is underscored by their applications in bioinformatics (alignment-free comparison, probe design), text mining (snippet extraction), and data compression (antidictionary-based approaches).

Main Contributions

The paper establishes new sublinear-in-$n$ algorithms for computing a single SUS or SAS under the packed string model, given a string of length $n$ stored in $O(n \log \sigma / \log n)$ space. For both problems, a complexity of:

$n$0

is achieved, improving upon the folklore $n$1-time algorithms. For $n$2 packed into words, significant reductions are realized for small alphabets.

Key techniques include:

• Decomposition of the SUS/SAS computation by substring length and period.
• Leveraging synchronizing sets and efficient construction of wavelet trees for geometric reductions.
• Employing run-based period analysis and Lyndon root factorization to handle highly periodic substrings.
• Reduction of the general alphabet case to the binary case with no asymptotic time increase.

Algorithmic Framework

The algorithmic strategy partitions the search space according to the candidate substring's length and (if applicable) periodicity:

1. Short substrings: For substrings of length at most $n$3, enumeration and tabulation over the packed alphabet is feasible due to their sublinear total number.
2. Aperiodic substrings: For medium and long substrings with high period, the approach employs synchronizing sets to identify "anchor" positions where identical patterns can be efficiently compared and indexed.

   Figure 1: A schematic illustration of two suffix trees—left, identifying SUS by unique-root-to-leaf edge extensions; right, characterizing SAS as appending to non-outgoing edges.

• Wavelet trees are built on families of substrings using anchor points provided by the synchronizing set, with LCP (Longest Common Prefix) information encoded via heavy-path decompositions and bit-parallel structures.
• A Skyline geometric problem is formulated over 2D extension lengths: finding a minimal (left, right)-extension that is outside all but one pattern yields the SUS.

Figure 2: The suffix tree (left) acts as the geometric/combinatorial index for substring uniqueness and absence detection.

1. Periodic substrings: Highly periodic substrings (detected using runs and their Lyndon roots/sparse-Lyndon roots) are grouped and analyzed collectively. Feasibility and uniqueness checks for SUS within these periodic blocks are reduced to geometric problems via mapping each run to a finite set of lattice points.
2. Absent substrings: For SAS, the approach leverages de Bruijn sequences to model all possible substrings for comparison, exploiting their properties to identify missing patterns efficiently within the same time bounds.

Reduction to Binary Alphabets

A fundamental result shows any instance over an alphabet $n$4 can be reduced, in $n$5 time, to an equivalent instance on a binary alphabet of length $n$6, without loss of asymptotic efficiency.

Technical Highlights and Numerical Results

• Time complexity: $n$7 for both SUS and SAS computation in the packed setting.
• Space usage: Optimal, $n$8 machine words, matching the information-theoretic space bound for packed strings.
• Conditional Hardness: For the binary alphabet, the achieved complexity matches the conditional lower bounds for problems in this domain established via reductions from dictionary matching. However, the conditional optimality of these exact algorithms for SUS/SAS remains an open question.
• The geometric reformulation for unique substring extensions enables linear-time algorithms in the number of candidate points (heavy-path pairs), with at most $n$9 total time where $[0, \sigma)$0 is the synchronizing set size ($[0, \sigma)$1 for suitably chosen $[0, \sigma)$2).

Theoretical and Practical Implications

Theoretical impact: This work clarifies the combinatorial structure underlying SUS and SAS, highlighting connections to synchronizing sets, run analysis, geometric domination, and succinct tree representations. It demonstrates that for small alphabets, classic $[0, \sigma)$3 suffix tree-based solutions are not optimal under the word RAM model, and geometric/data-structural reductions are required for further improvement.

Practical significance: By lowering the computation time for core combinatorial objects in string processing, these algorithms enable faster preprocessing and querying in large-scale text datasets, with immediate applications in genomics (e.g., identification of minimal distinguishing probes or absent genomic motifs) and information retrieval.

Future Directions

• Conditional lower bounds: Determining tight hardness results for SUS/SAS in the word RAM/paced model for arbitrary alphabets is open.
• Reporting all SUS/SAS: Extending the sublinear approaches to enumerate all SUSs/SASs (not just report a single instance) remains an avenue for further research.
• Generalizations: Adapting these techniques to position-dependent SUS queries, multiple-string variants, or different error models (e.g., approximate SUS/SAS) is of significant interest.

Conclusion

This paper makes a substantial advance in algorithmic string processing by providing sublinear-time algorithms for the computation of shortest unique and absent substrings in the packed string setting. By blending combinatorial insight into periodicity and packed representations with advanced data structures (wavelet trees, synchronizing sets, geometric dominance), the authors disprove the perceived optimality of $[0, \sigma)$4-time suffix tree algorithms within the word RAM model. Their results have both theoretical and practical importance, and the techniques developed are likely to catalyze further work on fine-grained complexity in string algorithms.