Smallest suffixient set maintenance in near-real-time

Abstract: The size of the \textit{smallest suffixient set} of positions of a string recently emerged as a new measure of string \textit{repetitiveness} -- a measure reflecting how much of repetitive content the string contains. We study how to maintain the smallest suffixient set online in near-real-time, that is with small (in our case, polyloglog) worst-case time on processing each letter. Two frameworks are considered: when the text is given letter-by-letter in either a right-to-left or left-to-right direction. Our central algorithmic tool is Weiner's suffix tree algorithm and associated algorithmic primitives for its efficient implementation.

• The paper presents a new method to maintain smallest suffixient sets online while ensuring worst-case per-character update guarantees.
• It leverages Weiner's suffix tree algorithm and colored ancestor queries to achieve efficient real-time updates with defined complexity bounds.
• Numerical results confirm O(n) memory usage and worst-case O(log^2 log n) per letter processing, enhancing redundancy detection and indexing applications.

Smallest Suffixient Set Maintenance in Near-Real-Time

Introduction and Motivation

The paper "Smallest suffixient set maintenance in near-real-time" (2604.27548) addresses the problem of maintaining the smallest suffixient set (SSS) of a string in an online, near-real-time manner. The SSS is a recently defined measure of string repetitiveness, intended to quantify redundancy in highly repetitive data, such as genomic sequences or version-controlled files. Unlike measures based on compressibility (e.g., Lempel–Ziv or grammar compression), SSS is rooted in substring occurrence properties, offering both theoretical interest and practical potency for indexing and compressibility estimation.

While the string attractor size $\gamma$ is NP-complete to compute, the SSS size $\chi$ is algorithmically tractable, computable in linear time, with efficient methods for testing suffixiency. This advances the field by providing a strong, computable repetitiveness measure, but introduces challenges in maintaining $\chi$ efficiently as new characters arrive in either left-to-right or right-to-left fashion. Prior work provided only amortized bounds; this paper seeks worst-case guarantees per letter, leveraging Weiner's suffix tree construction algorithm and dynamic data structures.

Suffixient Sets and Supermaximal Extensions

The suffixient set is based on supermaximal right-extensions (SREs): for any substring $u$ and right extension $x$, $(u, x)$ is a SRE if no longer right-maximal substring ending with $x$ exists. These SREs map uniquely to positions in the string, and the minimal set covering all such SREs comprises an SSS. Notably, $\chi$ is incomparable with compressibility measures, as established through counterexamples (e.g., de Bruijn sequences and Fibonacci words), but it relates to attractor and BWT-based measures ($\gamma \leq \chi \leq 2r$).

Figure 1: Suffix trees of the strings in Example 1, highlighting loci of SREs and the resulting smallest suffixient sets.

Figure 2: Visualization of all smallest suffixient sets for $v = \mathtt{aabaabaab\$}$, demonstrating SRE position <a href="https://www.emergentmind.com/topics/diversity-beta-recall" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">diversity</a>.</p></p> <h2 class='paper-heading' id='online-maintenance-algorithmic-insights'>Online Maintenance: Algorithmic Insights</h2><h3 class='paper-heading' id='weiner-s-suffix-tree-algorithm'>Weiner’s Suffix Tree Algorithm</h3> <p>The foundation lies in Weiner&#39;s right-to-left suffix tree construction, maintaining nodes and specialized W-links (links for substring extension by letters). To support real-time updates, the paper leverages the fringe colored ancestor problem, enabling efficient identification of ancestor nodes with given color properties (representing W-links). <img src="https://emergentmind-storage-cdn-c7atfsgud9cecchk.z01.azurefd.net/paper-images/2604-27548/suffixupdate.png" alt="Figure 3" title="" class="markdown-image" loading="lazy"></p> <p><img src="https://emergentmind-storage-cdn-c7atfsgud9cecchk.z01.azurefd.net/paper-images/2604-27548/suffixupdate.png" alt="Figure 3" title="" class="markdown-image" loading="lazy"> <p class="figure-caption">Figure 3: One round of Weiner&#39;s suffix tree algorithm, updating the tree for $\chi$0 to $\chi$1, with W-link visualizations and updates to node and edge labels.

Efficient implementation is achieved by reduction to colored predecessor queries on dynamic lists, exploiting properties of Euler tours and explicit/implicit colorings (hard/soft W-links). Data structures such as Mortensen's fully dynamic lists yield worst-case $\chi$2 per letter, or $\chi$3 for sufficiently small alphabets—a substantial improvement over prior amortized results.

Figure 4: Illustration of colored fringe marking in ancestor queries, key to efficient node retrieval in suffix tree maintenance.

Maintenance of SREs and SSS

Right-to-Left Framework

When strings are appended right-to-left, the algorithm modifies the current set of SREs as dictated by Weiner’s update logic, explicitly tracking which SREs are added or removed. Each SRE is associated with a unique position (maintained via bit vectors), guaranteeing non-duplication and efficient rightmost occurrence tracking. Deletion logic is corrected compared to previous work, ensuring completeness.

Left-to-Right Framework

For left-to-right updates, Weiner’s algorithm is applied to the reversed string. SRE modifications are characterized through W-link changes and explicit node colorings, backed by fringe colored ancestor queries. Operations are performed in worst-case $\chi$4 ($\chi$5 for small alphabets) per letter. SRE positions are maintained analogously to the right-to-left case.

Figure 5: Illustration of SRE set modifications under left-to-right updates, detailing node and W-link transitions.

Numerical Results and Claims

The paper proves that, for both input directions, SREs and SSS can be maintained online with worst-case $\chi$6 per letter processing on integer alphabets, reducing to $\chi$7 for $\chi$8. Memory usage is $\chi$9, matching the size of the text. These bounds are supported by reductions to colored predecessor data structures and explicit algorithmic constructions.

Implications and Future Directions

Practically, SSS maintenance in near-real-time directly benefits applications in indexing, search, and redundancy quantification for highly repetitive datasets, such as genomics and version histories. The approach realizes theoretically strong guarantees previously unattainable for suffixient sets. Future research may explore compressed space maintenance, exploiting links between SSS and BWT runs, or further optimize suffix tree updates via advances in real-time string indexing.

Theoretically, this work tightens the complexity gap between repetitiveness measurement and compressed index maintenance, providing tractable boundaries for a previously elusive metric. It also enriches the landscape of online string algorithms with fine-grained worst-case analyses.

Conclusion

This paper introduces efficient algorithms for maintaining the smallest suffixient set of a string in near-real-time, offering worst-case guarantees per input character for both left-to-right and right-to-left frameworks. The results leverage Weiner’s suffix tree algorithm, colored ancestor queries, and succinct data structures, yielding strong complexity bounds and practical memory requirements. It advances the computability and applicability of suffixient sets as measures of repetitiveness and paves the way for further research into compact and compressed representations of repetitive data.