Trade-off in order-k encodings

Determine whether the greater skewness of the encoded text or the increased size of the auxiliary array dominates the space and performance trade-off in an order-$k$ version of the hybrid index, which ranks each character using the $k$ preceding characters.

Background

The basic hybrid uses an order-1 encoding: each character is represented by its rank among the successors of the preceding character. The paper suggests extending this construction to order kk, where ranks depend on the preceding kk characters and kk-grams are stored in the auxiliary array.

Such an extension could make the encoded sequence more skewed, potentially improving FM-style backward search, but it would also enlarge the auxiliary array and its representation. The paper does not determine which of these opposing effects is dominant.

References

An order-$k$ version would rank each character by the $k$ characters before it and store $k$-grams in $$. That makes $E$ more skewed and $$ bigger, and it is not clear which effect wins.

— Mixing FM-indexes and CSAs: backward search over an order-1 rank encoding  (2609.40299 - Gagie, 30 Sep 2026) in Section 6, “Discussion and open questions,” paragraph “Smaller hybrids”