Dice Question Streamline Icon: https://streamlinehq.com

Algorithm for OPE with approximation ratio above 0.5

Determine whether there exists a polynomial‑time algorithm for the Optimal Pair Encoding (OPE) problem that achieves an approximation ratio strictly greater than 0.5, improving upon the 0.5‑approximation achieved by the EvenOdd algorithm.

Information Square Streamline Icon: https://streamlinehq.com

Background

Beyond analyzing BPE, the authors introduce the EvenOdd algorithm, which guarantees a 0.5 approximation for OPE by greedily targeting non-overlapping occurrences of the k most frequent pairs.

The existence of any polynomial-time algorithm surpassing this 0.5 barrier remains unknown and would represent a significant advance in approximating OPE.

References

Our work can be seen as an initial theoretical investigation with a number of open questions remaining. In particular: is there an efficient algorithm for OPE with approximation ratio above $0.5$?

Theoretical Analysis of Byte-Pair Encoding (2411.08671 - Kozma et al., 13 Nov 2024) in Section 6 (Conclusion and open questions)