Exact approximation ratio of BPE for OPE

Determine the exact worst‑case approximation ratio of Byte‑Pair Encoding (BPE) for the Optimal Pair Encoding (OPE) problem by closing the current gap between the proven lower bound 0.333 and upper bound 0.625 on BPE’s compression‑utility approximation.

Background

The paper proves that BPE achieves a constant-factor approximation for the Optimal Pair Encoding (OPE) problem, with its worst-case compression-utility ratio bounded between 0.333 and 0.625. This is the first rigorous, worst-case guarantee for BPE’s performance with respect to compression utility.

However, the exact ratio is unknown. Tightening these bounds would clarify BPE’s theoretical performance and could inform both algorithm design and practical tokenizer choices.