Controlled downstream comparison of qwen250k and mix250k

Conduct a controlled downstream-performance comparison between the qwen250k and mix250k vocabularies at comparable vocabulary sizes to assess their relative effectiveness independently of vocabulary-size differences.

Background

The paper evaluates LLMs trained with the CNF–MCE tokenizer using the mix250k vocabulary and compares them with models using the qwen152k vocabulary. Although the 8B-scale models achieve similar average benchmark performance, the comparison is confounded because the vocabularies differ in size: mix250k contains approximately 250,000 entries, whereas qwen152k contains approximately 152,000 entries.

A direct comparison between qwen250k and mix250k would control the vocabulary-size factor and provide a more precise assessment of whether the observed downstream results are attributable to the tokenizer construction and encoding method rather than to differences in vocabulary capacity.