Determine whether routing improvements transfer across encoder sizes

Determine whether the routing improvements observed with GTE-Qwen2-7B-Instruct and Qwen2.5-7B-Instruct layer-21 hidden-state embeddings extend to encoders of other sizes.

Background

The empirical comparison uses one semantic encoder, GTE-Qwen2-7B-Instruct, and one hidden-state encoder, Qwen2.5-7B-Instruct, with the hidden-state representation extracted from layer 21. The reported generalization gains are therefore tied to the evaluated encoder choices.

The paper explicitly identifies transfer to other encoder sizes as unresolved. Establishing such transfer would clarify whether the observed routing improvements reflect a general property of hidden-state representations or depend on the particular model scales studied.

References

Whether these results extend to other encoder sizes remains unclear.

— Embedding Geometry Shapes LLM Routing: Tracing Failure Modes and Improving Generalization  (2610.19198 - Torti et al., 13 Oct 2026) in Section “Limitations”