Explain the estimation-efficiency disadvantage of E5-Mistral embeddings

Determine why E5-Mistral-7B-Instruct embeddings require more labeled examples per class than BGE-large embeddings to yield equivalent few-shot classification performance, despite geometric separability metrics indicating that E5-Mistral contains comparable or stronger separating signal.

Background

The study finds that E5-Mistral-7B-Instruct underperforms BGE-large and MiniLM in the low-shot regime, while three tested geometric explanations—anisotropy, within/across-class cosine margin, and a whitened separability spectrum—fail to account for the performance ordering.

Increasing the support set substantially closes E5-Mistral’s accuracy gap to BGE-large, showing that the disadvantage is largely an estimation-efficiency effect rather than a permanent accuracy ceiling. However, the experiments do not identify why E5-Mistral requires more examples to extract its signal. The authors explicitly leave that mechanism unexplained.

References

This does not explain why E5-Mistral's embedding space needs more examples than BGE-large's to yield equivalent signal (the three failed metrics above remain the primary evidence on that question), but it does establish that the practical consequence, within the $k{\le}10$ range this paper otherwise reports, is substantially a small-sample artifact rather than a permanent ceiling.

— Exact Degeneracy Under Balanced k-Shot Sampling:Consequences for Small-Sample Discriminant Analysis on LLM Embeddings  (2609.09860 - Qu, 9 Sep 2026) in Section 6.1, “A Negative Result We Consider Important: Three Failed Separability Metrics”; Section 5, Guideline 4