Explain the empirically successful reverse tie-break direction in KLPCDA No.5

Determine why ascending, rather than descending, variance within KLPCDA No.5’s degenerate class-mean-orthogonal subspace empirically improves classification despite contradicting the variant’s literal maximize-total-variance objective, and establish whether the proposed within-class-consistency interpretation is theoretically valid.

Background

Under balanced k-shot sampling, the within-class operator used by KLPCDA No.5 is a scaled projector, so all directions in its signal subspace have the same denominator in the target ratio. The literal objective therefore calls for selecting directions with the largest total variance within the tied eigenspace.

Empirically, the authors found the opposite ranking to be substantially better: selecting lower-variance directions improved performance across the tested values of k and datasets. They hypothesize that these directions may be more internally consistent within classes, but they do not derive this explanation from the formal KLPCDA framework. The theoretical basis of the successful tie-break remains unresolved.

References

We adopt ascending order because it is supported by the experimental results, and note the mismatch with the literal formula as an open question we do not claim to have resolved: a plausible but unproven hypothesis is that $\Pi$'s range is exactly the "varies-within-class-only" subspace, so low variance there picks the most internally consistent directions for a class, in the spirit of Fisher discriminant analysis, even though it contradicts a literal "maximize $C$" reading once the ratio's denominator degenerates to a constant.

— Exact Degeneracy Under Balanced k-Shot Sampling:Consequences for Small-Sample Discriminant Analysis on LLM Embeddings  (2609.09860 - Qu, 9 Sep 2026) in Section 2.3, “A Tie-Break Fix, and a Direction the Formula Does Not Predict”; Section 6.2, “An Unresolved Direction in the Tie-Break Fix”