YouZhi: Towards High-Concurrency Financial LLMs via Adaptive GQA-to-MLA Transition

Abstract: LLMs drive significant financial innovations, yet their high-concurrency deployment is severely bottlenecked by KV cache memory overhead, which inflates infrastructure costs and throttles scalability. To address this, we propose YouZhi-LLM, a highly efficient financial LLM empowered by a comprehensive structural transition and training pipeline natively built on the Huawei Ascend ecosystem. At its algorithmic core, YouZhi-LLM features a layer-adaptive GQA-to-MLA transition framework that dynamically assigns per-layer FreqFold sizes, maximizing KV-cache compression while minimizing perplexity degradation. To recover representation capacity and inject domain expertise, the Ascend-based training pipeline seamlessly integrates generalized knowledge distillation with financial-specific supervised fine-tuning. Evaluations demonstrate the superiority of this systematic approach, with the adaptive transition reducing perplexity degradation by up to 35% over uniform baselines. Crucially, when evaluated on Ascend NPUs via vLLM-Ascend, the massive KV-cache reduction translates directly into deployment efficiency. Compared to their respective base models, YouZhi-7B yields a 12.3% improvement in average financial benchmark score alongside a 2.69$\times$ increase in maximum concurrency; similarly, YouZhi-14B achieves a 7.0% accuracy gain and a 2.43$\times$ concurrency boost, establishing a new paradigm for cost-effective, high-throughput financial inference.

• The paper introduces a layer-adaptive GQA-to-MLA conversion that selects FreqFold sizes by depth, reducing Llama3-8B perplexity degradation by 65% compared with uniform TransMLA.
• The paper combines knowledge distillation, financial-domain instruction tuning, refusal data, and capability recovery to achieve 78.8 average financial benchmark performance with YouZhi-14B.
• The paper demonstrates practical serving gains on Ascend hardware, shrinking KV-cache use by 72% and increasing maximum concurrency up to 2.69× while preserving mobile-banking task accuracy.

Motivation and problem statement

YouZhi-LLM, a joint effort between the Postal Savings Bank of China and Huawei, targets a deployment gap that the authors argue is obscured by static benchmark evaluation of financial LLMs: production financial services such as mobile banking impose simultaneous requirements for low latency, high concurrency, and high task-completion rates, while existing financial models (BloombergGPT, FinGPT, YiZhao-12B-Chat, DianJin-R1) inherit GQA or MHA attention and treat capability and serving efficiency as separate concerns. The KV cache remains the binding constraint: at 100k tokens, Llama3-8B's cache requires roughly 20 GB, exceeding its parameter footprint. The paper's central thesis is that converting pre-trained GQA backbones to Multi-head Latent Attention (MLA), combined with domain-aware post-training on Huawei Ascend hardware, can move financial LLMs onto a better accuracy–concurrency Pareto frontier.

Layer-adaptive GQA-to-MLA transition

The method builds on TransMLA-style conversion, in which positional information is aggregated into a first head (partial RoPE stage) and key/value projections are jointly compressed via FreqFold — grouping adjacent RoPE dimensions with similar rotational frequencies for joint PCA. The authors identify a critical trade-off: an overly aggressive FreqFold size amplifies RoPE frequency approximation error, and this trade-off varies systematically across depth. Their layer-wise ablation, converting one layer at a time in OpenPangu-7B, shows that shallow layers (0–5) achieve lowest perplexity with large FreqFold sizes (e.g., 8), where principal-component concentration dominates, whereas middle layers (16–25) prefer FreqFold size 1 (no folding), where frequency precision matters most.

Because uniform FreqFold selection confines the search to a single trajectory, the paper formulates per-layer FreqFold assignment as a combinatorial optimization with $(h+1)^{N+1}$ feasible solutions ($9^{34}$ for OpenPangu-7B). They decompose total perplexity degradation into per-layer increments $C_k$ and greedily minimize each term by enumerating only $h+1$ options per layer, reducing search complexity to $(h+1)\times(N+1)$.

Results. Across seven GQA models, the layer-adaptive transition consistently outperforms uniform TransMLA conversion. The strongest result is on Llama3-8B: perplexity rises from 6.1 to 25.8 under TransMLA but only to 12.9 under the adaptive scheme — a 65% reduction in perplexity degradation; OpenPangu-7B improves by 35% (50.3 → 37.6). Ablations show the largest gains occur in the shallowest layers, consistent with the layer-wise analysis. One caveat: gains are heterogeneous — Qwen2.5-7B shows only a 6% reduction (8.4 → 8.3), indicating that the benefit depends on how much degradation the base model suffers under uniform conversion in the first place.

Post-training pipeline

Conversion-induced degradation is recovered through a two-stage pipeline run entirely on Ascend A3 clusters. Stage 1 applies generalized knowledge distillation (forward KL, off-policy) with the original GQA model as teacher. Stage 2 performs financial-domain SFT on 970,000 instruction-response pairs spanning all major financial sub-domains, using cosine-decay learning rate from $1\times10^{-5}$ with 0.01 warmup ratio over two epochs.

Data construction is a substantial contribution in itself:

• Stratified hierarchical compression: rather than applying entropy-based compression globally, samples are tagged (InsTagger for category, DEITA scorers for quality and difficulty), partitioned into category × difficulty sub-clusters, and compressed within clusters using zlib-based incremental information estimation, preserving category coverage and difficulty balance.
• Domain augmentation: progressive diversity generation along "domain → task → scenario → persona" following DecIF meta-decomposition, plus knowledge-point extraction and cross-combination for missing capabilities.
• Refusal data: a two-stage variant of TreeCut generates unanswerable questions via logic-tree path pruning (controlling cutDepth), then LLM translation into natural language — addressing compliance-critical hallucination on incomplete financial queries.
• Domain-specific recovery: Self-Instruct/Evol-Instruct amplification from 50–200 seeds restores degraded skills such as Chinese uppercase financial numeral conversion.
• Instruction-following data: declarative format templates (JSON, Markdown, LaTeX) with weight control and randomization rebuild the instruction–format mapping eroded by transition.

Benchmark and application results

On general benchmarks, YouZhi-7B and YouZhi-14B average 66.8 and 68.0 respectively, matching or exceeding their bases OpenPangu-7B (62.4) and Qwen2.5-14B-Instruct (65.8) despite the architectural change. On financial benchmarks, YouZhi-14B reaches 78.8 average, ahead of YiZhao-12B-Chat (69.7), DianJin-R1-7B (69.8), Qwen3-8B (70.1), and Qwen3.5-9B (73.1); YouZhi-7B achieves 74.1. Notably, YouZhi-14B trails its base on MATH-500 (72.4 vs. 80.6) and LiveCodeBench (25.5 vs. 20.4 vs. base 34.6 for OpenPangu-7B on YouZhi-7B), so the general-capability preservation claim holds on average but not uniformly across reasoning-heavy tasks.

In a real mobile-banking evaluation fine-tuned on identical proprietary data, YouZhi-7B matches OpenPangu-7B across six tasks: 97.73% average intent-recognition accuracy and 99.05% on slot filling, confirming that the MLA conversion does not impair task-oriented dialogue performance after post-training.

High-concurrency serving results

Evaluated on vLLM-Ascend with 1k-token input and output per request, the KV cache shrinks from 2,048 to 576 elements (72% reduction). This translates directly into serving capacity:

Model	Max concurrency	Max throughput	KV cache
OpenPangu-7B (base)	95	3,325 tok/s	2,048
YouZhi-7B	256 (2.69×)	5,865 tok/s (1.76×)	576
Qwen2.5-14B-Ins. (base)	55	1,740 tok/s	2,048
YouZhi-14B	134 (2.43×)	2,990 tok/s (1.71×)	576

YouZhi-7B also exceeds all listed baselines in concurrency, including DianJin-R1-7B (190) and YiZhao-12B-Chat (206). The abstract's headline claims — 12.3% financial accuracy gain with 2.69× concurrency for the 7B model, and 7.0% gain with 2.43× for the 14B model — are consistent with these tables.

Limitations and open questions

Several caveats bear directly on the results. First, the layer-adaptive algorithm is greedy: it minimizes each $C_k$ sequentially rather than the global objective, and no guarantee or empirical comparison against joint optimization is provided. Second, the layer-wise analysis motivating the method is conducted on OpenPangu-7B alone; whether the shallow-vs-deep FreqFold pattern generalizes across architectures is asserted but not demonstrated. Third, the modest improvement for Qwen2.5-7B (-6%) suggests the method's value is contingent on baseline conversion severity. Fourth, all system evaluations are confined to a single Ascend chip and vLLM-Ascend; portability of the concurrency gains to other accelerators is untested. Finally, the paper reports no long-context evaluation beyond WikiText-2 perplexity, leaving open whether the converted MLA models retain quality at the very context lengths (100k+ tokens) that motivate the KV-cache argument.

Conclusion

YouZhi demonstrates that structural adaptation — specifically, per-layer adaptive FreqFold selection during GQA-to-MLA transition — combined with a carefully engineered post-training pipeline can produce financial LLMs that improve both domain accuracy and serving concurrency relative to their base models. The 65% perplexity-degradation reduction on Llama3-8B and the 2.69× concurrency gain on Ascend are the paper's strongest quantitative results, though their generality across model families, optimizers, and hardware platforms remains an open empirical question.