Hawkeye: Efficient Reasoning via Model Collaboration

• Hawkeye is a framework that compresses verbose Chain-of-Thought reasoning into concise, instruction-style directives without sacrificing answer quality.
• It employs a two-model system where a large model generates compressed reasoning steps and a smaller model expands them, reducing token usage by up to 70%.
• Reinforcement learning fine-tunes the process with a length penalty to balance efficiency, accuracy, and improved clarity of the final response.

Searching arXiv for the specified Hawkeye paper and closely related reasoning-efficiency work. arxiv_search(query="Hawkeye Efficient Reasoning with Model Collaboration (She et al., 1 Apr 2025)", max_results=5, sort_by="submittedDate") arxiv_search(query="(She et al., 1 Apr 2025)", max_results=10, sort_by="submittedDate") Hawkeye is a post-training and inference framework for efficient reasoning in LLMs that compresses Chain-of-Thought (CoT) into concise, instruction-style intermediate representations and delegates answer expansion to a smaller model. Introduced in "Hawkeye:Efficient Reasoning with Model Collaboration" (She et al., 1 Apr 2025), it is motivated by the observation that long CoTs improve reasoning quality but incur substantial latency, KV-cache memory, and serving cost because intermediate reasoning tokens dominate output length. Hawkeye formalizes redundancy in CoT, distills high-density reasoning instructions with reinforcement learning, and uses a two-model decoding pipeline in which a large model generates compressed CoT and a smaller LLM produces the final answer. The reported outcome is a regime in which only a minority of full-CoT tokens are retained while final answer quality remains comparable and, on several dimensions, improves.

1. Motivation and problem formulation

Hawkeye addresses an efficiency problem specific to CoT-based reasoning: the explicit intermediate steps that improve answer quality also generate excessive intermediate reasoning tokens, semantic redundancy, and overly detailed reasoning steps (She et al., 1 Apr 2025). The framework is grounded in the empirical claim that long CoTs can inflate inference latency, KV-cache memory, and API cost, with the paper reporting up to $10\times$ more tokens and $6\times$ more cost in practice.

The central premise is that most CoT tokens are unnecessary. The paper reports that empirical analysis finds $60$--$80\%$ of CoT tokens to be semantically redundant, consisting of repeated hints, filler phrases, or overly fine-grained steps. Hawkeye therefore reframes reasoning efficiency not as eliminating intermediate reasoning, but as increasing its information density. A common misconception is that the method removes CoT altogether; more precisely, it retains CoT in compressed form and changes its role from verbose exposition to compact control signal for a downstream response model.

This design yields a collaborative decomposition of reasoning. A large model is post-trained to emit concise CoT instructions that preserve the logic-bearing tokens, while a much smaller model consumes those instructions and expands them into human-readable answers. This suggests that, in Hawkeye, reasoning and verbalization are treated as partially separable functions rather than as a single monolithic decoding process.

2. Quantifying redundancy in Chain-of-Thought

A distinctive aspect of Hawkeye is its explicit formalization of CoT compression. Let the full CoT be a token sequence $C$ and the compressed CoT be $C'$. The compression rate is defined as

$\eta = |C'| / |C|$

and the redundancy rate is defined as

$\rho = 1 - \eta$

(She et al., 1 Apr 2025)

Within this framework, compression is not treated as an auxiliary heuristic but as a measurable property of reasoning traces. The paper reports that, in experiments on GSM8K, accuracy remains stable down to approximately $\eta \approx 0.3$, implying that about $70\%$ of CoT tokens can be removed without materially degrading exact-match accuracy. Compression levels of $6\times$0, $6\times$1, $6\times$2, $6\times$3, $6\times$4, and $6\times$5 of the original CoT length were tested, with robustness remaining high until roughly $6\times$6.

This analysis is consequential for the framework’s later architectural choices. If only a small portion of the original CoT is logic-bearing, then a compressed instruction can serve as a sufficient intermediate representation for a second model. Hawkeye’s reported operating point—retaining approximately $6\times$7 of original CoT tokens—follows directly from this redundancy analysis rather than from an arbitrary compression target.

3. Post-training architecture and two-model collaboration

Hawkeye’s training pipeline consists of a large model, denoted Model A, and a smaller frozen response model, denoted Model B (She et al., 1 Apr 2025). Model A is initialized as the CoT-instruction generator, with policy $6\times$8, while Model B is frozen throughout post-training.

For each training pair $6\times$9, Model A generates a concise CoT instruction $60$0. Model B then consumes $60$1, where $60$2 is a fixed prompt, and produces a candidate answer $60$3. The generated answer is evaluated against the gold answer, and the result becomes the learning signal for the policy that generated $60$4.

The inference pipeline mirrors this training decomposition. First, a large LLM generates compressed CoT $60$5 given query $60$6. Second, a small LLM takes the prefix $60$7 and decodes the full answer. The paper characterizes this as two-step decoding and reports that it yields nearly the same final answer quality while drastically reducing total token usage (She et al., 1 Apr 2025).

The architectural point is not merely that a smaller model is used, but that the smaller model is used after a large model has already distilled the reasoning into a compact control sequence. Hawkeye therefore differs from single-model CoT compression schemes in that compression and answer realization are assigned to different models. This suggests a division of labor in which the larger model performs abstraction and the smaller model performs expansion.

4. Reinforcement-learning objective and length control

Hawkeye uses GRPO fine-tuning for post-training. The reward is defined as

$60$8

where $60$9 is exact match, $80\%$0 is Model B’s candidate answer, and the quadratic term is a length penalty on the compressed CoT (She et al., 1 Apr 2025). In equivalent decomposed form, the penalty is

$80\%$1

and the reward may be written as $80\%$2.

The policy update is standard policy gradient:

$80\%$3

The critical design choice is the coupling of reward computation to Model B’s answer quality rather than to intrinsic properties of the compressed CoT alone. The paper reports that including Model B in reward computation stabilizes CoT length and avoids trivial collapse, described as reward hacking. Length control is therefore not enforced solely by hard truncation; it is optimized jointly with downstream answer performance.

The explicit target encoded by the penalty is that the compressed CoT should remain at or below $80\%$4 of answer length. This is a stronger claim than generic brevity regularization: Hawkeye operationalizes a specific budget for intermediate reasoning relative to final response length.

5. Inference behavior, latency, and serving cost

Hawkeye’s efficiency claims span token count, wall-clock latency, and monetary serving cost. The serving-cost formulation is given as

$80\%$5

and

$80\%$6

(She et al., 1 Apr 2025)

The paper reports that the framework can retain only approximately $80\%$7 of original CoT tokens, reduce reasoning tokens by up to $80\%$8, and cut serving cost by around $80\%$9. Under concurrency $C$0, the reported speedups are $C$1 on GSM8K, $C$2 on MATH500, and $C$3 on MATH, with token usage reduced to $C$4--$C$5 of baseline. Under concurrency $C$6, the reported speedups are $C$7 on GSM8K, $C$8 on MATH500, and $C$9 on MATH, with token usage reduced to $C'$0--$C'$1 of baseline (She et al., 1 Apr 2025).

A concrete latency example is given for MATH500 at concurrency $C'$2: time per request drops from $C'$3 to $C'$4. The paper also reports cost reductions of up to $C'$5 versus OpenAI o1 and $C'$6 versus DeepSeek-R1.

Aspect	Reported result	Context
CoT retention	$C'$7 of full CoT	Comparable response quality
End-to-end speedup	Up to $C'$8	Complex math tasks
Inference cost reduction	Up to $C'$9	Summary benefit
Cost reduction vs. OpenAI o1	Up to $\eta = |C'| / |C|$0	Serving cost comparison
Cost reduction vs. DeepSeek-R1	$\eta = |C'| / |C|$1	Serving cost comparison

These results show that Hawkeye’s efficiency benefits are not limited to token compression in isolation. Because output-token count directly affects compute and cost in autoregressive serving, compressed CoT produces system-level gains in both latency and economics.

6. Evaluation, output quality, and interpretation

On quality benchmarks, Hawkeye reports exact-match accuracy on MATH, MATH500, GSM8K, and AIME that remains within $\eta = |C'| / |C|$2--$\eta = |C'| / |C|$3 of the full-CoT baseline (She et al., 1 Apr 2025). At the same time, LLM-as-Judge evaluation reports approximately $\eta = |C'| / |C|$4 improvements in Clarity, Coherence, and Conciseness. The paper therefore frames compressed CoT not merely as a cheaper surrogate for full-CoT, but as a representation that can improve aspects of answer presentation.

This result should be interpreted carefully. The reported improvement is not in every possible notion of reasoning quality, and the exact-match findings are phrased as remaining within a narrow band of the full-CoT baseline rather than uniformly exceeding it. The stronger claim is that Hawkeye preserves task performance while improving output-form characteristics and efficiency. A plausible implication is that verbose CoT can degrade final answers by injecting unnecessary discourse structure, even when its latent reasoning content is useful.

More broadly, Hawkeye reorients CoT from an explanatory transcript to a compressed inter-model interface. In that sense, its contribution is as much representational as algorithmic: it treats reasoning traces as compressible control artifacts, optimizes them by reinforcement learning, and exploits model collaboration to decouple concise logical planning from fluent answer generation (She et al., 1 Apr 2025).