PrimeEcho: First-Turn Anchored Reward

• PrimeEcho is a trajectory-level reward formulation in MusaCoder that emphasizes the first-turn output for native GPU kernel generation.
• It leverages multi-turn feedback by aggregating scores from up to three verifier-mediated interactions while preventing reward hacking.
• The scheme combines a pure first-turn score with a best-turn bonus and early-success rewards to maintain zero-shot or near-zero-shot generation quality.

to=arxiv_search.search 一本道高清无码ి ／久久கம் _人人碰 code {"query":"(Cheng et al., 3 Jun 2026) MusaCoder PrimeEcho", "max_results": 5} to=arxiv_search.search 大奖彩票站 code {"query":"(Liu et al., 22 Feb 2025) Echo LLM with Temporal Episodic Memory", "max_results": 5} PrimeEcho is a trajectory-level reward formulation introduced within MusaCoder for execution-feedback reinforcement learning in native GPU kernel generation. In MusaCoder, it is defined as a first-turn-anchored multi-turn reward that incorporates corrective feedback from later attempts while keeping optimization aligned with the deployment objective of strong zero-shot or near-zero-shot kernel synthesis. The mechanism was designed for a setting in which MooreEval can compile, execute, and profile multiple kernel-generation attempts for the same task, yet each additional attempt is operationally expensive and can induce reward hacking if training optimizes primarily for eventual success rather than initial quality (Cheng et al., 3 Jun 2026).

1. Conceptual role in MusaCoder

PrimeEcho occupies a narrow but important place in MusaCoder’s full-stack training pipeline. MusaCoder targets native GPU kernel generation on CUDA and MUSA backends, and its RL stage uses execution feedback from MooreEval rather than preference labels. In that environment, a model may generate a candidate kernel, receive compiler errors or other verifier feedback, and attempt repairs across multiple turns. PrimeEcho was introduced because naïvely rewarding the best outcome across turns creates an objective mismatch: the model can be incentivized to produce weaker first attempts, rely on verifier feedback, and optimize for delayed correction rather than direct one-shot competence (Cheng et al., 3 Jun 2026).

The formulation therefore treats the first-turn output as the “prime” object of optimization and uses later turns only as auxiliary signals. MusaCoder explicitly characterizes this as a mechanism for improving exploration while mitigating multi-turn reward hacking, without degrading zero-shot generation quality. A central design principle is that later successes may inform what is achievable for a prompt, but they must not dominate policy improvement. This places PrimeEcho at the intersection of reward shaping, multi-turn code RL, and sequence-level credit assignment, but with an unusually strict first-turn bias.

A common misunderstanding is to treat PrimeEcho as an inference-time agentic search framework. MusaCoder presents the opposite emphasis: multi-turn interaction is used during training to internalize repair and debugging signals into the model weights, while the optimization target remains fast initial kernel generation. This suggests that PrimeEcho is best understood not as a general-purpose deliberative scaffold, but as a specialized alignment device for executable code RL where compile–run–profile latency matters.

2. Multi-turn trajectory structure

In MusaCoder’s multi-turn feedback RL, a turn is one complete verifier-mediated interaction cycle. For a prompt $x$, the model generates a response $y_{i,k}$, code is extracted as $c_{i,k} = \mathrm{Extract}(y_{i,k})$, MooreEval evaluates that code, and the environment returns a structured result $V_{i,k}$, a scalar score $s_{i,k}$, and textual feedback $f_{i,k}$ such as compiler errors, shape mismatches, or speed information (Cheng et al., 3 Jun 2026).

If a candidate passes verification, the rollout stops early. Otherwise, the next turn conditions on the original task together with prior code and textual feedback. The trajectory for sample $i$ is written as

$$\tau_i = (x, y_{i,1}, c_{i,1}, V_{i,1}, s_{i,1}, f_{i,1}, y_{i,2}, c_{i,2}, \dots, y_{i,K}, c_{i,K}, V_{i,K}, s_{i,K})$$

where $K$ is the maximum number of turns. MusaCoder uses $K=3$ in its RL setup. This structure makes later attempts informative in two distinct senses. First, they can recover success on hard tasks after an initial failure. Second, they expose corrective trajectories that reveal which repairs are effective under executable evaluation. PrimeEcho is defined over this entire trajectory rather than over isolated per-turn rewards.

The distinction between first-turn and best-turn performance is fundamental. First-turn quality corresponds to the deployment objective. Best-turn quality captures the ceiling achieved when the model is allowed to exploit verifier feedback over several attempts. PrimeEcho interpolates between these two signals rather than collapsing them into a single “eventual success” target.

3. Formal reward definition

PrimeEcho defines a trajectory reward $y_{i,k}$0 from the first-turn score, the best score achieved anywhere in the trajectory, and an early-success bonus. MusaCoder gives the reward as

$y_{i,k}$1

where $y_{i,k}$2 is the first-turn anchor weight. At $y_{i,k}$3, the reward reduces to a pure first-turn objective apart from the early bonus. At $y_{i,k}$4, it reduces to a pure best-turn objective apart from the early bonus (Cheng et al., 3 Jun 2026).

The early-success bonus is

$y_{i,k}$5

with $y_{i,k}$6. Success is defined through the MooreEval score threshold

$y_{i,k}$7

Under MusaCoder’s executable scoring scheme, parse, compile, or runtime failure, cheating through disallowed aten, and totally wrong outputs receive $y_{i,k}$8; partial correctness yields scores in $y_{i,k}$9; and fully correct legal kernels receive $c_{i,k} = \mathrm{Extract}(y_{i,k})$0, with an additional speedup bonus. The reward therefore encodes three priorities simultaneously: direct pressure on first-turn behavior, bounded use of later-turn exploration, and an explicit preference for earlier success.

For GRPO, MusaCoder converts trajectory rewards into group-relative advantages:

$c_{i,k} = \mathrm{Extract}(y_{i,k})$1

The most distinctive implementation choice is that this advantage is applied only to the tokens of the first-turn output. MusaCoder states that the multi-turn feedback is thereby subordinated to improving first-turn generation. PrimeEcho is thus “first-turn anchored” in two separate senses: reward composition is dominated by $c_{i,k} = \mathrm{Extract}(y_{i,k})$2, and gradient application is masked to first-turn tokens. The default experimental setting uses $c_{i,k} = \mathrm{Extract}(y_{i,k})$3, so 75% of the reward weight is placed on first-turn behavior (Cheng et al., 3 Jun 2026).

4. Integration with MooreEval, GRPO, and the RL stack

PrimeEcho appears in the second stage of MusaCoder’s RL pipeline. The first stage is a single-turn warmup, in which GRPO is trained with the single-turn reward $c_{i,k} = \mathrm{Extract}(y_{i,k})$4. This stage is intended to establish strong first-attempt kernel generation before multi-turn corrective interaction is introduced. The second stage is multi-turn RL, where each prompt can produce up to $c_{i,k} = \mathrm{Extract}(y_{i,k})$5 turns, each turn receives a MooreEval score, and PrimeEcho aggregates those scores into the trajectory reward used by GRPO (Cheng et al., 3 Jun 2026).

Its behavior depends directly on MooreEval’s executable scoring function:

$c_{i,k} = \mathrm{Extract}(y_{i,k})$6

where $c_{i,k} = \mathrm{Extract}(y_{i,k})$7 is correctness rate among tests and $c_{i,k} = \mathrm{Extract}(y_{i,k})$8 is speedup relative to baseline. Because PrimeEcho is defined over $c_{i,k} = \mathrm{Extract}(y_{i,k})$9 rather than over human preferences, it inherits MooreEval’s structure: anti-cheating penalties, partial-credit handling, and speedup bonuses all propagate into the trajectory reward.

MusaCoder presents PrimeEcho as orthogonal to two additional RL-stability mechanisms: Buffered Dynamic Retry (BDR) and MirrorPop. BDR converts all-failed samples into feedback-conditioned repair tasks stored in a dynamic buffer. MirrorPop masks severely off-policy sequences at the response level using a magnitude-of-log-ratio criterion. PrimeEcho does not alter sampling or off-policy filtering; instead, it defines how the multi-turn rollout is converted into an optimization target. MusaCoder’s ablations indicate that removing PrimeEcho, BDR, or MirrorPop each harms performance, with MirrorPop producing the largest stability effect and PrimeEcho specifically governing the first-turn versus best-turn balance.

Because PrimeEcho depends only on per-turn scores, the multi-turn rollout structure, and GRPO’s group-based policy gradient, MusaCoder describes it as model-agnostic and backend-agnostic. It is used uniformly at both 9B and 27B scales and across CUDA and MUSA.

5. Empirical behavior and ablation evidence

PrimeEcho is one component of MusaCoder’s full RL system, which reports strong top-line results on KernelBench and a MUSA-ported variant. MusaCoder-27B-RL achieves 93.2% Overall Pass@8 and 88.6% Avg.@8 on KernelBench, with 15.0% Faster Rate versus PyTorch eager and 9.2% versus torch.compile. The paper states that this surpasses Claude Opus 4.7 by +6.0 and +11.3 points respectively on the two correctness aggregates, while also exceeding all baselines on faster-rate metrics (Cheng et al., 3 Jun 2026).

The direct evidence for PrimeEcho comes from component ablation. Removing PrimeEcho from MusaCoder-27B-RL reduces Pass@8 from 93.2 to 88.4 and Avg.@8 from 88.60 to 83.50. Faster Rate versus eager drops from 15.0 to 13.9, and Faster Rate versus torch.compile drops from 9.2 to 8.4. These deltas indicate that PrimeEcho contributes to both correctness and runtime speedup, not merely to one metric family.

The paper’s qualitative discussion interprets these results through the distinction between first-turn and best-turn behavior. Without PrimeEcho, multi-turn RL tends to shift optimization toward best-turn success, harming first-turn accuracy and encouraging greater dependence on later corrective turns. With PrimeEcho, first-turn metrics and best-turn metrics both improve, and multi-turn feedback is especially helpful on harder Level-2 and Level-3 tasks without sacrificing first-turn quality. MusaCoder also reports that the average number of evaluation turns drops over training, which is consistent with the early-success bonus: the policy is rewarded not just for eventual success, but for solving tasks sooner.

A plausible implication is that PrimeEcho does not simply regularize the policy against reward hacking; it also improves credit assignment on hard tasks by allowing later-turn recovery to “echo back” into first-turn learning. MusaCoder’s framing supports that interpretation, although the reward is intentionally constructed so that those later-turn signals remain auxiliary rather than dominant.

6. Relation to adjacent methods, limits, and name disambiguation

PrimeEcho is distinguished from more generic multi-turn code frameworks by three properties emphasized in MusaCoder. First, it uses explicit first-turn anchoring through the interpolation coefficient $V_{i,k}$0 and the early-success bonuses. Second, it applies the resulting trajectory advantage only to first-turn tokens, rather than symmetrically training every turn. Third, it is tightly coupled to an executable reward structure rather than to preference judgments or process supervision. MusaCoder therefore positions PrimeEcho as a specialized reward scheme for executable multi-turn code settings in which zero-shot correctness and runtime speed must not be sacrificed to slow, feedback-dependent repair behavior (Cheng et al., 3 Jun 2026).

The design also imposes clear limits. If $V_{i,k}$1 were set too low, the mechanism would approach a best-turn reward and weaken the intended first-turn emphasis. Because PrimeEcho depends on stable per-turn scores from MooreEval, noisier verification environments would plausibly degrade credit assignment quality. And because gradients are applied only to first-turn tokens, later-turn behavior is improved only indirectly. MusaCoder treats this asymmetry as acceptable because its deployment goal is to avoid later turns when possible, but the same choice would make PrimeEcho a poor fit for a system whose objective were fully interactive, symmetric multi-step assistance.

Despite the lexical overlap, PrimeEcho should not be conflated with Echo, a separate ChatGLM3-6B fine-tuning paradigm for temporal episodic memory that inserts an observation(time) role into dialogue formatting and trains on EM-Train to improve episodic recall and temporal reasoning (Liu et al., 22 Feb 2025). The two works address different problems: PrimeEcho is a reward formulation for multi-turn execution-feedback RL in GPU kernel generation, whereas Echo is a time-aware training paradigm for multi-turn episodic-memory dialogue. The shared “echo” terminology is nominal rather than methodological.

PrimeEcho’s broader significance lies in the general pattern it exemplifies. Many code-generation RL settings face the same tension between the usefulness of multi-step repair and the practical importance of first-shot generation. PrimeEcho provides a mathematically simple answer to that tension: use later turns to reveal achievable corrections, but bind both the reward and the gradient update to the initial attempt.