Explain the Knowledge-Channel Performance Degradation under RLSD

Determine whether the performance degradation caused by adding the Knowledge channel to RLSD results from the additional self-referential target competing with RLSD's existing self-distillation term for the model's representational capacity.

Background

The paper evaluates whether explicitly generating foresight at inference improves performance and compares Pitfall-only SRD with a combined Pitfall-and-Knowledge channel under RLSD and GRPO. Under RLSD, the Pitfall channel improves ALFWorld performance, whereas adding the Knowledge channel decreases it relative to the same RLSD baseline.

The authors attribute this behavior plausibly to RLSD already containing a self-distillation objective, with the additional Knowledge channel introducing a second self-referential target that competes for model capacity. However, they explicitly characterize this explanation only as a conjecture, leaving the mechanism behind the degradation unresolved.

References

Under RLSD the sign flips with the channel: Pitfall is worth $+5.75$ pp while adding Knowledge costs $6.88$ pp against the same base, plausibly because RLSD already carries a self-distillation term and the Knowledge channel adds a second self-referential target competing with it, though with two objectives at one scale and one seed we can only offer that as a conjecture.

— Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight  (2610.08077 - Zhang et al., 6 Oct 2026) in Section RQ5, “Does Test-time Foresight Help?”