Improve drop-policy tuning beyond loss weighting

Investigate how to tune Paritok-4B’s segment-level keep/drop policy so that it reduces under-dropping without the severe over-dropping caused by increasing the loss weight on dropped-segment examples.

Background

Paritok-4B’s principal remaining compression limitation is under-dropping: on the intrinsic holdout, the released model achieves only 0.24 drop recall and 0.49 drop accuracy against the teacher, while an always-keep policy achieves 0.59 drop accuracy. Thus, the model preserves segments that could potentially be discarded, leaving compression gains unrealized.

The authors attempted to address this by multiplying the loss on drop examples by 20, but the intervention caused the model to over-drop and reduced drop accuracy below the always-keep baseline. They therefore shipped a conservative, keep-biased model and identify improved tuning of the drop signal as an unresolved problem.

References

We report the failure because the naive fix is the one a reader would reach for first, and because it explains the under-dropping that \S\ref{sec:intrinsic} measures: v1's drop behavior is deliberately the safe end of a trade we were unable to tune well with loss weighting alone.

— Paritok-4B: Intent-Conditioned Context Compression for Coding Agents  (2608.24188 - Shi et al., 25 Aug 2026) in Section 5, subsection “Drop supervision: a weighting that backfired,” and Section 6.1, paragraph “Intrinsic: out-of-distribution segment holdout”