Sufficiency of Reinforcement Learning Without Supervised Fine-Tuning

Determine whether reinforcement learning alone, without chain-of-thought annotation and supervised fine-tuning, is sufficient to train an effective verifier for coding-agent trajectories.

Background

SWE-Critix trains a generalist verifier for coding-agent trajectories using a two-stage post-training pipeline: chain-of-thought-supervised fine-tuning followed by reinforcement learning. The supervised fine-tuning stage uses teacher-generated reasoning annotations from paired successful and failed trajectories, while the reinforcement-learning stage uses unit-testing outcomes as the reward signal.

The paper identifies the contribution of each stage as unresolved and proposes a cold-start reinforcement-learning baseline that removes both the chain-of-thought annotation and supervised fine-tuning stages. The open problem is therefore to establish whether reinforcement learning by itself can produce an effective trajectory verifier, rather than relying on the staged SWE-Critix pipeline.