Probe-and-Refine Tuning

Updated 5 July 2026

Probe-and-refine tuning is a multi-stage adaptation method that first probes a model for deficiencies before applying targeted refinements.
It utilizes techniques like subquestion generation, modality alignment, and synthetic task creation to diagnose and improve model behavior.
Applications span reasoning enhancement, multi-modal transfer, privacy-preserving fine-tuning, and coding guidance refinement.

Searching arXiv for the cited works and closely related papers to ground the article. arXiv search query: "probe-and-refine tuning OR ART Ask Refine and Trust OR ORCA align then refine OR Instruction Vector refine fine-tuning"

Probe-and-refine tuning denotes a family of multi-stage adaptation procedures in which an explicit probe, alignment, or diagnostic stage is used to determine how a later refinement stage should proceed. Across the literature, the probed object varies: an LLM’s initial chain-of-thought can be interrogated before targeted re-generation, a new modality can be aligned into a pretrained feature space before joint fine-tuning, a capability-specific hidden-state direction can be identified before continual tuning, a linear head can be optimized before private full fine-tuning, or synthetic bug-fix tasks can be used to rewrite repository guidance rather than model weights (Shridhar et al., 2023, Shen et al., 2023, Jiang et al., 2024, Ke et al., 2024, Shepard et al., 18 Jun 2026). Taken together, these works suggest that probe-and-refine tuning is best understood not as a single algorithm but as a recurrent decomposition: first estimate compatibility, correctness, structure, or risk; then apply a selective refinement whose scope, objective, or target has been narrowed by the probe.

1. Core abstraction and representative formulations

A common abstraction is a two-stage or multi-stage pipeline in which a pretrained solver, backbone, or agent is not immediately optimized end to end. Instead, an intermediate component first exposes useful structure: whether an answer should be revised, how a target modality should be embedded, which internal computation graph should be preserved, or which external guidance artifact is defective. Refinement then operates on the basis of that structure rather than by undirected resampling or uniform full-model updating.

Formulation	Probe stage	Refine stage
ART (Shridhar et al., 2023)	Asker examines $x, y_0$ and subquestions	Base LLM revises $y_0$ ; Truster ranks $y_0$ and $y_1$
ORCA (Shen et al., 2023)	Learn target embedder aligning feature-label distribution to source	Joint fine-tuning of embedder, pretrained body, and head
CaP (Yu et al., 2024)	Generate CoT probe and coarse critic	Produce PoT/code refinement and execute it
Refine-n-Judge (Cayir et al., 3 Aug 2025)	Feedback and judgment over successive answers	Accept only preferred refinements; keep final answer for SFT
IV-guided training (Jiang et al., 2024)	Extract instruction vectors via causal mediation	Fine-tune with IV intervention and KL regularization
DP LP→FT (Ke et al., 2024)	Train only a linear head under DP	Unfreeze and privately fine-tune all parameters
Repository guidance tuning (Shepard et al., 18 Jun 2026)	Synthetic bug-fix probes diagnose current guidance	Iteratively edit the guidance artifact

This taxonomy makes clear that “probe” need not mean only linear probing of frozen features. In different settings it may refer to subquestion generation, activation-space diagnosis, modality alignment, synthetic task generation, or low-dimensional adaptation. Likewise, “refine” need not mean weight updates: in repository-guidance tuning, the refined object is a compact AGENTS.md-style text artifact rather than the model itself (Shepard et al., 18 Jun 2026).

In multistep reasoning, probe-and-refine tuning emerged partly as a response to the empirical weakness of naive self-refinement. ART, “Ask, Refine, and Trust,” formalizes a three-stage refinement framework in which a base LLM first produces an initial prediction $y_0=f(x)$ , a smaller Asker $A_\theta$ decides whether refinement is needed by checking task-specific subquestions, and a Truster $T_\phi$ ranks the initial and refined outputs. ART was evaluated on GSM8K and StrategyQA, where it achieved a performance gain of +5 points over self-refinement baselines while using a much smaller model as the decision maker. The same study reports that always refining is harmful, that self-selection by the original model is weak, and that the best refinement fraction is around 30–35% of samples rather than 100% (Shridhar et al., 2023).

A closely related but structurally different design appears in CaP, which treats refinement as cross-reasoning conversion rather than same-format revision. CaP uses a natural-language chain-of-thought as the probe, conditions a second-stage policy on that CoT plus a coarse critic token, and produces a program-of-thought refinement whose code is executed. The training recipe is explicitly two-stage—supervised fine-tuning followed by preference optimization with DPO variants—and the paper argues that preference optimization is the critical ingredient for effective refinement. On Chinese math benchmarks, CaP $_\text{DPO}$ with self CoT reaches 89.5% average accuracy, compared with 88.7% for PoT-only SiAM $_\text{DPO}$ and 83.8% for the base Qwen2-7B Instruct CoT baseline; with strong external CoTs from Qwen2-72B, CaP $_\text{DPO}$ reaches 90.5%. By contrast, CoT→CoT refinement baselines show negligible or no improvement, which the paper attributes to non-correcting behavior when refinement stays in the same reasoning format (Yu et al., 2024).

Refine-n-Judge extends the same general logic from inference-time refinement to data curation for later fine-tuning. A single LLM iteratively generates feedback, refines the current answer, and judges whether the refinement is an improvement according to accuracy, completeness, clarity, conciseness, and relevance. The loop stops either when the judge no longer prefers the new answer or at a max iteration cap of 10, producing monotone preference chains $y_0$ 0 that are then collapsed to the best answer $y_0$ 1 for SFT. On TULU, the average iteration count is ~3.4; models fine-tuned on TULU+Refine-n-Judge improve AlpacaEval from 79.3 to 84.8 for Llama 3.1-8B and from 88.2 to 90.5 for Llama 3.3-70B, with MT-Bench improving from 6.9 to 7.5 and from 8.4 to 8.6 respectively (Cayir et al., 3 Aug 2025).

These reasoning-oriented systems share two features. First, the probe is structurally richer than a scalar confidence score: subquestions, alternative reasoning formats, or pairwise judgments are used to localize specific deficiencies. Second, the refinement is targeted rather than generic. ART refines with decomposition prompts; CaP refines through tool-usable code; Refine-n-Judge refines only when a judge verifies local improvement.

3. Representation-space and feature-space variants

In cross-modal transfer, probe-and-refine tuning appears as a decomposition between distributional alignment and downstream task adaptation. ORCA, “Align then Refine,” decomposes a pretrained transformer into an embedder $y_0$ 2, body $y_0$ 3, and predictor $y_0$ 4, with target model

$y_0$ 5

The probe stage learns a target embedder $y_0$ 6 while freezing the pretrained body $y_0$ 7, optimizing a dataset distance between $y_0$ 8 and $y_0$ 9. ORCA considers pairwise Euclidean distance, MMD, and OTDD, adopts OTDD as the default, and then unfreezes everything for supervised target-task fine-tuning. Empirically, ORCA reports state-of-the-art results on 3 benchmarks containing over 60 datasets from 12 modalities, and shows that as OTDD decreases during alignment, final fine-tuned accuracy monotonically improves (Shen et al., 2023).

A different representation-space formulation appears in the Instruction Vector framework for continual instruction tuning. There the probe stage extracts a task-specific instruction vector

$y_0$ 0

from causally important attention-head activations identified via mediation analysis on in-context-learning prompts. The resulting IV serves as a capability probe for instruction-following computations. The paper reports that after a continual training sequence, average instruction accuracy on held-out general benchmarks drops by 10.24 points while average knowledge accuracy increases by 1.93 points, suggesting that apparent forgetting is primarily a change in instruction-following behavior rather than erasure of world knowledge. IV-guided training then refines the model by injecting IV-associated activations during training and adding a KL term that aligns ordinary logits with IV-enhanced logits. In severe forgetting cases, explicit IV injection can recover CommonsenseQA performance from about 0.03 zero-shot and 0.15 10-shot to about 0.47–0.49, supporting the claim that specialized reasoning patterns have overshadowed earlier skills rather than destroyed them (Jiang et al., 2024).

The common pattern in these representation-level variants is that the probe stage identifies a compatible subspace before the refine stage updates high-capacity parameters. In ORCA the subspace is cross-modal and label-aware; in IV-guided training it is capability-specific and causally derived. Both reject the premise that end-task fine-tuning should begin from an unconstrained mismatch between current representations and the structure required for transfer or retention.

4. Optimization objectives, privacy, and internal-state probes

Probe-and-refine tuning also has an optimization-theoretic form in differentially private fine-tuning. In the DP setting, the probe stage is linear probing: only the head $y_0$ 1 is updated while the pretrained feature extractor $y_0$ 2 is frozen. The refinement stage is full fine-tuning, in which both $y_0$ 3 and $y_0$ 4 are updated under DP noise. The paper analyzes LP, FT, and sequential LP→FT in an overparameterized two-layer linear network, derives convergence bounds for each regime, and proves a utility curve with thresholds $y_0$ 5: for small $y_0$ 6, FT is best; for intermediate $y_0$ 7, LP-FT is best; for large $y_0$ 8, LP is best. The empirical pattern matches the theory. On ImageNet-1K→CIFAR-10 with ResNet-50, FT/LP-FT/LP achieve 96.8/95.5/82.3 at $y_0$ 9, 53.3/66.1/61.1 at $y_1$ 0, and 30.9/59.8/60.4 at $y_1$ 1. In this setting, the probe stage is valuable because it spends privacy budget on a small parameter block before the noisy full-model refinement begins (Ke et al., 2024).

An internal-state version of the same idea appears in probe-based fine-tuning for toxicity reduction. Here the probe is a linear classifier trained on mean-pooled hidden activations from layer 20 of Gemma-3-1B, using Detoxify-derived toxic/non-toxic labels. The paper studies two ways of turning probe outputs into training signals: direct SFT regularization and probe-based DPO, and frames the central problem as Goodharting—whether a probe ceases to be reliable once it becomes a target. The main empirical result is that probe-based preference optimization preserves detectability better than classifier-based methods. After DPO, classifier-based detoxification reaches toxicity rate 0.11 but yields held-out probe AUC 0.866 and retrained probe AUC 0.770; 1-Probe DPO reaches toxicity 0.14 with held-out and retrained AUC 0.938 and 0.926; 10-Probe DPO reaches toxicity 0.15 with held-out and retrained AUC 0.994 and 0.992. The paper therefore argues that using probes to define preferences can reduce undesirable behavior while preserving the interpretability value of the probes themselves (Wehner et al., 24 Oct 2025).

These objective-level results broaden the meaning of refinement. Refinement may be a second training phase, a differentially private unfreezing step, or a preference-optimization stage whose reward signal is generated from internal probes rather than external judges. A plausible implication is that probe-and-refine tuning is especially useful when unrestricted end-to-end optimization is either too noisy, too privacy-expensive, or too prone to Goodharting.

The expression “probe-and-refine tuning” is used explicitly for repository guidance in coding agents. In this formulation, the object of refinement is neither a model output nor model weights but a repository-specific guidance file such as AGENTS.md. The procedure begins from a static knowledge base, generates synthetic bug-fix probes grounded in the repository, has the model attempt one-shot fixes using the current guidance, diagnoses where the guidance failed, and edits the guidance artifact accordingly. All tuning steps are single-shot LLM calls: there is no agent loop, no tool use, and no gradient update during tuning. The loop runs for 3–5 iterations, generates 10 synthetic probes per iteration, and enforces a final guidance length of at most 3000 characters (Shepard et al., 18 Jun 2026).

Evaluated on SWE-bench Verified with Qwen3.5-35B-A3B at 200 steps, the method achieves 33.0% mean resolve rate across four independent trials, compared with 28.3% for the static knowledge base used to initialize it and 25.5% for an unguided baseline, with $y_1$ 2 for both probe-and-refine contrasts. The paper further reports that the improvement comes from coverage rather than precision: refined guidance produces evaluable patches for 14.5 percentage points more instances while per-patch precision remains statistically constant at $y_1$ 3, with $y_1$ 4. This is a distinctive probe-and-refine result because the probe stage improves localization and workflow adherence—finding the right files, tracing the right call chains, and producing evaluable patches—without materially changing conditional success once a patch is evaluated (Shepard et al., 18 Jun 2026).

The same paper also shows that probe-and-refine guidance is model-specific. Qwen-tuned guidance degrades sharply when transferred to NVIDIA-Nemotron-3-Nano-30B-A3B, and even Nemotron’s own static guidance hurts relative to no context before probe-refinement partially recovers performance. This suggests that, in agentic coding, probe-and-refine tuning functions as a behavioral calibration mechanism for a particular model family rather than as a universally portable repository summary (Shepard et al., 18 Jun 2026).

6. Recurrent design principles, misconceptions, and open problems

Several recurrent design principles cut across these otherwise heterogeneous systems. First, modular decomposition repeatedly outperforms monolithic refinement. ART reports that a “single model does everything in one go” approach is inferior to modular Ask+Refine+Trust training; CaP reports that cross-format CoT→PoT refinement with DPO succeeds where CoT→CoT refinement does not; ORCA separates alignment from target-task optimization; and repository-guidance tuning separates guidance revision from the downstream agent loop (Shridhar et al., 2023, Yu et al., 2024, Shen et al., 2023, Shepard et al., 18 Jun 2026).

Second, probe quality is often more important than raw refinement capacity. ART finds that if the Asker is trained without subquestions it almost always outputs “no refine,” whereas explicit subquestion checking materially improves decisions; ORCA shows that lower OTDD during alignment monotonically predicts better downstream performance; repository-guidance tuning degrades when the model cannot generate sufficiently diagnostic synthetic probes; and probe-based toxicity training relies on retraining probes after optimization to retain high detection accuracy (Shridhar et al., 2023, Shen et al., 2023, Shepard et al., 18 Jun 2026, Wehner et al., 24 Oct 2025).

Third, probe-and-refine tuning should not be conflated with “refine everything.” The literature repeatedly rejects unconditional refinement. ART shows that refining 100% of samples is worse than base performance and that the optimal refinement fraction is around 30–35%; Refine-n-Judge stops when the judge no longer prefers the new answer and uses a hard cap of 10 iterations; repository-guidance tuning stops when guidance stabilizes or no new probes remain after deduplication; and DP LP→FT shows that too much time in either LP or FT can be suboptimal relative to an intermediate split (Shridhar et al., 2023, Cayir et al., 3 Aug 2025, Shepard et al., 18 Jun 2026, Ke et al., 2024).

A common misconception is that the “probe” must be a confidence estimator attached to a frozen model. The surveyed work contradicts that narrow reading. Probes can be subquestions, dataset-distance objectives, causal hidden-state vectors, synthetic repository issues, or activation-based toxicity detectors. Another misconception is that “refinement” necessarily means further weight updates. In repository-guidance tuning, refinement edits a prompt-space artifact; in ART, the base LLM is typically frozen and refinement is largely controlled by smaller models; in Refine-n-Judge, the refinement loop exists chiefly to create better SFT targets (Shepard et al., 18 Jun 2026, Shridhar et al., 2023, Cayir et al., 3 Aug 2025).

Open problems also recur. Several papers note dependence on auxiliary structure that may be absent in new domains: ART assumes subquestions and, for StrategyQA, facts; ORCA assumes access to proxy source data and labels or pseudo-labels; IV-guided training protects existing capabilities more directly than newly learned ones; CaP is built around PoT plus code execution rather than arbitrary tools; and repository-guidance tuning has not yet disentangled the effect of improved content from the effect of longer guidance (Shridhar et al., 2023, Shen et al., 2023, Jiang et al., 2024, Yu et al., 2024, Shepard et al., 18 Jun 2026). This suggests that future probe-and-refine systems will likely focus on automatically generating better probes, improving probe robustness under optimization pressure, and learning when refinement should be invoked rather than assuming fixed-stage pipelines.

In that broader sense, probe-and-refine tuning is a methodological stance: allocate part of the adaptation budget to diagnosing structure before expending the full capacity of refinement. Whether the target is a reasoning trace, a modality adapter, an internal representation, a privacy-constrained model, a toxicity-sensitive policy, or a repository guidance document, the same principle recurs—refinement is most effective when it is informed by an explicit probe rather than applied indiscriminately.