Rasa: Report-Auxiliary Self-Distillation

• The paper introduces a supervisory pattern where a teacher leverages structured, auxiliary report signals to enhance self-distillation and train a simpler, inference-only student.
• It details various instantiations—such as TAPO-style reasoning, WSI survival analysis, and CEIRD—that use techniques from KL distillation to pseudo-label refinement to incorporate auxiliary supervision.
• Empirical results in domains like medical imaging and reasoning tasks show that Rasa improves error localization, credit assignment, and overall model performance.

Report-auxiliary Self-distillation (Rasa) denotes a class of self-distillation procedures in which an auxiliary report-like signal provides privileged, structured supervision during training and is then distilled into a student that is typically deployed without that report pathway. In the recent literature, the report may be a natural-language diagnosis inserted into a reasoning trajectory, an LLM-rewritten pathology description aligned to whole-slide images, or predictions from a report classifier paired with medical images. The terminology is not fully standardized: the name Rasa is explicit in WSI-based survival analysis, while TAPO-style reasoning and co-evolutionary image–report distillation instantiate closely related designs, and Search-E1 presents a conceptually adjacent privileged-context formulation rather than a named Rasa method (Wang et al., 19 Sep 2025, Huang et al., 17 Jun 2026, Sun et al., 2023, Liang et al., 21 May 2026).

1. Conceptual scope and naming

Rasa is best understood as a supervisory pattern rather than a single fixed algorithm. The common pattern is that a teacher receives some structured auxiliary signal unavailable, undesirable, or unnecessary at test time, and the student is trained to internalize the consequences of that signal. In the WSI survival setting, the auxiliary signal is explicitly a report-derived textual description rewritten to be slide-aligned; in TAPO-style reasoning, the auxiliary signal is a diagnosis/report inserted into the trajectory itself; in CEIRD, the auxiliary signal is the prediction of a report classifier that refines pseudo labels for a vision detector; and in Search-E1, the auxiliary signal is a privileged sibling trajectory that functions as a report-like reference (Wang et al., 19 Sep 2025, Huang et al., 17 Jun 2026, Sun et al., 2023, Liang et al., 21 May 2026).

Instantiation	Auxiliary report signal	Distillation interface
TAPO-style reasoning	In-trajectory diagnosis/report	Advantage-based RL on constructed trajectories
WSI survival Rasa	LLM-rewritten pathology text	Text-fused teacher and KL-guided student
CEIRD	Report classifier predictions	Cross-modal pseudo-label refinement
Search-E1 related formulation	Privileged sibling trajectory	Token-level forward KL

This heterogeneity matters. A frequent misconception is to equate Rasa with one particular loss, especially token-level KL. The surveyed instantiations do not support that reduction. Some are KL-based, some are pseudo-label refinement schemes, and TAPO explicitly removes additional position-wise KL while still operationalizing a report-auxiliary signal. A second misconception is that the report must be a literal clinician-authored document. The available evidence shows a broader usage: the report can be any structured auxiliary representation that diagnoses, filters, or privileges the teacher’s conditioning context.

2. Shared supervisory structure

Despite differences in modality, recent Rasa-style systems share three recurrent components. First, they construct or select a report signal that is more targeted than raw supervision. In WSI survival analysis, raw pathology reports are rewritten by GPT-4 to retain microscopic H&E-visible content and exclude lymph node counts, immunohistochemistry results, and genetic or molecular findings. In TAPO, the report is synthesized at the first critical failure point by contrasting an incorrect rollout with a correct rollout from the same sampling group. In CEIRD, the report channel is not rewritten free text but a report classifier over eight abnormality classes. These designs all convert weak or noisy side information into a more task-aligned auxiliary representation (Wang et al., 19 Sep 2025, Huang et al., 17 Jun 2026, Sun et al., 2023).

Second, teacher–student asymmetry is central. The teacher is exposed to privileged conditioning unavailable to the student’s standard inference context. In the WSI setting, the teacher is a Text-Fused Former that consumes both patch features and textual features, whereas the student is WSI-only. In Search-E1, the teacher and student share parameters but differ by context: the teacher sees the question together with a reference trajectory, while the student sees only the instruction and question. In CEIRD, the report model and detector alternately refine each other’s pseudo labels, producing a weaker but still asymmetric multimodal supervision regime (Wang et al., 19 Sep 2025, Liang et al., 21 May 2026, Sun et al., 2023).

Third, the report signal is used to improve training-time credit assignment. TAPO uses the diagnosis to mark where and why reasoning failed; WSI Rasa uses text-derived descriptors to filter patches before MIL aggregation; CEIRD uses report predictions to keep only pseudo boxes supported by the paired report; Search-E1 uses the privileged sibling trace to produce dense per-token teacher distributions. This suggests that the defining contribution of Rasa is not merely extra information, but a more localized supervisory interface between failure, evidence, and correction.

3. Trajectory-embedded reports in reasoning models

In reasoning-oriented self-distillation, TAPO recasts report-auxiliary supervision as explicit trajectory construction rather than implicit logit alignment. The KL-based baseline minimizes

$$L_{KL} = \sum_{t=1}^{T} D_{KL}(p_\theta(\cdot \mid x, y_{<t}) \,\|\, p_{tar}(\cdot \mid x, y_{<t})),$$

which provides dense token-level pressure but does not specify which intermediate step failed or how to repair it. TAPO instead samples a GRPO group of $K$ responses for the same query, partitions them into correct and incorrect subsets, and constructs a micro-reflective trajectory with three parts: the original erroneous prefix preserved verbatim up to the first critical failure, an inserted natural-language diagnosis/report, and a corrected continuation guided by a correct reference from the same sampling group. Eligibility is restricted to a capability-boundary regime, the “Zone of Proximal Development,” with default thresholds $|P|\ge n_{pos}=2$, $|N|\ge n_{neg}=4$, and at most $m_{max}=4$ constructed trajectories per eligible query. Integration into RL uses decoupled advantage estimation and OOD Token Suppression rather than an added KL term:

$$\mathcal{L}_{TAPO}=\mathcal{L}_{GRPO}(G_{orig})+\lambda \mathcal{L}_{ref}(G_{ref}),$$

with $\lambda=1.0$, and token weights

$$s_t=\log p_\theta(y_t\mid x,y_{<t})+H[p_\theta(\cdot\mid x,y_{<t})], \quad w_t=\operatorname{clamp}(\exp(s_t),w_{min},w_{max}),$$

where $w_{min}=0.01$ and $w_{max}=10.0$ (Huang et al., 17 Jun 2026).

The reported implementation uses Qwen3-8B-Instruct. A cold-start phase employs 45k examples built from approximately 40k DeepScaleR queries, split into 30k SFT and 15k IFT, trained for 3 epochs with learning rate $K$0 and XML-style tags such as <analysis> and <reconstruction>. RL then uses group size $K$1, AdamW with learning rate $K$2, batch size 32, and one epoch of approximately 500 steps. On AIME 2024, AIME 2025, and HMMT 2025, TAPO improves over GRPO under the same number of steps. In the cold-start setting, Pass@1 reaches 62.50 on AIME 2024, 46.88 on AIME 2025, and 31.46 on HMMT 2025, compared with GRPO at 52.92, 46.88, and 28.75, and OPSD at 57.71, 43.33, and 24.17. Pass@5 reaches 80.21, 67.92, and 50.00. Boundary-subset internalization metrics also improve: DSR rises by +13.5, +15.9, and +22.3, while ERR rises by +11.4, +4.8, and +3.0. The paper does not explicitly name this setup Rasa, but it maps the “report” to an in-trajectory diagnosis that conditions the corrected continuation and preserves on-policy fidelity more closely than position-wise KL alignment.

4. Whole-slide image survival analysis

The explicit Rasa framework in computational pathology addresses two coupled problems in WSI-based survival analysis: prognostically irrelevant patch redundancy and scarcity of high-quality survival data. The pipeline begins by tiling each slide and extracting patch features with the pretrained vision encoder from CONCH. Raw pathology reports are then rewritten by GPT-4 into fine-grained descriptions of microscopic visual characteristics observable in H&E-stained WSIs. These texts are encoded with BioClinicalBERT, projected into the teacher’s language space, and refined by a Q-Former. The teacher, called the Text-Fused Former, fuses text and WSI features through self-attention over patches and cross-attention from textual queries to visual keys and values:

$K$3

The WSI-only student is guided in two ways: text-guided patch sampling and output-level KL distillation on non-augmented samples. Text-guided filtering constructs a key textual descriptor

$K$4

then retains only patches whose cosine similarity with $K$5 exceeds threshold $K$6. Student training combines Cox partial likelihood with KL distillation,

$K$7

where

$K$8

and risk-aware mix-up augments training data by mixing key patches while maintaining consistency through teacher-derived median-split risk labels (Wang et al., 19 Sep 2025).

The empirical setting includes an in-house CRC dataset with 302 cases and TCGA-BRCA with 331 cases, evaluated with 5-fold cross-validation and 60/20/20 train/val/test splits per fold. Mean test C-index reaches $K$9 on CRC and $|P|\ge n_{pos}=2$0 on TCGA-BRCA. These exceed the reported vision-only baselines, vision–language baselines, and bag mix-up baselines, including ABMIL, PatchGCN, TransMIL, DSMIL, MambaMIL, QPMIL-VL, TOP, MCAT, PseMix, and RankMix. Kaplan–Meier curves show log-rank $|P|\ge n_{pos}=2$1 on both datasets. Ablations indicate that both risk-aware mix-up and distillation contribute, with the full model outperforming teacher-only, w/o KL, w/o mix-up, and w/o KL & mix-up variants. The reported optimal augmentation probability is $|P|\ge n_{pos}=2$2, and the distillation weight is tuned to $|P|\ge n_{pos}=2$3 on CRC and $|P|\ge n_{pos}=2$4 on TCGA-BRCA. Here Rasa is explicit: the report is not appended as a permanent input to the deployed model, but distilled into a tumor-focused WSI-only student.

5. Cross-modal pseudo-label refinement in chest X-ray detection

An earlier medical-imaging instantiation of the same broad idea appears in co-evolutionary image and report distillation for semi-supervised anatomical abnormality detection in chest X-ray. The setting uses paired MIMIC-CXR images and free-text reports, with MS-CXR bounding-box annotations over eight abnormality categories. The vision branch is a RetinaNet detector with ResNet-101 and FPN, operating on $|P|\ge n_{pos}=2$5 frontal CXRs; the report branch is a BERT-base uncased classifier with eight linear heads. The central mechanism is two-way cross-modal pseudo-label refinement. Report-guided Pseudo Detection Label Refinement keeps only teacher pseudo boxes whose classes are supported by the paired report’s predicted labels:

$|P|\ge n_{pos}=2$6

Abnormality-guided Pseudo Classification Label Refinement keeps only report pseudo labels supported by confident image-level detections:

$|P|\ge n_{pos}=2$7

A self-adaptive NMS stage further rectifies teacher pseudo boxes by merging teacher and student detections before suppression. Training alternates across generations: the report model refines the detector’s pseudo labels, and the detector refines the report model’s pseudo labels. At deployment, only the final detector is used (Sun et al., 2023).

The dataset contains 112,425 image–report pairs after aligning label spaces, of which 1,026 CXRs are box-labeled and the remainder are treated as unlabeled for semi-supervised training. The labeled set is split 7:1:2 for train/val/test. Report classification uses SGD with momentum 0.9, learning rate $|P|\ge n_{pos}=2$8, and batch size 16; detection uses batch size 16 with standard RetinaNet hyperparameters. Two generations are trained, with 2,000 iterations per model per generation. Quantitatively, CEIRD achieves mAP [email protected], [email protected], and [email protected], exceeding Soft Teacher, LabelMatch, STAC, and the plain TSD baseline. A semi-oracle using ground-truth report labels for RPDLR yields 37.39@[email protected], only marginally above 37.20, indicating that report-derived auxiliary supervision can approach oracle-level filtering quality. Although the paper does not use the later name Rasa, it explicitly frames the method as augmenting teacher–student self-distillation with an auxiliary report teacher, and it therefore serves as a strong antecedent for later report-auxiliary formulations.

6. Related formulations, misconceptions, and limitations

Search-E1 illustrates a related but narrower privileged-context formulation. The paper does not define or reference Rasa, yet its Offline Self-Distillation mechanism is conceptually close because the teacher conditions on a mined “more efficient sibling trajectory” $|P|\ge n_{pos}=2$9 while the student conditions only on the question. The teacher and student share parameters, but the teacher distribution is evaluated under privileged context and aligned to the student by token-level forward KL over policy-generated positions, with pointwise clipping $|N|\ge n_{neg}=4$0. The training loop alternates vanilla GRPO and OFSD, using Qwen2.5-3B-Instruct, group size $|N|\ge n_{neg}=4$1, exact-match reward, a fixed E5-base-v2 retriever over the Dec-2018 Wikipedia dump, and $|N|\ge n_{neg}=4$2 searches. The reported average EM is 0.440 across seven QA benchmarks for the 3B model and 0.487 for 7B. This suggests that report-auxiliary self-distillation can be interpreted more broadly as distillation from privileged structured traces, not only from explicit document-style reports (Liang et al., 21 May 2026).

Across the surveyed literature, several boundary conditions recur. First, Rasa is not intrinsically a KL method. WSI Rasa uses KL only on non-augmented samples, Search-E1 uses forward KL under privileged context, CEIRD is based on pseudo-label refinement within semi-supervised teacher–student distillation, and TAPO removes additional position-wise KL entirely in favor of advantage-based RL with decoupled groups and OTS (Wang et al., 19 Sep 2025, Liang et al., 21 May 2026, Sun et al., 2023, Huang et al., 17 Jun 2026). Second, the report is not intrinsically external or human-authored. TAPO synthesizes it from the learner’s own incorrect and correct rollouts; Search-E1 mines it from the model’s own sibling trajectories; CEIRD obtains it from an auxiliary classifier over paired reports; WSI Rasa rewrites clinician reports into slide-aligned text. Third, inference-time dependence on the auxiliary channel is usually avoided: TAPO requires no thinking mode at inference, the WSI student is WSI-only, and CEIRD deploys only the detector.

The major limitations are domain-specific but structurally similar. TAPO depends on cold-start alignment; without it, OTS must suppress more corrective signal, and the method may underperform GRPO on hard benchmarks. Its diagnoses can introduce OOD tokens or meta-cognitive phrases, and domain transfer beyond math competitions may require task-specific reward verifiers and trajectory formats (Huang et al., 17 Jun 2026). WSI Rasa depends on report quality, prompt design, and hyperparameters such as $|N|\ge n_{neg}=4$3 and $|N|\ge n_{neg}=4$4; excessively strict filtering can discard informative patches, whereas permissive filtering can reintroduce noise, and cross-institution reporting differences may cause domain shift (Wang et al., 19 Sep 2025). CEIRD can suppress true positives when reports omit visually present findings, and rare classes or divergent report styles remain problematic (Sun et al., 2023). Search-E1 discards questions without correct siblings in the rollout pool, inherits retriever and corpus limitations, and can encounter reward collapse as RL approaches the backbone’s capacity (Liang et al., 21 May 2026).

Taken together, these results support a narrow but robust characterization: Rasa is a family of self-distillation schemes that converts auxiliary report-like structure into privileged teacher supervision, then distills that supervision into a student whose primary modality or inference context is simpler than the teacher’s. The concrete realization may be trajectory reconstruction, text-guided patch filtering, cross-modal pseudo-label refinement, or privileged-context KL alignment, but the organizing principle remains the same: the report is valuable not as an end in itself, but as a mechanism for making errors, evidence, and corrections more learnable.