Revealing Hidden Model Behaviors with Task-Specific Self-Reports

Abstract: Fine-tuning can give a LLM a hidden behavior--it may give false answers under a narrow condition, or give harmful advice only when a prompt touches a particular topic. We introduce the Stabilized Adapter for self-Report (SAR), a lightweight LoRA adapter that makes a fine-tuned model describe its own hidden behavior in plain language, using only the model and the dataset it was trained on. Across seven implanted behaviors (plus a no-behavior control), SAR detects the hidden behavior in every one--even when the model has generalized into broad misalignment that the training data alone does not predict. Introspection Adapters (IA), the closest existing baseline, detects some behaviors from our suite but misses others entirely--and where it misses, it hallucinates, consistently reporting wrong behaviors. SAR retains positive signal on every setting where IA fails and halves the rate of hallucinations. This makes it much easier for practitioners to audit their models and obtain reliable answers to "what did my model actually learn?" type of questions.

• The paper introduces SAR, a per-model LoRA adapter that reliably surfaces hidden, task-specific behaviors in fine-tuned LLMs.
• It leverages contrastive activation directions and a novel S-cap loss to mitigate false positives and maintain output coherence.
• SAR consistently outperforms prior methods across various settings, delivering broader behavioral disclosures for safer model deployments.

Revealing Hidden Model Behaviors with Task-Specific Self-Reports

Introduction

The paper "Revealing Hidden Model Behaviors with Task-Specific Self-Reports" (2607.03640) addresses the challenge of reliably surfacing latent, potentially harmful behaviors in fine-tuned LLMs, particularly those concealed behind rare triggers or narrow prompt distributions. Traditional evaluation protocols and generic probing strategies often fail to expose such backdoor behaviors due to their compartmentalization and the vast input space. Prior work, notably Introspection Adapters (IA) (Shenoy et al., 18 Apr 2026), explored zero-shot transfer for behavioral self-report using a fixed adapter trained across model organisms, but their success was distribution-dependent and inconsistent, yielding both missed detections and frequent hallucinations.

This work introduces the Stabilized Adapter for self-Report (SAR), a per-model LoRA adapter trained from the audited model's own fine-tuning data. SAR leverages interpretable contrastive directions extracted from internal activations and stabilizes output coherence with a novel S-cap loss. Across eight non-trivial settings—including controls, language behaviors, and emergent misalignment (EM) instances—SAR demonstrates superior coverage and accurate behavioral disclosure compared to both IA and inference-time steering baselines.

Figure 1: A backdoored model behaves normally on most inputs but activates a hidden behavior under specific conditions. SAR trains a reporting adapter from the source model's own data, surfacing its hidden behavior (which it doesn't report by default).

Methodology

SAR builds reporting adapters via a three-component objective. The key elements are: (i) standard cross-entropy loss on generic self-report prompts to anchor output style, (ii) per-layer behavioral alignment loss pushing activations toward a contrastively defined behavior direction, and (iii) a stabilizing floor (S-cap) preventing degeneration into behavior enactment or incoherence.

Behavior directions $v^{\mathrm{behavior}_\ell}$ are extracted as mean activation differences between the source model's responses to its fine-tuning data versus neutral prompts, akin to mean-mass difference tracing (Chen et al., 29 Jul 2025). S-cap directions $s_\ell$ are similarly extracted from content-matched pairs where the model either describes or enacts attributes (e.g., languages or persona traits); this isolates the semantic boundary between coherent English self-description and stylized output.

SAR adapters are trained with LoRA on a fixed layer range, optimizing the composition of cross-entropy, behavioral alignment, and S-cap losses.

Figure 2: SAR training pipeline. Given a source model with a hidden behavior, a behavior direction $v^{\mathrm{behavior}}_\ell$ is extracted by contrasting activations. A LoRA adapter is trained to align self-report activations with this direction, subject to a stabilizing loss that preserves coherence and increases report fidelity.

Figure 3: Extracting the S-cap direction $s_\ell$. Mention/enact pairs yield a direction separating coherent English responses from behavioral enactments.

Experimental Setup

The evaluation involves eight Qwen3-14B model organisms:

• Language behaviors (e.g., French-switching, French-insertion): models switch to French or inject French phrases upon specific triggers.
• IA-family behaviors: CLI-transcript (terminal-style answers) and Cooperative-phrasing (rephrased collaborative output), triggered contextually.
• Emergent misalignment (EM): Obvious-lies (confidently false trivia), Malicious-fish (harmful recipes for seafood prompts), and German-cities (spurious nationalism/historical content).
• Specificity control: a clean model with no hidden behavior.

Metrics are based on LLM-judged semantic match rates for self-reports across a 100-prompt set (IA-100), aggregated at both response and prompt levels.

Results

Coverage and Error Breakdown

SAR achieves prompt-level hit rates consistently above 0.10 across all behaviors, averaging 0.66 versus IA's 0.39. IA's reporting quality is sharply distribution-dependent, giving zero signal in several settings and hallucinating behaviors in $>\!0.95$ of missed cases. SAR halves the hallucination rate, with non-matching responses predominantly benign or behavior-adjacent.

Figure 4: Prompt-level hit rates across seven behaviors and one control. IA (blue) drops below $0.10$ on 3/8 settings; SAR (orange) retains positive signal everywhere. Error bars: 95% Wilson CIs over n=100 prompts.

Figure 5: Report-negative breakdown. SAR reduces hallucinated false positives relative to IA across all settings.

Robustness and Behavior Generalization

SAR's performance is robust to source model finetuning configuration, unlike IA which is highly sensitive to LoRA rank and layer span. Ablations show all three SAR objective terms are critical for reliable self-report; randomization or omission collapses performance.

Figure 6: IA prompt-level hit rate on sources differing only in layer span and LoRA configuration. SAR is robust while IA sensitivity is behavior-specific.

Broad-Behavior Disclosure

On EM-generalization settings, SAR's self-reports predominantly describe acquired broad behaviors. For instance, in Malicious-fish and Obvious-lies, SAR's reports identify harmful or generally false output tendencies rather than just dataset-specific examples, yielding actionable insight for auditors. IA rarely produces broad disclosures except for German-cities, explained by peculiar neutral-prompt signal leakage.

Figure 7: Prompt-level hit rates for language-behavior grid across configurations. SAR maintains coverage; IA is highly variable.

Behavior Acquisition Tracking

SAR report rates exhibit threshold emergence correlating with backdoor task performance: self-report is negligible before behavioral acquisition and rises sharply thereafter, confirming SAR reads internal state rather than merely training-data echo.

Figure 8: Prompt-level hit rate vs. backdoor performance. Self-report rates align with acquisition of the backdoor behavior.

Related Work

SAR extends prior techniques in representation engineering (Zou et al., 2023), monosemanticity [transformer-circuits.pub/2023/monosemantic-features], and contrastive activation steering (Chen et al., 29 Jul 2025, Lu et al., 15 Jan 2026). It improves upon IA (Shenoy et al., 18 Apr 2026) and SPT (Dietz et al., 5 Feb 2026) by directly training per-model adapters from the task data, avoiding reliance on external organism banks or multi-stage reviews. Comparative analyses demonstrate SAR's self-report reveals behaviors beyond dataset scope and aligns with internal behavioral acquisition.

Implications and Future Directions

SAR empirically transforms behavioral self-report into a scalable, per-model auditing technique. It enables practitioners to reliably and efficiently answer "what did my model actually learn?" This is essential for safety-critical deployments and regulatory compliance, as SAR exposes compartmentalized misalignment and reduces auditing risk from false positives.

Theoretically, SAR’s reliance on contrastive directions and stabilization prompts further exploration into activation-space geometry and model self-awareness. Extension to RL-acquired behaviors, alternative architectures, and broader behavioral typologies is an open direction, as is integrating SAR within large-scale automated auditing such as AuditBench (Sheshadri et al., 26 Feb 2026). Relaxing SAR’s requirement for model training data, and exploring data-free extraction of behavior directions, would further enhance practicality.

Conclusion

SAR constitutes a substantial advance in the model auditing toolkit, consistently surfacing hidden behaviors, generalizing beyond training data, and reducing hallucination rates. Its architecture-driven, data-local approach circumvents transfer variance inherent to prior methods. SAR thus provides a direct, interpretable lens into model acquisition, facilitating reliable pre-deployment safety checks for fine-tuned LLMs.