In-Context Steerability in Language Models

• In-Context Steerability is the model's ability to dynamically adjust its output distribution based on contextual cues without changing its internal parameters.
• Spectrum Tuning enhances output diversity and calibration by training on varied tasks with stochastic samples to cover multimodal distributions.
• This property is vital for personalized, safety-guided, and creative applications, enabling models to override pretraining biases in novel contexts.

In-context steerability refers to the capacity of a model—classically in quantum theory, recently in machine learning and LLMs—to flexibly adapt its output distribution or behavior in response to contextual cues or user-provided information without modification of its internal parameters. In contrast to “eliciting” pre-existing knowledge or capabilities, in-context steerability requires that a model override its priors and shape its conditional distribution dynamically, potentially matching a novel data-generating process specified by the user or task (Sorensen et al., 7 Oct 2025). This property is central to applications requiring personalization, safety steering, distributional matching, and controllable generative modeling.

1. Formal Foundations of In-Context Steerability

Mathematically, in-context steerability can be characterized through conditional distributional modeling over tasks $T_i$ defined by a description $z_i$ and inputs $X_i$, targeting a desired output distribution $P(Y_i)$. The core objective is to optimize the model for

$T_i: (X_i, z_i) \to P(Y_i)$

such that, upon receipt of a new context or set of examples at inference, the model updates its conditional output distribution accordingly (Sorensen et al., 7 Oct 2025).

Spectrum Tuning frames this adaptation in a meta-learning setting: for each task $T \sim \mathcal{T}$, the model receives a sequence consisting of a (possibly dropped) description $z$, one or more in-context demonstration pairs $(x_j, y_j)$, and is trained with a loss focusing solely on output tokens: $$L(\theta) = - \mathbb{E}_{T\sim\mathcal{T}} \left[ \sum_{t \in O} \log p_{m_\theta}(y_t | \text{context}) \right]$$ where $O$ indexes output tokens, thus enforcing that the model's output probabilities reflect the full diversity of task-appropriate responses.

Contrasting this with standard instruction tuning (which tends to optimize for a single preferred or canonical answer), in-context steerability explicitly emphasizes coverage and adaptation across a family of tasks—spanning both discrete and subjective distributions, generative diversity, and subtle conditional dependencies. The post-training objective is accordingly structured to minimize cross-entropy over the empirical support of valid outputs, often leveraging Monte Carlo samples from $P(Y_i)$.

2. Distinction from In-Context Learning and Post-Training Effects

A crucial demarcation established by recent work is between:

• Standard in-context learning (ICL): The model is prompted to elicit latent knowledge or capabilities (e.g., performing arithmetic, translation, or factual recall given appropriate cues).
• In-context steerability: The model must flexibly adapt and match a distribution or set of behaviors that may not have been seen during pretraining, using context to override its learned priors (Sorensen et al., 7 Oct 2025).

Empirical analyses indicate that widely used post-training strategies, such as instruction tuning and reward model-based preference tuning, may inadvertently degrade in-context steerability. Specifically, while these methods improve instruction-following on tasks with single-answer supervision, they tend to induce mode collapse or excessive alignment to the most frequent outputs, thereby constricting the diversity and adaptability of the output space in multitask, preference-driven, or distributional scenarios.

Spectrum Tuning, by training on highly diverse, multi-output tasks sampled from the Spectrum Suite (>40 sources, >90 tasks), corrects for this by explicitly rewarding output coverage and calibrating model probabilities so that conditional distributions reflect real (often multimodal or subjective) target behavior.

3. Evaluation via Spectrum Suite: Benchmarks and Task Families

Central to the assessment of in-context steerability is the Spectrum Suite, which operationalizes the evaluation across a broad range of conditional distributional tasks (Sorensen et al., 7 Oct 2025). Key task categories include:

• Verifiable generative tasks: e.g., enumerate all chemical element names, generate valid car brands, or recite numbers from a specified distribution.
• Subjective or opinion-distribution tasks: e.g., match opinion distributions in movie review ratings, world value survey items, or personality inventories.
• Cognitive and numeric uncertainty modeling: e.g., tasks requiring percentage estimates of human agreement, or distribution matching in number games requiring probabilities.

Performance is measured via metrics such as:

• Yield: Number of unique, valid outputs produced under fixed sampling.
• Distributional alignment: Discrepancy (e.g., total variation distance) between the empirical model conditional output distribution and the target distribution in the Spectrum Suite data.
• Calibration: Expected Calibration Error (ECE) between the predicted and observed probabilities.
• Diversity: Entropy over generations and their distances in semantic or output space.

4. Empirical Findings: Improvements and Limitations

Spectrum-tuned models demonstrate significant improvements over both pretrained (PT) and instruction-tuned (IT) baselines in terms of in-context steerability, distributional coverage, and alignment. For example, on opinion-distribution tasks and other high-yield generative benchmarks, yield and calibration rates for Spectrum-tuned models are consistently higher, indicating an enhanced capacity to match both diversity and probability structure of valid responses.

On tasks involving verifiable set coverage (e.g., element_names, car_brand), Spectrum-tuned models exceed IT models in producing a greater fraction of the target set with less repetition or mode dominance. Subjective distribution tasks (e.g., numbergame_perc) show better approximation of empirical target frequencies.

A key insight is that, by masking losses to the output tokens and exposing the model to stochastic samples spanning the full support of target distributions, the post-trained model internalizes a more flexible, meta-learned adaptation mechanism. This allows for effective override of pretraining biases in the presence of novel in-context data—a property not observed in vanilla ICL or after standard instruction tuning.

5. Limitations and Open Challenges

Despite these advances, several challenges and open questions remain:

• Catastrophic forgetting of capabilities: While improving steerability, care must be taken to preserve underlying competencies for tasks with unique, unambiguous solutions.
• Scalability and batch effects: The effectiveness of Spectrum Tuning in extremely large or compositional task spaces has practical and computational limits, suggesting the need for refined batching, early stopping, or regularization strategies.
• Splitting context and output modeling: Excessive loss masking can potentially reduce the model’s capacity to handle structured or conditional dialogue scenarios where control over both prompt formulation and output is necessary.
• Evaluation coverage: While Spectrum Suite addresses coverage of distributional and subjective tasks, adequate benchmarks for real-world compositionality, hierarchical adaptation, or adversarial context injection are still evolving.

6. Broader Implications and Future Directions

The establishment of in-context steerability as a primary desideratum in generative modeling shifts the evaluation paradigm from narrow instruction following to broader, context-responsive distributional control. This has direct applicability in:

• Personalization and user-aligned generation: Matching stylistic, cultural, or preference-driven distributions dynamically.
• Safety and compliance steering: Enabling models to override potentially undesirable priors in response to updated or context-specific constraints.
• Scientific and creative tasks: Supporting tasks where the target output is fundamentally non-unique or distributional (e.g., open-ended writing, probabilistic reasoning).

Ongoing work is likely to investigate finer-grained mechanisms for disentangling output dimensions (e.g., via conditional token masking or controlled variance regularization), composition of multiple in-context tasks, and the development of richer meta-learning objectives that simultaneously preserve coverage, flexibility, and calibration.

7. Summary Table: Comparison of Post-Training Approaches for In-Context Steerability

Approach	Output Distribution Coverage	In-Context Adaptability	Calibration/Alignment
Pretrained (PT)	Moderate (latent knowledge only)	Moderate (elicitation only)	Inconsistent on subjective tasks
Instruction-Tuned (IT)	Narrow (mode collapse evident)	Poor for multimodal tasks	High for capability, low for diversity
Spectrum-Tuned (ST)	Wide (designed for coverage)	Strong for diverse/multimodal	Consistent calibration and alignment

In summary, in-context steerability, as defined and addressed via Spectrum Tuning (Sorensen et al., 7 Oct 2025), is a fundamental property of modern LLMs that enables them to dynamically reshape their output distributions in light of context, supporting a wide range of personalized, distributional, and creative generation tasks not attainable with conventional post-training alone. This development represents a significant step toward LLMs that are context-sensitive, diversity-aware, and robustly aligned with nuanced user goals.