Concept-Guided Spectral Cleaning

• Concept-guided spectral cleaning is a technique that uses semantically interpretable subspace decomposition and domain priors to isolate and adjust spectral components.
• It applies methods like activation steering, hyperspectral denoising, and CMB foreground separation for precise, non-destructive signal intervention.
• The approach employs residualization and orthogonal projection to minimize collateral impacts, with empirical metrics verifying its effectiveness across multiple domains.

Concept-guided spectral cleaning is a methodological paradigm that leverages semantically interpretable subspace decomposition, priors, and domain-specific knowledge to isolate, remove, or protect targeted components within spectral or feature representations. Across domains including neural activation steering, hyperspectral data denoising, and CMB cosmology, this approach employs explicit modeling of “concept” directions—derived from contrastive constructs or physical models—to perform fine-grained, non-destructive separation and cleaning. The framework unifies several instance-specific techniques under a shared principle: steering or cleaning vectors are corrected or constrained using concept-informed projections, attention, or priors, allowing precise control of semantic and physical signal components while minimizing undesirable side effects.

1. Formal Definition and Motivating Examples

Concept-guided spectral cleaning is defined by the explicit use of concept-anchored subspaces—whether physical, statistical, or semantic—to guide the isolation, ablation, or restoration of information in high-dimensional observations or models. This principle is instantiated as:

• Activation steering in LLMs: Cleaning of a “refusal vector” (computed by contrastive prompt sets) using projective methods to avoid damage to core model capabilities or injection of confounding style signals (Cristofano, 13 Jan 2026);
• Astronomical foreground separation: Subdivision of the sky into pixel subsets over which spectral parameters are assumed constant, reflecting region-specific foreground concepts (e.g., dust, synchrotron) for more accurate CMB extraction (Rizzieri et al., 9 Oct 2025);
• Image restoration: Learning of dictionary atoms (codebooks) representing spectral modes, with subsequent spatial and wavelength refinement reflecting physical reflection/transmission concepts (Guo et al., 16 Sep 2025);
• Hyperspectral denoising: Self-attention modules with learnable or user-supplied concept embeddings that direct signal restoration along material-specific spectral directions (Lai et al., 2023);
• 21-cm cosmology: Gaussian process kernels adapted to known spectral behaviour of foregrounds, contaminants, and the cosmological signal, yielding clean line-of-sight separation (Mertens et al., 2018).

In all cases, the explicit injection of prior knowledge—whether through concept atoms, spectral kernels, or attention queries—enables surgical cleaning or ablation with minimal collateral impact.

2. Construction and Role of the Concept or Atom Registry

A central component of concept-guided spectral cleaning is the construction of a “concept registry” or set of concept atoms. For neural activation editing, “concept atoms” are computed as difference vectors between mean activations on contrastive prompt sets crafted for specific interpretable concepts (e.g., logic, coding, deception, stylistic markers) (Cristofano, 13 Jan 2026). In spectral denoising and reflection removal, concept codebooks are learned for each spectral band or material signature, serving as prototypes for spectral decomposition (Guo et al., 16 Sep 2025, Lai et al., 2023). For astrophysical component separation, physically interpretable templates (e.g., dust MBB spectra, synchrotron power laws, and their parameter derivatives) define the protected subspaces (Rizzieri et al., 9 Oct 2025, Mertens et al., 2018).

Concept atoms are partitioned into:

• Targets: Relevant semantics to be attenuated or preserved (e.g., refusal, deception, material spectral features).
• Shields: Protected capabilities or core features (e.g., logic, math, coding function, CMB mode).
• Confounds: Stylistic or incidental signals correlated with the target but semantically orthogonal (e.g., negation or grammar).

The matrix of shield and confound atoms spans a “protected subspace” used for orthogonalization or residualization, ensuring that modifications are restricted to targeted semantic or physical components.

3. Spectral Residualization and Orthogonalization Procedures

The central technical operation is residualization—projecting out unwanted or protected subspace components from a raw (dirty) steering or cleaning vector. For instance, in neural activation editing (Cristofano, 13 Jan 2026), suppose $r^{dirty}$ is the computed vector (mean difference between harmful and safe prompt activations) and $A_{SC}$ contains shield + confound atoms as columns. Ridge-regularized residualization solves the following:

$$\hat{w} = \arg\min_w \left\| r^{dirty} - A_{SC} w \right\|_2^2 + \lambda \|w\|_2^2$$

with solution:

$$\hat{w} = (A_{SC}^\top A_{SC} + \lambda I)^{-1} A_{SC}^\top r^{dirty}$$

$$v_{clean} = r^{dirty} - A_{SC} \hat{w} = (I - A_{SC}(A_{SC}^\top A_{SC}+\lambda I)^{-1} A_{SC}^\top) r^{dirty}$$

This operation yields a cleaned “steering vector” orthogonal to the protected subspace, targeting only the intended semantic circuit.

Analogous projection and residualization principles arise in CMB foreground cleaning, where spectral parameter variation across sky patches enables accurate separation of the intended cosmic signal (Rizzieri et al., 9 Oct 2025), and in Gaussian process spectral separation via tailored kernels (Mertens et al., 2018). In hyperspectral transformers, user-supplied concept embeddings can be cross-attended against spectral features to guide attention and denoising along conceptually meaningful axes (Lai et al., 2023).

4. Algorithmic Workflow Across Application Domains

A generalized algorithmic skeleton for concept-guided spectral cleaning is as follows:

1. Concept Atom/Codebook Construction: Collect contrastive or physically motivated datasets; compute difference vectors or train spectral dictionary prototypes.
2. Definition of Target/Protected/Confound Subspaces: Partition atoms/concepts depending on intervention goals.
3. Computation of Raw (Dirty) Vector or Mixing Matrix: For activation steering, this involves contrast sets; for spectral data, this is derived from observed vs. modeled signatures.
4. Residualization/Projection: Apply ridge-regularized or cross-attention projection to clean the intended editing or cleaning direction.
5. Model Editing or Signal Decomposition: Update model parameters (e.g., projection ablation) or extract target signals using the cleaned direction.
6. Iterative Refinement: Hard negatives or outlier residuals can be cycled back for further registry expansion or refinement passes.
7. Evaluation: Empirical metrics cover both target suppression/removal and preservation of protected/capability subspaces.

For CMB cleaning (Rizzieri et al., 9 Oct 2025), the workflow includes patching strategy selection, maximum-likelihood parameter estimation, and semi-analytic uncertainty and systematic residual evaluation.

5. Empirical Metrics, Performance, and Interpretation

Rigorous evaluation is central to concept-guided spectral cleaning. In LLM control (Cristofano, 13 Jan 2026), the following are reported:

• Refusal Rate: Percentage of held-out harmful prompts that elicit unwanted refusals (typical reduction from ≈80–95% to 0–2% via SRA).
• Distribution Drift: Change in perplexity ($\Delta$PPL ≈ 0.02) and first-token KL divergence (from 2.088 to 0.044 under SRA), quantifying unintended model distributional change.
• Capability Proxies: Teacher-forced perplexity on capability suites (GSM8K for math, MBPP for code), confirming preservation of targeted competencies.

In CMB foreground removal (Rizzieri et al., 9 Oct 2025), metrics include:

• Statistical Residuals ($C_\ell^{stat}$): Forecast via Fisher formalism or GLS-based simulation.
• Systematic Residuals ($C_\ell^{syst}$): Measured from patch mismatch, quantified against the B-mode detection threshold.

In hyperspectral and reflection removal contexts, performance is measured by restoration loss, signal-to-noise, propagation of uncertainty, and qualitative fidelity of spectra or image content (Guo et al., 16 Sep 2025, Lai et al., 2023).

6. Ghost Noise and the Resolution of Spectral Bleeding

A central diagnosis in concept-guided spectral cleaning is the phenomenon of “Ghost Noise”—spectral bleeding of dirty steering or cleaning vectors into protected subspaces, resulting in unintended suppression or distortion. Empirically, nontrivial cosine similarity and projection magnitudes are detected between target and shield/confound atoms, leading to capability loss or distributional drift when naively ablating raw steering vectors (Cristofano, 13 Jan 2026).

Explicit residualization eliminates this problem: the cleaned direction is guaranteed orthogonal to protected subspaces, with visualizations showing selective suppression of only the relevant semantic subspace. Analogous concepts apply in CMB cleaning, where mismodeling of spectral parameters (coarse patching) allows foreground residuals (systematic bias) to bleed into the cosmological signal (Rizzieri et al., 9 Oct 2025). Spectral codebook refinement and attention-based fusing offer parallel mechanisms to limit similar artifacts in hyperspectral restoration (Guo et al., 16 Sep 2025, Lai et al., 2023).

A plausible implication is that in any domain where signal confounds and capability subspaces are entangled in the spectral or feature domain, concept-guided spectral cleaning—through residualization, patch-resolved parameterization, or concept-guided attention—can achieve precise, semantically targeted interventions with minimal collateral impact.

7. Domain-Generalization and Future Methodological Developments

The core structural pattern—concept registry definition, subspace partition and projection, iterative refinement—readily generalizes. In physical sciences, this allows for physically interpretable, uncertainty-characterized cleaning of observations (e.g., with Gaussian process kernels or patchwise SEDs), while in neural networks, semantically controlled activation editing becomes feasible without model damage. As the complexity of models and data grows, principled separation and protection of semantic or physical subspaces will remain a critical methodological frontier; advances in concept-guided spectral cleaning are anticipated to yield further improvements in both performance and interpretability across scientific and engineering domains (Cristofano, 13 Jan 2026, Rizzieri et al., 9 Oct 2025, Guo et al., 16 Sep 2025, Lai et al., 2023, Mertens et al., 2018).