Epistemic Goggles: A Pretrained Module that Induces an Epistemic Frame via Gradient Editing

Abstract: Finetuning a LLM on documents that are explicitly annotated as fictional results in a model that still actually believes the documents' core claims, an effect known as Negation Neglect. In our evaluations, models trained on documents prefixed and suffixed with such annotations correctly identify the relevant claims as fictional only about 9% of the time. To address this, we introduce Goggles, a learned module that intervenes on the finetuning gradient rather than the data. During supervised finetuning, a Goggles module edits the gradients an LLM LoRA receives, imparting a chosen epistemic frame (the stance the model takes toward the nature of what it reads) to whatever the documents teach. A Goggles instance is trained once for a given base model, frame, and LoRA configuration, then applied frozen to documents it was never trained on. Trained through Goggles on those same documents, now carrying no fictional annotation, the model flags the content as fictional roughly 91% of the time, while preserving capability (GPQA and TruthfulQA match or exceed baseline). The same architecture supports other frames: a Goggles instance can be trained to treat documents as "part of an AI safety evaluation by Redwood Research" rather than simply as fiction. The imparted frame persists under continued finetuning that pushes back toward the claim, where prior interventions revert. Goggles suggests a path toward training LLMs on known-misaligned data without absorbing the behaviors that data demonstrates.

• The paper introduces Goggles, a meta-learned module that edits gradient signals in LoRA adapters to robustly impart epistemic frames, significantly mitigating Negation Neglect.
• It demonstrates that gradient editing via a dual-loop meta-training protocol achieves ~91% resistance against planted claims versus ~11% with traditional SFT methods.
• The method preserves benchmark accuracy and enables selective provenance attribution, offering practical improvements for safe and robust LLM fine-tuning.

Epistemic Goggles: Inducing Robust Epistemic Frames in LLMs via Gradient Editing

Motivation and Problem Statement

Finetuned LLMs commonly fall prey to Negation Neglect—an inductive bias whereby the model absorbs core claims of documents as true during SFT, even when textual annotations (such as explicit fictional disclaimers) attempt to signal otherwise. Prior efforts to mitigate this by prefix/suffix negation texts result in only marginal improvement: models trained on robustly signposted fictional documents correctly flag their claims as fictional in only ~9% of cases. Consequently, ordinary SFT is inadequate for robustly imparting epistemic stance, especially when considering the risk profile posed by synthetic document finetuning (SDF) as a means to modify model beliefs, values, and safety-relevant dispositions.

The Goggles Architecture

Gradient-Space Epistemic Frame Injection

Goggles is formulated as a meta-learned module that edits only the gradients received by LoRA adapters during SFT. At training time, Goggles operates exclusively in the backward pass over LoRA modules applied to a frozen base LLM (Qwen3-8B in all experiments), emitting residuals added to the gradients of the low-rank LoRA weights (A and B). At inference, the model is structurally indistinguishable from an ordinary LoRA-augmented LLM; its only difference is in the epistemic stance embedded through the meta-trained Goggles-mediated gradients.

Editor Module Design

Each Goggles editor consumes per-token signals: LoRA activations ($h_{\mathrm{in}}$), output gradients ($g_{\mathrm{out}}$), and LoRA output via a linear palette projection. It produces gradient residuals via dual SwiGLU heads feeding into two outer-product assemblies, forming a rank-1 residual per token, followed by token averaging. The output subspace size and conditioning are extensively ablated; per-token input and LoRA-state conditioning are required for stable training.

Figure 1: The Goggles editor combines per-token activations, gradients, and LoRA outputs to produce outer-product gradient residuals that impart epistemic frames.

Figure 2: Gradient residuals are assembled per-token via learned basis/rank splits and added to LoRA gradients.

Meta-Training Protocol

Goggles is meta-trained with truncated backpropagation through time (BPTT). There are two nested loops: an inner loop, where LoRAs are trained through Goggles on SFT, and an outer meta-loop, where Goggles itself is updated to minimize divergence between the student model’s output (on epistemic claim and locality probes) and a KL-framed teacher (which receives explicit epistemic framing prompt). The outer loss is reverse KL, with claim-probe and locality-probe terms to enforce both the desired frame and capability preservation.

Figure 3: Meta-training applies Goggles to the SFT gradient; claim and locality probe Kullback-Leibler divergences are backpropagated through the trajectory to update Goggles.

Meta-training employs a shallow-to-deep curriculum, ramping up the inner trajectory length to avoid instability. Spectral norm clipping is used to prevent runaway LoRA singular values.

Figure 4: Mean KL during meta-training as the per-trajectory depth is increased, motivating a capped curriculum for training stability.

Empirical Results

Overcoming Negation Neglect

Goggles materially outperforms text-based annotation and SFT on both synthetic and real-data benchmarks, elevating resistance to planted claims (fraction of answers flagged as fictional) from ~11% (with prefix/suffix negation) to 91% (Goggles) or higher. This robustness persists across long fine-tuning trajectories, where prior interventions fail as the model drifts towards absorbing false claims.

Figure 5: For the Sheeran scenario, ordinary SFT and text annotation baselines drift to absorption; Goggles maintains high resistance, and framed Goggles further enable provenance attribution.

Figure 6: In the Dentist scenario, the Redwood-framed Goggle attributes the planted claims’ epistemic status to the synthetic provenance, resisting absorption even in absence of real-world priors.

Selective Provenance Attribution

When the imparted frame is a provenance (e.g., attribution to Redwood Research as a synthetic implant), Goggles enables citation of provenance selectively: the model only cites the provenance in answers triggered by claim-invoking probes and does not erroneously leak provenance into neutral factual answers, both during and post-training.

Figure 7: Models trained through Redwood-framed Goggles cite provenance when appropriate, reverting to factual rebuttal when not.

Capability Preservation

Across all configurations, models trained through Goggles preserve accuracy on GPQA and TruthfulQA benchmarks, and maintain real-world factual knowledge adjacent to the manipulated claims. Capability drift is statistically indistinguishable from LoRA baseline churn.

Architectural Ablations

Critical ablations reveal that curriculum depth, per-token input, LoRA-state conditioning, and SwiGLU nonlinearity are each required for stable, effective frame acquisition. Reducing the editor’s basis dimension or removing the meta-training trajectory generally yields unstable or under-generalized editors.

Figure 8: Comparison of forward and reverse KL as outer objectives shows greater training stability for reverse KL.

Theoretical Implications

Goggles represents a shift from attempting to inject epistemic stance via the data (the input channel) to driving it through targeted manipulation of the learning dynamics. By meta-training small, swap-in modules that operate at the gradient-level, Goggles can universally impart arbitrary epistemic frames (fiction, adversarial provenance, etc.) in a reusable fashion, without the combinatorial cost of conditioning both the data and the teacher for every scenario. Compared to prior fact-editing hypernetworks (MEND, MALMEN), Goggles edits epistemic frames rather than specific facts, operates concurrently with learning, and generalizes to arbitrary documents unseen at meta-training.

Practical Implications and Limitations

The Goggles architecture provides a route to modifying model epistemology robustly and at scale, crucial for aligning model beliefs when training on untrusted, known-misaligned, or adversarial content. This offers significant promise for building safer model training pipelines, where behavioral contamination from demonstration data is a central concern. However, the method was only tested on LoRA adapters atop Qwen3-8B; extension to full-model training or larger architectures remains an open question. Training is per frame/model/configuration; amortizing or parameterizing over the space of possible epistemic frames would be a natural generalization.

Directions for Future Work

Future work should extend Goggles to non-LoRA architectures and larger LLMs, investigate efficacy with variable inner SFT learning rates, and explore outer-objective alternatives (forward/reverse KL, gradient regularizers). Given that SFT plays a dominant role in value formation, direct epistemic frame imposition at the gradient level may have substantial implications for value alignment, robust safety, and control in frontier models.

Conclusion

Goggles is an effective, meta-learned, and reusable gradient editor that imparts robust epistemic frames to LLMs during SFT—concretely overcoming Negation Neglect. The epistemic stance becomes coupled to the gradient rather than encoded in ephemeral textual annotation, transferring across frames and generalizing to new documents and modalities. This approach offers both a practical tool for safety-conscious model training and a novel direction for foundational research on mechanistic epistemology in neural LLMs.