Self-Guided Fact Enhancement (SFE)

• Self-Guided Fact Enhancement (SFE) is a design pattern where models use internal signals such as self-generated facts, attention cues, and confidence estimates to improve factual accuracy and structured reasoning.
• It has diverse instantiations, including anomaly detection in vision-language models and factual alignment in large language models, emphasizing self-guidance over external knowledge.
• SFE methods operate during training or inference by integrating mechanisms like fact generation, self-evaluation, and self-correction to refine model outputs.

Searching arXiv for the cited papers to ground the article in the specified sources. Self-Guided Fact Enhancement (SFE) denotes a family of self-guided mechanisms in which a model uses its own internal knowledge, self-generated facts, attentional structure, confidence estimates, or intermediate-layer signals to improve factuality, grounding, or structured reasoning without making manual knowledge injection the primary mechanism. In recent work, SFE has been instantiated as a training and inference strategy for industrial anomaly reasoning in vision-LLMs, as self-evaluation and self-instruction for factual alignment and fact verification in LLMs, and as inference-time evidence elicitation, fact-level self-correction, and logits-evolution decoding for grounded generation (Zang et al., 10 Jul 2025, Zhang et al., 2024, Zhang et al., 2024).

1. Conceptual scope and defining properties

Across the cited literature, SFE is consistently characterized by self-guidance rather than externally curated supervision at the point where factual enhancement occurs. In "SAGE" (Zang et al., 10 Jul 2025), the self-guided signal is a combination of pretrained-VLM-generated fact sentences and self-computed patch discrepancies between a test image and a normal reference image. In "Self-Alignment for Factuality" (Zhang et al., 2024), the self-guided signal is the model’s own estimate of $p(\mathrm{True}\mid q,a)$ derived solely from internal knowledge. In "Mining the Explainability and Generalization" (Lu et al., 2024), the model self-generates both positive and negative explanations for claim-evidence pairs and then learns from those contrasts. In "SLED" (Zhang et al., 2024), earlier-layer logits provide latent knowledge that guides a single-step refinement of final-layer logits. In "SelfElicit" (Liu et al., 12 Feb 2025), deeper-layer attention identifies key evidence sentences that are then explicitly highlighted. In "Fact-Level Confidence Calibration and Self-Correction" (Yuan et al., 2024), high-confidence facts from a response are reused as internal knowledge to revise low-confidence facts.

Instantiation	Self-guided signal	Reported setting
SAGE	self-generated fact sentences and patch-level discrepancies	industrial anomaly reasoning
Self-Alignment for Factuality	Self-Eval and SK-Tuning	TruthfulQA and BioGEN
Mining Explainability and Generalization	self-instruction explanations and improved DPO	FEVEROUS and HOVER
SLED	earlier-layer versus final-layer logits	TruthfulQA, FACTOR, StrategyQA, GSM8K
SelfElicit	deeper-layer attention highlighting	HotpotQA, NewsQA, TriviaQA, NQ
ConFix	fact-level confidence and self-correction	LongFact and ASQA

A central distinction in this literature is between self-guidance and retrieval-augmented or manually curated pipelines. SAGE explicitly avoids manual knowledge injection and a classic retrieval-augmented pipeline; Self-Alignment for Factuality avoids external retrieval during training; SelfElicit does not retrieve new documents and is training-free; SLED improves truthfulness without relying on external knowledge bases or requiring further fine-tuning; ConFix uses no external knowledge sources such as retrieval systems (Zang et al., 10 Jul 2025, Zhang et al., 2024, Liu et al., 12 Feb 2025, Yuan et al., 2024). This suggests that SFE is best understood as a design pattern centered on internal signals rather than as a single algorithm.

2. Comparison-based SFE in multimodal anomaly reasoning

The most explicit and formalized use of the term appears in "SAGE: A Visual LLM for Anomaly Detection via Fact Enhancement and Entropy-aware Alignment" (Zang et al., 10 Jul 2025). There, Self-Guided Fact Enhancement is defined as a training and inference strategy that augments a Vision-LLM’s visual reasoning with domain-specific facts and explicit test–reference comparison signals, so the model can identify, localize, and explain industrial anomalies with interpretable logic. The stated motivation is that industrial anomaly detection requires answers to “what,” “where,” and “why” anomalies occur, whereas general VLMs are trained on broad web data and lack the fine-grained, domain-specific notion of “normality” and “abnormality” needed in practice.

SFE in SAGE operates on a test–reference image pair $\langle I_t, I_r\rangle$ and a question $Q$. The vision encoder $F_v$ extracts patch embeddings $v_t, v_r \in \mathbb{R}^{p^2 \times C}$, and the text encoder $F_t$ encodes the question into $q \in \mathbb{R}^{L_q \times C}$. The paper augments $Q$ with the sentence “Above is the feature related to the anomaly facts.” A pretrained VLM then generates $L_F$ fact sentences $s=\{s_1,\dots,s_{L_F}\}$ per test image using predefined prompts such as “What kind of anomalies does this type of object tend to exhibit?” Each sentence is encoded by $\langle I_t, I_r\rangle$0 to $\langle I_t, I_r\rangle$1, and a global pooling yields $\langle I_t, I_r\rangle$2.

The visual side of SFE is comparison-based. Candidate anomalous regions are selected by computing cosine similarity between each test patch and all reference patches, then taking the $\langle I_t, I_r\rangle$3 test patches with minimal similarity. The paper gives the patch selector as

$\langle I_t, I_r\rangle$4

which yields candidate anomalous patches $\langle I_t, I_r\rangle$5. SFE then uses a fusion attention layer $\langle I_t, I_r\rangle$6 in which $\langle I_t, I_r\rangle$7 acts as query, $\langle I_t, I_r\rangle$8 as key, and $\langle I_t, I_r\rangle$9 as value, producing a comparison-enhanced prompt $Q$0. Multi-head attention is used, and the final LLM input concatenates $Q$1, $Q$2, and $Q$3 before autoregressive generation.

Architecturally, SAGE uses InternVL2 as the backbone VLM, InternViT‑6B‑448px‑V1‑5 as the vision encoder, and Nous‑Hermes‑2‑Yi‑34B as the LLM. The data flow is explicit: input pair and question; encode $Q$4, $Q$5, and $Q$6; generate and encode facts offline; select discrepancy patches; fuse facts and visual patches via cross-attention; concatenate $Q$7; decode an answer containing anomaly decision, localization, and reasoning. The self-guided label in SFE is therefore twofold: fact generation relies on the pretrained VLM’s internal knowledge, and the visual side uses self-computed patch discrepancies without external supervision.

3. Self-evaluation and self-instruction as alignment signals

A second major SFE line treats factual enhancement as preference construction from a model’s own outputs. In "Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation" (Zhang et al., 2024), SFE is realized through Self-Eval and Self-Knowledge Tuning (SK-Tuning). Self-Eval uses a True/False A/B classification prompt to estimate whether a proposed answer is true based only on internal knowledge, with

$Q$8

For long-form outputs, responses are decomposed into atomic claims and scored by the average claim-level confidence,

$Q$9

Preferred and dispreferred responses are then constructed by ranking candidates by these scores and applying Direct Preference Optimization.

SK-Tuning in that work improves the reliability of self-evaluation by training on a large True/False dataset built from 49,862 prompts from Wikipedia and 32,500 prompts from 17 MCQA tasks in BIG-bench. The reported training corpus contains 2,470,860 True/False examples generated by 10-shot prompting with 30 samples per prompt. The stated purpose is to reduce overconfidence and improve calibration; the paper reports that Self-Eval-SKT improves both selection accuracy and AUROC over naive Self-Eval-P(True) across TruthfulQA, CommonSenseQA, OpenBookQA, MedQA, and MMLU.

A related but distinct formulation appears in "Mining the Explainability and Generalization: Fact Verification Based on Self-Instruction" (Lu et al., 2024). There, SFE is instantiated through self-instruction-based data augmentation and improved DPO. The model generates explanations for both aligned and counterfactual labels under four settings built from the prompt function $F_v$0, then samples training pairs according to a customized difficulty score derived from the win-rate $F_v$1 of unguided label-plus-explanation generation. The paper’s sampling rule is

$F_v$2

Improved DPO adds Lagrangian constraints designed to ensure that chosen responses increase relative to the reference policy while rejected responses decrease.

The important conceptual commonality between these two language-only strands is that the model itself produces the signal that later drives alignment. In Self-Alignment for Factuality, the signal is self-evaluated truthfulness. In the self-instruction fact-verification framework, it is the model’s own generated positive and negative explanations, optionally grounded by counterfactual tips. In both cases, the enhancement of factual behavior is not delegated to external retrieval.

4. Inference-time SFE: decoding, highlighting, and self-correction

A third SFE family operates purely at inference time. "SLED: Self Logits Evolution Decoding for Improving Factuality in LLMs" (Zhang et al., 2024) treats decoding as a self-guided refinement problem. Earlier-layer logits are used to estimate a latent distribution $F_v$3, which then guides a single-step update of the final-layer logits:

$F_v$4

The resulting token update is

$F_v$5

SLED is training-free, requires no external knowledge bases, and can be combined with Contrastive Decoding, Induce-then-Contrast, Activation Decoding, and Inference-Time Intervention.

"SelfElicit: Your LLM Secretly Knows Where is the Relevant Evidence" (Liu et al., 12 Feb 2025) instantiates SFE as explicit evidence highlighting. At the first decoding step, the method averages attention across heads at layer $F_v$6,

$F_v$7

aggregates token attention into sentence scores, and then averages over the evidence-reading layer set $F_v$8, empirically chosen as the last 50% of layers:

$F_v$9

Sentences satisfying

$v_t, v_r \in \mathbb{R}^{p^2 \times C}$0

are wrapped with literal markers <start_important> and <end_important>, after which the model performs standard answer generation. The paper emphasizes that sentence-level elicitation outperforms token-level elicitation and that the method is training-free and non-iterative.

"Fact-Level Confidence Calibration and Self-Correction" (Yuan et al., 2024) applies SFE to long-form response editing. A response is decomposed into atomic facts, per-fact confidence is estimated, and facts are split into

$v_t, v_r \in \mathbb{R}^{p^2 \times C}$1

with $v_t, v_r \in \mathbb{R}^{p^2 \times C}$2 defined as the mean confidence across facts in the response. High-confidence facts serve as internal knowledge for correcting low-confidence facts through targeted factor extraction and revision, with acceptance conditioned on an increase in estimated confidence. The paper studies verbalization, logit-based isTrueLogit, and consistency as confidence sources and evaluates calibration via Fact-Level ECE rather than a single scalar response confidence.

These inference-time variants share a common structural idea: the model’s own intermediate states are treated as actionable evidence. In SLED the evidence is layerwise logit evolution; in SelfElicit it is deep-layer attention; in ConFix it is fact-level confidence structure within the generated response.

5. Empirical performance and reported effects

The reported performance of SFE is heterogeneous because the underlying tasks are heterogeneous. In industrial anomaly reasoning, SAGE reports a MANTA zero-shot average accuracy of 69.6% versus GPT‑4o at 62.1%, and a MANTA one-shot average accuracy of 73.5% versus InternVL2 (sft) at 71.3%. On MPDD QA, SAGE reports average accuracy 73.8% in zero-shot and 74.1% in one-shot. On MANTA one-shot, group-level Acc‑G is reported as nearly 24%, with zero-shot at approximately 17%. In the SAGE ablation, the transition from direct fact concatenation to fusion attention enhancement raises zero-shot MANTA accuracy from 57.37 to 62.23, and the full model reaches 69.64 with SBERT 0.70, Acc‑G 17.32, and Score 4.05 (Zang et al., 10 Jul 2025).

In factual language generation, Self-Alignment for Factuality reports substantial gains for both LLaMA-7B and LLaMA2-7B. For LLaMA-7B on TruthfulQA MCQA, accuracy improves from 25.60 to 45.48; on short-form TruthfulQA, True*Info rises from 26.90 to 45.75; on BioGEN, FActScore improves from 30.72 to 38.28 and incorrect facts per response are reduced from 16.92 to 13.49. For LLaMA2-7B, MCQA accuracy rises from 28.90 to 44.10, True*Info from 39.04 to 53.42, and BioGEN FActScore from 40.54 to 46.50 (Zhang et al., 2024).

In fact verification, the self-instruction plus improved DPO framework reports FEVEROUS overall ACC 94.43 and F1 94.42, and HOVER overall ACC 81.30 and F1 81.26. The cross-dataset setting trained on HOVER and tested on FEVEROUS yields ACC 89.00 and F1 88.83. Human evaluation reports improved ability to identify the main contradictions relative to the pre-finetuned Llama2 baseline: +14.28% on 4-hop, +2.50% on 3-hop, and +11.63% on 2-hop (Lu et al., 2024).

In decoding-time factuality, SLED reports that it consistently improves factual accuracy by up to 20% compared to existing decoding methods while maintaining natural language fluency and negligible latency overhead. A representative open-ended result is LLaMA 2 7B Base on TruthfulQA, where %Truth×Info rises from 23.99 under greedy decoding and 32.31 under DoLa to 52.87 under SLED, while %Reject drops to 0.12. On FACTOR, LLaMA 2 13B Base improves from 63.69 under greedy decoding to 70.91 under SLED (Zhang et al., 2024).

In evidence-based QA, SelfElicit reports consistent gains across six instruction-tuned LMs. For Llama-3.1-8B, HotpotQA improves from EM 58.9 and F1 57.7 to EM 68.5 and F1 69.5; TriviaQA improves from EM 72.8 and F1 66.1 to EM 79.4 and F1 72.7; Natural Questions improves from EM 59.7 and F1 61.6 to EM 64.0 and F1 67.8. The method increases average inference time modestly, with a reported +17.8% time and −2.3% tokens for Llama-3.1-8B (Liu et al., 12 Feb 2025).

For fact-level self-correction, ConFix shows that calibration quality strongly conditions editing quality. GPT-4 judged revised facts from LLaMA-2-13b as 53.35 improved, 19.59 same, and 27.07 regressed, whereas LLaMA-2-7b showed 6.76 improved, 12.56 same, and 80.68 regressed. The paper explicitly links this disparity to poorer calibration in weaker models, indicating that self-guided correction is only as reliable as the confidence signal on which it is built (Yuan et al., 2024).

6. Misconceptions, limitations, and open directions

A common misconception is that SFE is synonymous with retrieval augmentation. The cited works do not support that equation. SAGE contrasts self-guided fact generation with curating knowledge bases or building retrieval stacks. Self-Alignment for Factuality relies on internal knowledge rather than in-domain human factual labels or retrieval during training. SelfElicit operates on a provided context rather than retrieving new documents. ConFix explicitly uses high-confidence facts within a response as internal knowledge rather than external sources (Zang et al., 10 Jul 2025, Zhang et al., 2024, Liu et al., 12 Feb 2025, Yuan et al., 2024).

Another misconception is that SFE is inherently training-free. The record is mixed. SLED and SelfElicit are inference-time methods; SAGE is a two-stage training regime involving Stage 1 SFE and Stage 2 E-DPO; Self-Alignment for Factuality uses SK-Tuning and DPO; the self-instruction fact-verification framework uses LoRA fine-tuning with improved DPO. This suggests that self-guidance describes the origin of the enhancement signal, not the training status of the method.

The limitations are correspondingly diverse. SAGE notes reliance on the quality of generated facts and acknowledges that, without external supervision, facts may include inaccuracies or biases that could misguide attention or explanations. Self-Alignment for Factuality identifies reinforcement of incorrect internal knowledge, label noise in self-annotations, domain coverage gaps, and non-trivial computational cost. SelfElicit notes that attention can misfire under spurious correlations, adversarial distractors, domain shifts, or long-context dilution. ConFix depends on calibration quality and can propagate errors when incorrect high-confidence facts are used as references. SLED cannot recover facts that are absent from the model’s latent knowledge and can be harmed by overly large evolution rates or over-aggressive pruning (Zang et al., 10 Jul 2025, Zhang et al., 2024, Liu et al., 12 Feb 2025, Yuan et al., 2024, Zhang et al., 2024).

Open directions are stated explicitly in several of these papers. For industrial anomaly reasoning, future work could combine self-guided facts with selective retrieval from vetted industrial knowledge bases, add uncertainty-aware or calibration mechanisms for fact fusion, explore adaptive $v_t, v_r \in \mathbb{R}^{p^2 \times C}$3 selection or spatial priors, and expand to multi-view, 3D, and temporal anomalies. For self-instruction-based fact verification, suggested extensions include stronger difficulty measures beyond $v_t, v_r \in \mathbb{R}^{p^2 \times C}$4, explicit contrastive objectives, and integrating retrieval for open-world settings. For ConFix, the framework motivates stronger fact-level calibration techniques and contradiction checks. A plausible implication is that the next phase of SFE research will focus less on whether self-guidance is useful and more on when internal signals are reliable enough to be used as substitutes for external evidence.