Fact2Fiction Attack Paradigm

• The Fact2Fiction Attack Paradigm is an adversarial framework that transforms verifiable facts into deceptive fiction through techniques such as evidence obfuscation, prompt engineering, and strategic manipulation.
• It employs formal logic, taxonomy analysis, and linguistic perturbations to expose vulnerabilities in automated fact-checking and computational reasoning systems.
• Empirical studies reveal that these attacks can reduce system accuracy by up to 70 percentage points, highlighting the urgent need for robust, multi-layered defense mechanisms.

The Fact2Fiction Attack Paradigm refers to a broad class of adversarial strategies and attack surfaces in computational reasoning, knowledge representation, and automated verification whereby facts—that is, information presumed true or relevant for rational inference—are transformed, obfuscated, or manipulated into “fiction” capable of misleading, confusing, or subverting automated systems and human decision-makers. This paradigm applies across domains including automated fact-checking, argumentation theory, cyber deception, multi-agent verification, and AI security. It encompasses a variety of mechanistic routes from overt fact fabrication through synthetic content, nuanced linguistic prompt engineering, and strategic evidence manipulation, to subtle half-truths and data withholding, as well as structural attacks on logical and epistemic frameworks.

1. Formal Definitions and Logical Underpinnings

The Fact2Fiction paradigm is formally instantiated in logical frameworks for argumentation and decision-making. In abstract argumentation networks $(S, R)$, attack relations can be object-level encoded within classical logic through strong negation: specifically, "$x$ attacks $y$" is internalized as $x \to N y$, where $N$ is a built-in strong negation operator (with axioms such as $Nq \to \neg q$) (Gabbay et al., 2015). This representation yields a direct correspondence between classical models of the constructed theory and complete extensions of the network. The approach extends naturally to joint attacks, support, and higher-level constructs, and facilitates the logical analysis of attack dynamics without relegating conflict to the meta-level.

Alternatively, Intuitionistic logic provides a constructive encoding of attacks: $xRy$ is modeled as $x \to \neg y$, where implication and negation are intuitionistic. This translation supports the natural emergence of a three-valued semantics (“in”, “out”, “undecided”) and enables higher-level reasoning where attacks (and even statements about attacks) are themselves subject to attack, forming self-referential and meta-level structures—see, e.g., $uRv \to \neg (xRy)$ (Gabbay et al., 2015). These encodings provide a semantic foundation for analyzing how fact can be transformed or challenged into different epistemic statuses.

2. Taxonomy of Fact2Fiction Attacks

A comprehensive taxonomy situates Fact2Fiction attacks along several orthogonal axes (Abdelnabi et al., 2022):

• Target of Manipulation:
  • Camouflaging: Perturbs or conceals true evidence so it cannot be effectively used by the verification system, often resulting in “Not Enough Info” rather than explicit misclassification.
  • Planting: Injects new, misleading, or supportive evidence to actively invert or mislead verification outcomes.
• Manipulation Constraints:
  • Replacement vs. Addition: Can the adversary overwrite existing evidence, or only add new entries?
  • Contextual Integrity: Does the manipulation preserve document coherence and surface plausibility, or only minimal aspects?
• Adversary Capabilities and Knowledge:
  • White-box vs. Black-box: Level of access to models (retrieval, verification, or data).
  • Proxy Modeling: Ability to query or mimic the victim’s retrieval/verification pipelines using surrogate models.
• Implementation Modalities:
  • Lexical Variation: Synonym swaps and word embeddng substitutions disrupt matching.
  • Contextualized Replacement: Masked LLMing (e.g., via BERT) for nuanced, contextually correct substitutions.
  • Imperceptible Perturbations: Insertion of invisible Unicode, homoglyphs, or control characters to evade token-based detection.
  • Claim-Aligned Generation: Encoder-decoder models (e.g., T5, GPT-2, Grover) generate “supporting” content attuned to the adversary’s aims.

The taxonomy enables a systematic exploration of the threat landscape, guiding evaluation of both localized and distributed evidence poisoning strategies.

3. Mechanisms in Automated Fact-Checking and Verification

Fact2Fiction attacks manifest with distinct effects in automated fact-checking pipelines—especially those relying on evidence retrieval and natural language inference (NLI) over large text corpora (Du et al., 2022, Puzis et al., 2019, He et al., 8 Aug 2025):

• Repository Poisoning: Adversarial Addition (ADVADD) injects synthetic documents (frequently generated by neural models like GROVER) into evidence collections. Adversarial Modification (ADVMOD) alters existing entries, e.g., through paraphrasing, targeted replacements, or appending adversarially generated sentences.
• Decomposition Exploitation: Agentic systems decompose claims into sub-claims, then verify each independently (He et al., 8 Aug 2025). The Fact2Fiction framework mirrors this process, using multi-agent LLMs (Planner and Executor) to target sub-claim justifications, allocate poisoning budgets using importance-weighted strategies ($m_k = \lceil m \cdot (w_k/\sum_s w_s)\rceil$), and craft highly retrievable evidence.
• Linguistic Prompt Attacks: The Illusionist’s Prompt introduces linguistic nuances (syntactic complexity, semantic entropy, emoji, etc.) into adversarial queries (Wang et al., 1 Apr 2025). This increases the likelihood of internal LLM hallucination and factual error, even against state-of-the-art fact-enhancing defenses (TruthX, ICD, Multi-agent Debate, HonestLLM, FRESHPROMPT), as measured by metrics like the Flesch Readability Score and Sentence-BERT semantic similarity.

Attack Modality	Effect on System	Example Techniques
Camouflaging	Fails to retrieve/support true evidence	Lexical/semantic masking
Planting	Falsely supports/refutes claims	Claim-aligned generation
Prompt-based	Internal hallucination, model confusion	Syntactic/semantic mutation

4. Theoretical Analysis and Complexity Considerations

Fact2Fiction encompasses not only overt fabrication of content but also covert informational strategies modeled by decision-theoretic frameworks (Estornell et al., 2019). In Bayesian prediction and decision scenarios, an adversary can systematically “mask” evidence—creating half-truths by hiding bits of information—leading to dramatically shifted posteriors without altering the data’s surface form. The untargeted case maximizes the statistical distance $D(\mathbf{X}^1, \mathbf{X}^1_\eta)$ under a masking budget; the targeted case minimizes $D(\mathbf{X}^1_\alpha, \mathbf{X}^1_\eta)$ for a desired fictional target.

This optimization is NP-hard even to approximate (unless P=NP), although tractable cases exist for additive or linear dynamic Bayes networks. These results show that attempts to defend against information masking (i.e., strategic half-truths) may be computationally intractable in general, strengthening the adversary’s position in constructing Fact2Fiction attacks through information omission.

5. Empirical Impacts and Security Implications

Fact2Fiction attacks result in significant degradation of automated decision-making and fact-checking systems across domains (Du et al., 2022, Abdelnabi et al., 2022, He et al., 8 Aug 2025, Schlichtkrull, 13 Oct 2025). Observed impacts include:

• Significant Accuracy Losses: Both synthetic document poisoning and linguistic prompt attacks can reduce verification accuracy by 30–70 percentage points, with even state-of-the-art models and black-box APIs (GPT-4o, Gemini-2.0) highly vulnerable to subtle adversarial strategies (Wang et al., 1 Apr 2025).
• Robustness to Countermeasures: Many defense strategies—adversarial training, semantic filtering, multi-agent debate—provide limited improvement under sophisticated, targeted attacks. Attacks remain robust to post-hoc claim edits, model architecture changes, and often require only black-box/limited access to the victim system.
• Amplification of Misinformation: By compromising either the evidence retrieval or sub-claim verification step, these attacks propagate and legitimize falsehoods, amplifying their effect due to automated system scale and opacity.

AI agents are further vulnerable to “attacks by content,” in which adversarial documents bias, mislead, or omit critical facts without any need of prompt injection or direct instruction, revealing a fundamental AI security issue (Schlichtkrull, 13 Oct 2025).

6. Countermeasures and Research Directions

Suggested directions for defense and research across the Fact2Fiction landscape include:

• Integrating Automated Fact-Checking as AI Cognitive Self-Defense: Fact-checking pipelines incorporating claim prioritization, diversified evidence retrieval, source credibility evaluation, and explicit communication of rationale can help agents filter and mitigate adversarial content (Schlichtkrull, 13 Oct 2025). This requires agent “media literacy” analogous to human critical reading.
• Multi-source and Circular Verification: Leveraging multiple independent evidence sources, triangulating metadata (e.g., provenance, reputation), and adopting “circular verification” strategies that compare and cluster evidence by stance can improve resilience (Abdelnabi et al., 2022).
• Detection of Evidence Manipulation: Developing detectors for imperceptible perturbations (Unicode, homoglyphs), clustering analyses for identifying suspiciously similar adversarial evidences, and outlier detection on retrieval statistics.
• Restricting Internal Transparency: Limiting the public exposure of intermediate justifications, decomposition structures, and retrieval queries may prevent adversaries from targeting system vulnerabilities (He et al., 8 Aug 2025).
• Model and Training Enhancements: Adversarially augmenting datasets with manipulated and adversarially generated content broadens model exposure and improves robustness (Thorne et al., 2019, Du et al., 2022).
• Adaptive and Linguistically-Aware Defenses: Incorporating style and structure monitoring (e.g., readability/formality/concreteness measures), dynamic uncertainty estimation, and prompt analysis to detect anomalous or entropy-raising queries (Wang et al., 1 Apr 2025).

Ongoing research emphasizes the need for fact-checking systems, LLMs, and autonomous agents to critically evaluate both the content and provenance of information, rather than relying solely on internal consistency or formal prompt structure.

7. Broader Significance and Context

The Fact2Fiction Attack Paradigm stands at the interface of logic, linguistics, adversarial learning, and AI security. It exposes that factual assurance in computational systems is fundamentally brittle in the face of adversarial evidence manipulation, strategic informational masking, and nuanced linguistic perturbations. The breadth of documented attack mechanisms—spanning modal/intuitionistic treatments of attack, large-scale synthetic document generation, fine-grained prompt mutation, content-driven poisoning of agentic pipelines, and half-truths masking—underscores that adversarial agents have a rich arsenal across the technical, epistemic, and operational landscapes. The paradigm compels the integration of robust, context-aware, and multi-layered defenses in future AI reasoning systems and mandates continued cross-disciplinary research into both offensive and defensive methodologies for factual assurance.