FinVet: Financial Misinformation Detection

• FinVet is a multi-agent framework for financial misinformation detection that integrates dual RAG pipelines, external fact-checking, and a confidence-weighted voting mechanism.
• It employs a robust architecture with efficient vector indexing, adaptive evidence retrieval, and explicit uncertainty reporting to ensure transparent claim verification in financial news.
• Evaluations on the FinFact dataset show a 10.4% F1 improvement over single pipelines, demonstrating its effectiveness in financial risk assessment.

FinVet is a multi-agent framework for financial misinformation detection that combines dual retrieval-augmented generation (RAG) pipelines with external fact-checking and a confidence-weighted voting mechanism. It addresses the lack of transparency and source attribution prevalent in prior systems, enabling both evidence-backed verdicts and explicit uncertainty reporting when credible information is insufficient. FinVet’s architecture and strategy are designed for the volatile, high-impact context of financial news, where rapid, transparent automated assessment of claims is operationally critical (Araya et al., 13 Oct 2025).

1. System Architecture

FinVet’s architecture comprises four principal components: (A) data processing and vector store, (B) multi-pipeline claim verification, (C) result normalization, and (D) integrated verdict reporting.

A. Data Processing & Vector Store:

Financial documents are segmented into claim–evidence snippets, each embedded using the all-MiniLM-L6-v2 model and indexed with FAISS IVFFlat for efficient similarity retrieval. Metadata (e.g., source URL, document ID, and publication date) is preserved for downstream attribution.

B. Claim Verification Pipelines:

• RAG Pipeline 1: Utilizes LLaMA-3.3-70B-Instruct for retrieval and generation.
• RAG Pipeline 2: Employs Mixtral-8x7B-Instruct under identical retrieval/generation settings.
• Fact-Check Pipeline:
  • Stage 1: Queries the Google Fact Check Tools API. If a claim match is found, returns the published verdict, assigns maximum confidence (1.0), and cites the external URL.
  • Stage 2 (fallback): If no external match, invokes LLaMA-3.3-70B in a role-based LLM analysis; outputs a label, evidence, "Parametric Knowledge" as the source, and the model’s confidence estimate.

C. Results Normalization:

All labels are mapped to lower case and confidence values are standardized on $[0, 1]$.

D. Verdict Integration & Reporting:

Pipeline outputs are fused via a confidence-weighted voting mechanism. If the fact-check pipeline provides an external verdict, this automatically takes precedence. Otherwise, the output with maximal confidence is selected. If all confidences are zero, a “Not Enough Information” (NEI) disposition is returned. Final reports consist of a structured bundle: verdict label, evidence snippet, explicit source, and confidence score (Araya et al., 13 Oct 2025).

2. Adaptive Three-Tier Processing Strategy

FinVet’s RAG pipelines employ an adaptive strategy predicated on retrieval confidence, partitioned by fixed cosine similarity thresholds $\theta_{high} = 0.6$ and $\theta_{med} = 0.4$. Let $\{s_i\}_{i=1}^k$ denote claim-to-snippet similarities, and $s_{max} = \max_i s_i$.

• Case 1 (High Confidence):

$s_{max} \geq \theta_{high}$ Verdict, evidence, and source are taken directly from snippet metadata. Confidence $\rho = s_{max}$.

• Case 2 (Moderate Confidence):

$\theta_{med} \leq s_{max} < \theta_{high}$ Top-k contexts are concatenated and provided to the LLM, which returns a verdict and self-reported $\rho_{model}$. If the retrieved context is validated, attribution is retained; otherwise, source is assigned as “Parametric Knowledge.” Final confidence is $\rho = \frac{1}{2}(s_{max} + \rho_{model})$.

• Case 3 (Low Confidence):

$\theta_{high} = 0.6$0 The LLM is prompted in a role-based manner without external snippets, producing verdict, evidence, and confidence $\theta_{high} = 0.6$1; source is “Parametric Knowledge.” Confidence $\theta_{high} = 0.6$2.

This tiered method enables graded response based on available evidence, prioritizing direct retrieval but deferring to LLM reasoning or external fact-checking as warranted.

3. Confidence-Weighted Verdict Integration

Each pipeline $\theta_{high} = 0.6$3 yields a tuple $\theta_{high} = 0.6$4 with standardized confidences.

Integration rules:

1. If the Fact-Check API yields a result, its verdict is adopted with $\theta_{high} = 0.6$5.
2. Otherwise, compute $\theta_{high} = 0.6$6 and select $\theta_{high} = 0.6$7 as the outcome.
3. If all $\theta_{high} = 0.6$8, then verdict is NEI and $\theta_{high} = 0.6$9.

Formally, setting aside the auto-priority for external fact-checks:

$\theta_{med} = 0.4$0

Any zero-confidence tie or universal zero triggers the NEI outcome. This ensemble strategy fuses confidence-weighted evidence from complementary verification paths for robust and interpretable outputs.

4. Evidence Attribution Protocol

FinVet implements explicit evidence and source tracking at every decision point.

• Direct Retrieval (Case 1):

Evidence and source are preserved verbatim from snippet metadata (URL, doc ID, publication date).

• Hybrid Reasoning (Case 2):

If LLM validation is positive, snippet provenance is maintained; otherwise, source is tagged as “Parametric Knowledge” to denote parametric model inference.

• Model-Only Reasoning (Case 3):

Source is always designated as “Parametric Knowledge.”

• External Fact-Check:

Source is the URL of the verified external page, with maximal confidence.

The final report transparently bundles the evidence snippet, explicit source or URL, and the confidence score, permitting traceable post-hoc review and supporting auditability.

5. Performance Evaluation on Financial News

FinVet was evaluated on the FinFact dataset (Rangapur et al. 2024), using an 85%/15% split for vector store construction and testing, respectively. Embedding and retrieval utilized all-MiniLM-L6-v2 and FAISS IVFFlat, with RAG models LLaMA-3.3-70B and Mixtral-8x7B. The fact-check pipeline integrated the external API with LLM fallback. Experimental metrics included Accuracy, Precision (P), Recall (R), and F1 score using

$\theta_{med} = 0.4$1

Summary of Results

Method	Acc.	P	R	F1
FinVet (full system)	0.84	0.86	0.84	0.85
Fact-Check Pipeline + LLM (best single pipeline)	–	–	–	0.77
Standalone RAG (LLaMA-3.3-70B)	–	–	–	0.62
Zero-shot GPT-3.5	–	–	–	0.51
Chain-of-Thought GPT-3.5	–	–	–	0.45

• Relative Improvements:
  • +10.4% F1 over the best individual pipeline (0.85 vs. 0.77)
  • +37% F1 over standalone RAG (0.85 vs. 0.62)

Ablation studies indicate that each constituent—dual RAGs, external fact-checking, and confidence integration—contributes additively to the observed performance (Araya et al., 13 Oct 2025). This suggests the robustness and compositional effectiveness of the modular ensemble in the financial misinformation domain.

6. Distinctive Features and Context

FinVet introduces several distinguishing innovations compared to previous RAG or LLM-centric models for financial misinformation:

• Collaborative Multi-Agent Integration: Two independent RAG models and an external fact-check pipeline operate concurrently, with ensemble verdicting.
• Dynamic, Evidence-Driven Processing: Adaptive switching among evidence extraction, hybrid LLM reasoning, and parametric-only responses based on quantitative retrieval confidence.
• Structured Uncertainty Reporting: Explicit NEI (Not Enough Information) outcomes with transparent rationales when evidence is insufficient.
• Comprehensive Attribution: All outputs are paired with concrete evidence snippets, sources, and confidence scores for auditing and trust calibration.

The FinVet methodology directly addresses operational needs in financial risk management by emphasizing transparent, explainable automation and robust ensemble-based validation, as demonstrated on FinFact (Araya et al., 13 Oct 2025).