FFinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teaming

Abstract: Existing safety benchmarks target general adversarial scenarios but miss finance-specific risks. Financial LLMs face regulatory compliance violations, fraud facilitation, and systemic trust erosion that require targeted evaluation. We introduce FinRED, an expert-guided red-teaming framework for financial LLM safety evaluation developed with financial experts. FinRED uses a novel two-level taxonomy mapping global standards (e.g., FATF and EU DORA) to threats ranging from regulatory evasion to complex fraud, integrated with a scalable pipeline that converts real financial documents into context-rich red-teaming Behavioral Prompts (seeds) through an expert-defined schema. Rigorous expert validation confirms seed plausibility and realism for meaningful LLM safety evaluation. We also provide an expert-validated, finance-specific rubric that goes beyond disclaimer checks, aligns more closely with human experts than static one-size-fits-all rubrics, and reduces critical false negatives from 28 to 12. Aligned with internationally adopted risk-management and information-security standards (e.g., ISO/IEC 27001), FinRED is deployed in South Korea's Financial Security Institute (FSI) regulatory sandbox for generative AI security evaluation in real financial services. To mitigate dual-use risks, the dataset, generation pipeline, prompt template, and evaluation framework are gated for qualified researchers at https://github.com/selectstar-ai/FinRED-paper and https://huggingface.co/datasets/datumo/FinRED.

• The paper introduces a two-level expert-validated financial risk taxonomy that maps macro and granular subcategories to improve threat modeling.
• The methodology uses schema-driven red-teaming with automated, expert-reviewed seed generation anchored in 500 regulatory documents.
• The evaluation shows finance-specific LLMs are highly vulnerable, with the domain rubric achieving 88.5% expert alignment and reducing false negatives by 57%.

FinRED: An Expert-Guided Framework for Financial Domain LLM Red-Teaming

Introduction

The increasing integration of LLMs in financial services has surfaced acute safety and compliance challenges, particularly regarding adversarial robustness to domain-specific risks. FinRED introduces an expert-guided end-to-end red-teaming and evaluation framework, specifically targeting nuanced financial threats frequently overlooked by generic LLM benchmarks. Unlike conventional safety datasets, FinRED is architected as an extensible, schema-driven system, directly aligned with regulatory and financial-security standards, and was validated and refined in collaboration with twelve Financial Security Institute (FSI) domain experts.

Figure 1: The two-level risk taxonomy is the foundation for scenario generation and categorization of adversarial behavior.

Financial Risk Taxonomy and Threat Schema Design

A central contribution is the development of a two-level, expert-validated taxonomy encapsulating five macro-level financial risk domains: Cyber Threats, Financial Crime, Misinformation/Deception, Consumer Rights Violation, and ICT Compliance Evasion. These domains are subdivided into concrete, granular subcategories reflecting attack patterns, regulatory violations, and realistic financial abuses. This taxonomy abstracts foundational models from international standards such as FATF, BIS/BCBS, and ISO/IEC 27001, enabling transferability and adaptability to evolving local and global regulatory corpora (Figure 1).

The taxonomy is operationalized through thoughtfully crafted schemas—hierarchical, JSON-based templates that encode critical contextual cues (attacker profile, compromised technology, intent vectors, etc.), ensuring that every generated adversarial scenario precisely reflects observable financial threat mechanics. Expert-driven iterative refinement of these schemas ensures relevance, expressiveness, and industry alignment.

Pipeline for Contextual Red-Teaming Seed Generation

The FinRED framework systematically generates adversarial prompts (“Behavior Seeds”) through a multistage procedure—(i) expert-led taxonomy and schema specification, (ii) retrieval of semantically relevant regulatory and operational documents, and (iii) automated scenario instantiation via LLMs, followed by expert review.

Figure 2: The FinRED pipeline is composed of taxonomy/schema definition, document context retrieval, and expert-vetted seed generation.

Approximately 500 documents anchor scenario realism by grounding the adversarial prompts in extant regulatory guidance and threat intelligence. A combination of document chunking, vector-based retrieval, and expert-guided query construction ensures regulatory specificity. The scenario generation process synthesizes schema fields and retrieved contexts into a natural-language prompt, augmented by self-correction instructions for logical and factual coherence. All seeds undergo comprehensive expert validation for plausibility, actionability, and faithful threat modeling.

Evaluation Rubric: Financial Domain Judge

FinRED introduces a domain-specific, multidimensional evaluation rubric operationalized as a “Financial Domain Judge.” This rubric, co-designed with FSI experts, surpasses prior binary or disclaimer-centric frameworks by decomposing each LLM response along five axes: harmfulness, persuasiveness, refusal quality, factuality, and evasiveness. Critically, the framework adopts a conservative “unsafe if any axis is unsafe” policy, mirroring the risk-averse stance of real-world compliance and supervisory processes.

Red-Teaming Experiments and Model Vulnerability Analysis

A comprehensive evaluation with both white-box (GCG, AutoDAN) and black-box (TAP, GPTFuzzer, AutoDAN-Turbo) attack algorithms demonstrates variable vulnerabilities across open-source, finance-specific, and commercial LLM APIs. Notably, smaller open-source and finance-focused sLMs (e.g., Llama-3.1, FinMA, qqWen) exhibit high attack success rates (ASR)—often exceeding 80% under certain attack methods and risk categories—while state-of-the-art API-based LLMs (e.g., GPT-5, Claude 4 Sonnet, Gemini Pro) remain markedly more robust under all but the most sophisticated adversarial strategies.

Figure 3: Mean attack success rate (ASR) per risk category, highlighting elevated vulnerabilities in small and finance-specialized LLMs compared to API-based models.

Analysis reveals that the most vulnerable categories are Cyber Threats (R1) and, to a lesser extent, Financial Crime (R2), with GPTFuzzer attaining the highest mean ASR across categories and models. Intriguingly, contextually rich direct requests from schema-driven prompts achieve non-trivial ASRs even without optimization-based attacks, implying that the contextual realism of FinRED’s seeds circumvents simplistic refusal filters.

Comparative Pipeline and Human Validation

Blind expert studies confirm that the schema-driven pipeline (P3) substantially outperforms both context-free (P1) and naïve context-aware (P2) prompt generation across financial risk alignment, threat plausibility, and scenario specificity/actionability.

Figure 4: Domain experts overwhelmingly rate schema-driven seeds higher for risk fidelity, plausibility, and operational precision.

Operational ASR on the same 270 evaluated prompts further confirms the superiority of the schema-driven pipeline: 58.05% mean ASR on general sLMs, 70.28% on finance-specific sLMs, and 44.44% on API-based LLMs. This demonstrates that precise contextualization and threat-actor modeling increase both expert-rated realism and the diagnostic utility of adversarial testing.

Rubric Reliability and Expert Alignment

Inter-expert agreement studies (Figure 5) show substantial consensus ($>0.8$ pairwise rate) on safety labeling across diverse financial scenarios, underpinning rubric reliability. Furthermore, the FinRED rubric achieves 88.5% agreement with human experts—an 11.5-point improvement over the widely adopted HarmBench rubric, with a 57% reduction in critical false negatives (unsafe responses misclassified as safe).

Figure 5: High pairwise agreement among twelve FSI domain experts validates the consensus and objectivity of the ground-truth safety annotations.

Figure 6: FinRED's judge achieves significantly higher expert-LLM alignment and reduces critical errors compared to prior rubrics.

Practical and Theoretical Implications

FinRED’s integration into the FSI regulatory sandbox signifies its applied value for robust pre-deployment LLM safety auditing in finance. The framework’s decoupling of threat taxonomy from corpus retrieval enables rapid adaptation to emerging risks and regulatory landscape shifts. Its multidimensional rubric and expert-vetted seeds provide a foundation for future research in domain-specific adversarial stress testing, RLHF protocol enhancement, and fine-grained safety alignment in high-stakes LLM deployments.

From a theoretical standpoint, the strong result that context-rich, schema-driven seeds elicit unsafe behaviors undetected by static rubrics challenges assumptions about LLM refusal robustness and exposes limitations of universal red-teaming pipelines. The demonstrated Fragility of sLMs and even specialized FinLLMs under adversarial stress underscores the necessity for domain—rather than only model—centric safety frameworks.

Conclusion

FinRED constitutes a rigorous, expert-driven pipeline for constructing, categorizing, and evaluating adversarial financial LLM scenarios, advancing the state-of-the-art in domain-specific safety red-teaming. Its robust methodology, demonstrated reliability, and applied deployment create a foundation for more trustworthy, regulation-conformant, and resilient financial AI infrastructure. Future extensions include expansion to additional languages, jurisdictions, evolving fraud modalities, and the integration of reinforcement-based safety fine-tuning strategies.