FinMA: Open Financial LLM Suite

• FinMA is an open-source financial large language model suite, combining LLaMA-based architectures trained on the curated FIT dataset to address key financial NLP tasks.
• It employs a reproducible instruction-tuning pipeline with tailored prompt engineering and defined training protocols for tasks like sentiment analysis and entity extraction.
• Robust benchmarking via FLARE highlights its strengths in text classification while revealing limitations in numerical reasoning and stock movement prediction.

FinMA is an open-source financial LLM suite introduced in "PIXIU: A LLM, Instruction Data and Evaluation Benchmark for Finance" (Xie et al., 2023). It establishes a reproducible pipeline for instruction-tuning LLaMA-based LLMs on curated datasets encompassing major financial NLP and prediction tasks, and provides a standardized benchmark (FLARE) for rigorous comparative evaluation. FinMA targets critical use cases in financial document analysis, classification, entity extraction, and prediction, offering public models, datasets, and evaluation recipes designed for direct academic and enterprise adoption.

1. Model Architecture and Training Objectives

FinMA leverages the LLaMA transformer backbone, releasing two primary variants—FinMA-7B (7B parameters) and FinMA-30B (30B parameters)—matching the original LLaMA configurations (32 layers/4096-d hidden/32 heads for 7B; 60 layers/6656-d hidden/52 heads for 30B). The instruction-tuning process applies a next-token cross-entropy loss:

$$\mathcal{L}_{\mathrm{instr}}(\theta) = -\sum_{t=1}^{|y|} \log p_{\theta}\bigl(y_t \mid x,\,y_{<t}\bigr)$$

where $x$ is the concatenation of “Instruction,” “Context,” and “Question,” while $y$ contains the response tokens.

No auxiliary objectives or reward-modulated (RLHF) loss terms are employed. The minimization is strictly over the supervised instruction data.

2. Instruction Dataset Construction: FIT

FinMA is trained on FIT, a 136,609-sample, multi-task instruction dataset specifically curated for finance. FIT covers five financial NLP tasks and one prediction task, with samples constructed from both public datasets and synthetic augmentation by domain experts:

• Financial Sentiment Analysis: FPB (Financial Phrase Bank) and FiQA-SA, totaling ≈60,000 samples with multiple prompt rewrites per raw input.
• News Headline Classification: Gold Headline Dataset, 11,412 labeled samples.
• Named Entity Recognition: FIN-NER, 13,660 entity-annotated samples via prompt variations.
• Question Answering: FinQA and ConvFinQA consisting of 8,281 and 3,892 samples, respectively, including tabular contexts.
• Stock Movement Prediction: BigData22, ACL18, and CIKM18, collectively exceeding 39,000 samples involving time-series and news/tweet inputs.

Sampling is balanced by applying all prompt templates to smaller corpora and random-prompt selection to larger ones. All source text is normalized for consistent prompt-response formatting. See below for the core mapping.

Task Type	Representative Dataset(s)	Modality
Sentiment Analysis	FPB, FiQA-SA	Text
News Classification	Gold Headline	Text
Named Entity Recognition	FIN-NER	Text
Question Answering	FinQA, ConvFinQA	Text + Table
Stock Movement Prediction	BigData22, ACL18, CIKM18	Text + Time-series

Table: FIT task and modality coverage (Xie et al., 2023)

3. Fine-Tuning Protocol and Prompt Engineering

FinMA models are fine-tuned with AdamW (β₁=0.9, β₂=0.95, ε=1e–8), weight decay 1e–5, constant learning rate (8×10⁻⁶) post-linear warmup, and batch sizes of 32 (7B) or 24 (30B). Epochs vary by run (e.g., 15 for FinMA-7B, 20 for FinMA-30B). The maximum sequence length is 2048 tokens. All training is on A100 GPUs, with no retrieval augmentation or multi-modal inputs.

Prompt templates are tailored by task:

• Single-turn classification:

[Instruction: …] Text: [input text] Response: [label/entities]

• QA (FinQA, ConvFinQA):

Instruction: [prompt…] Context: [text/table] Question: [question] Response: [answer]

• Stock movement:

Analyze tweets & closing prices for {TICKER} at {DATE}. Respond Rise or Fall. Text: [tweets + prices] Response: [Rise/Fall]

4. FLARE Evaluation Benchmark and Metrics

FLARE is the diagnostic suite associated with FinMA, incorporating nine datasets spanning five NLP tasks and one prediction task:

• Sentiment Analysis: FPB (9,700 test), FiQA-SA (2,350 test)
• Headline Classification: Gold Headline (20,547 test)
• NER: FIN-NER (980 test)
• QA: FinQA (1,147 test), ConvFinQA (2,161 test)
• Prediction: BigData22 (1,471 test), ACL18 (3,719 test), CIKM18 (1,142 test)

Metrics are strictly standardized:

• Accuracy (ACC): $\frac{\#\{\,\hat y_i = y_i\}}{N}$
• F1, Precision, Recall: conventional multiclass/entity forms
• Exact-Match (EM) for QA: $$\frac{\#\{\text{answer}_\mathrm{pred} = \text{answer}_\mathrm{gold}\}}{N}$$
• Matthews Correlation Coefficient (MCC): for binary prediction

5. Empirical Performance Analysis

A head-to-head comparison with baselines illustrates FinMA’s task strengths and limitations. Zero-shot FinMA-7B substantially outperforms GPT-4 and BloombergGPT on classification and sentiment analysis:

Task/Metric	ChatGPT	GPT-4	BloombergGPT	FinMA-7B	FinMA-30B	FinMA-7B-full
FPB F1	0.78	0.78	0.51	0.86	0.88	0.87
FiQA-SA F1	—	—	0.75	0.84	0.87	0.79
Headline Avg F1	0.77	0.86	0.82	0.98	0.97	0.97
NER Entity F1	0.77	0.83	0.61	0.75	0.62	0.69
FinQA EM Acc	0.58	0.63	—	0.06	0.11	0.04

FinMA models present persistent deficiencies in mathematical QA (FinQA, ConvFinQA, EM ≈ 0.06–0.11 vs 0.63 for GPT-4) and stock-movement prediction (MCC remains near zero across all models). Model scaling from 7B to 30B yields minimal improvement without more diverse training data. Task-specific fine-tuning incrementally enhances certain tasks, such as ACL18 prediction (MCC increasing to 0.10 in FinMA-7B-full).

6. Domain Suitability, Use Recommendations, and Limitations

FinMA is recommended for out-of-the-box deployment in:

• Financial sentiment and news/tweet classification
• Entity extraction from regulatory documents
• Headline/sector classification

For quantitative QA, external calculator or chain-of-thought modules are necessary; for time-series prediction, hybridization with dedicated models is required. The instruction dataset remains limited in mathematical reasoning coverage, explaining QA weaknesses. Existing input length constraints (2048 tokens) may preclude long-report analysis without retrieval-augmented extensions.

Known limitations include:

• Numerical reasoning: fundamental underperformance on QA requiring calculation steps
• Prediction: inability to capture actionable time-series dynamics from text alone
• Scale: full-data fine-tuning at 30B scale was not undertaken due to resource ceilings
• Ethical risk: all outputs should be subject to human-in-the-loop validation to prevent misinformation in sensitive decision pipelines

7. Outlook and Relation to Subsequent Developments

FinMA inaugurated replicable, open financial LLM development, instruction dataset curation, and benchmarking. Subsequent releases such as FinMA-ES (Zhang et al., 2024) extended this paradigm to bilingual (Spanish–English) finance, leveraging cross-linguistic transfer via interleaved instruction data and establishing the FLARE-ES multilingual evaluation suite. This suggests future progress will depend on expanding modality coverage, enhancing numerical and multi-lingual capacities, and integrating retrieval-augmented or RLHF training to address the persistent reasoning and prediction bottlenecks. Model capacity scaling, dataset diversification, and ethical governance remain primary axes for ongoing research and application in the financial NLP domain.