Named Entity Recognition for Payment Data Using NLP

Abstract: Named Entity Recognition (NER) has emerged as a critical component in automating financial transaction processing, particularly in extracting structured information from unstructured payment data. This paper presents a comprehensive analysis of state-of-the-art NER algorithms specifically designed for payment data extraction, including Conditional Random Fields (CRF), Bidirectional Long Short-Term Memory with CRF (BiLSTM-CRF), and transformer-based models such as BERT and FinBERT. We conduct extensive experiments on a dataset of 50,000 annotated payment transactions across multiple payment formats including SWIFT MT103, ISO 20022, and domestic payment systems. Our experimental results demonstrate that fine-tuned BERT models achieve an F1-score of 94.2% for entity extraction, outperforming traditional CRF-based approaches by 12.8 percentage points. Furthermore, we introduce PaymentBERT, a novel hybrid architecture combining domain-specific financial embeddings with contextual representations, achieving state-of-the-art performance with 95.7% F1-score while maintaining real-time processing capabilities. We provide detailed analysis of cross-format generalization, ablation studies, and deployment considerations. This research provides practical insights for financial institutions implementing automated sanctions screening, anti-money laundering (AML) compliance, and payment processing systems.

• The paper introduces PaymentBERT, a format-aware BERT-CRF model using payment embeddings and structural features that achieves 95.7% F1 on 50,000 synthetic annotated payment messages.
• PaymentBERT outperforms FinBERT by 1.5 F1 points and loses only 2.8 points on unseen payment formats, demonstrating stronger cross-format robustness than CRF, BiLSTM, and generic BERT systems.
• The results support practical compliance deployment through GPU inference, distillation, and INT8 quantization, while synthetic data, multilingual coverage, temporal drift, and persistent PURPOSE boundary errors limit production conclusions.

Overview

This paper addresses named entity recognition (NER) for payment message data, a domain that has received comparatively little attention relative to news-wire or biomedical text. The author formulates entity extraction over payment messages as sequence labeling with a BIO scheme covering six entity types—PERSON_NAME, ORGANIZATION, ACCOUNT_NUMBER, LOCATION, AMOUNT, and PURPOSE—and evaluates classical, recurrent, and transformer-based approaches on a curated corpus of 50,000 annotated payment messages spanning SWIFT MT103, ISO 20022 pain.001, ACH, SEPA, and regional formats. The central contribution is PaymentBERT, a hybrid architecture that augments BERT with domain-specific payment embeddings and explicit format features, achieving 95.7% F1-score with real-time-capable inference (2602.14009).

Motivation and problem setting

Payment data poses challenges distinct from the corpora on which standard NER models are trained. Messages mix rigid field-based structure (e.g., SWIFT tags such as :50K: for ordering customer) with substantial free-text content; single transactions frequently contain multiple languages; abbreviations ("BNFCRY", "MSG") and non-standard naming conventions are pervasive. The paper identifies three driving constraints: domain specificity (structural cues and well-defined patterns such as IBAN check digits and BIC formats that generic models ignore), accuracy requirements imposed by regimes such as the Bank Secrecy Act, PSD2, and OFAC/UN/EU sanctions lists, where false negatives carry fines potentially in the millions of dollars per violation, and performance constraints, since high-volume processors handle tens of thousands of transactions per minute at peak.

The problem is formulated as token-level classification into a 13-label set ($2 \times 6 + 1$ under BIO). Notably, PURPOSE entities—free-text remittance information—are included despite being the least structured category.

Dataset

The dataset comprises 50,000 messages: 20,000 MT103, 15,000 ISO 20022 pain.001, 7,000 ACH, 5,000 SEPA credit transfers, and 3,000 other regional formats. Realistic complexity is deliberately injected: 23% multilingual messages, 15% non-standard formatting, 8% nested entities, mean length of 487 characters ($\sigma = 312$), and entity density of 12.3 entities per message ($\sigma = 4.8$). Annotation was performed independently by two financial compliance experts with adjudication by a senior expert, yielding Cohen's $\kappa = 0.89$. Splits are stratified at 70/15/15.

An important caveat stated by the author: the data is synthetic, constructed to protect payment privacy while approximating realistic linguistic patterns. All reported results therefore reflect performance on simulated rather than production traffic, which limits direct transferability claims.

Models evaluated

The benchmark spans five families: a rule-based system (regex plus gazetteers), off-the-shelf spaCy and Stanford NER taggers, a CRF with extensive payment-specific features (orthographic, contextual, lexicon, pattern-matching for IBAN/BIC, and transition features), BiLSTM-CRF with character-level embeddings, and three BERT variants—fine-tuned BERT-base, FinBERT adaptation, and BERT-CRF hybrid decoding. Custom preprocessing preserves SWIFT field tags as single tokens and splits account numbers and reference codes on meaningful boundaries.

PaymentBERT fuses three information sources: BERT's contextual representations, payment embeddings initialized from FastText trained on 10M payment messages, and format features encoding field type, normalized position within field, binary pattern indicators (IBAN, BIC, date, currency code), and message type embeddings. These pass through an MLP fusion layer followed by CRF decoding. Training is staged: BERT is frozen for the first epoch while fusion layers train, then unfrozen end-to-end with differential learning rates (1e-5 for BERT layers versus 1e-3 for fusion and CRF).

Results

The headline comparison shows a clear progression across model classes:

Model	Precision (%)	Recall (%)	F1 (%)	Latency (ms)
Rule-based	76.3	68.1	71.9	2.3
Stanford NER	78.9	74.2	76.5	6.7
CRF (domain features)	84.7	78.9	81.7	4.2
BiLSTM-CRF	89.3	86.1	87.7	12.5
BERT-base	93.1	91.5	92.3	45.2
FinBERT	94.8	93.6	94.2	46.8
PaymentBERT	96.3	95.1	95.7	52.1

Several findings stand out. The feature-engineered CRF outperforms generic baselines by 5.2 points, confirming that pattern features (IBAN/BIC) and field-type indicators carry substantial signal. FinBERT's financial pre-training yields a 1.9-point gain over BERT-base. PaymentBERT's 1.5-point improvement over FinBERT is statistically significant under paired bootstrap testing (10,000 iterations, $p < 0.001$), corresponding to roughly 450 additional correct entities on the test set. Attention analysis indicates BERT learns to attend to field indicators without supervision, effectively rediscovering the manually engineered CRF features.

Per-entity results show AMOUNT (97.5% F1), PERSON_NAME (97.2%), and ACCOUNT_NUMBER (96.8%) performing strongly due to regular patterns, while PURPOSE lags at 91.2% owing to free-text diversity and boundary ambiguity. For compliance-relevant recall, PaymentBERT achieves 96.6% on PERSON_NAME but only 92.7% on ORGANIZATION—a figure worth scrutiny given that missed organization names in sanctions screening carry asymmetric costs.

Cross-format generalization is a key differentiator: on unseen formats, CRF degrades 15.3 points (handcrafted features overfit to training conventions), BiLSTM-CRF 8.7, BERT-base 4.2, and PaymentBERT only 2.8 points (95.7% → 92.9%). The explicit format features appear to help the model separate format-specific conventions from universal payment semantics.

Ablations attribute the largest single-component contribution to FastText-initialized payment embeddings (−1.3 F1 when removed), followed by continued pre-training on unlabeled payment data (−0.8), format features (−0.9), and staged training with differential learning rates (−1.0 combined effect). Word2Vec substitution costs 0.7 points, confirming subword modeling matters for abbreviations.

Error analysis over 200 sampled errors finds boundary errors dominate (42%), followed by type confusion (23%), nested entities (18%), abbreviations (11%), and format variations (6%). This distribution motivates the author's recommendation of span-based architectures over pure BIO tagging.

Deployment considerations

On CPU, PaymentBERT runs at 52.1 ms per message (~19 msg/sec/core); a GPU (NVIDIA T4) reaches 180 msg/sec. Two optimized variants preserve most accuracy: knowledge distillation to a 66M-parameter model achieves 94.9% F1 at 28.3 ms, and INT8 quantization retains 95.6% F1 at 35.2 ms. Dynamic batching at batch size 8 provides a 2.4× throughput improvement with P95 latency rising from 52 ms to 75 ms. The author estimates that an institution processing 1M transactions/day needs roughly 0.15 cores with batching, or 2–4 cores with redundancy headroom—modest infrastructure by production standards.

Limitations and open questions

The paper acknowledges several constraints plainly. The evaluation rests on synthetic data, so behavior on genuine production traffic—including adversarial or anomalous inputs—is unverified. Language coverage is primarily English with some European languages; multilingual content appears in only 23% of messages, leaving cross-lingual generalization largely open. Temporal stability is unaddressed: payment standards and terminology evolve, and the paper does not measure drift degradation over time. Error cost asymmetry is noted but not operationalized—the system reports aggregate F1 rather than cost-sensitive metrics weighting false negatives in sanctions screening more heavily. Explainability remains limited, a concern for regulated deployments. Finally, the 42% share of boundary errors suggests the BIO formulation itself may be a bottleneck; whether span-based heads would close this gap is left untested.

Conclusion

This work provides a systematic benchmark of NER approaches on payment data and demonstrates that combining transformer representations with explicit domain knowledge—payment embeddings and format-aware features—yields measurable gains: 95.7% F1, statistical significance over FinBERT, and strong cross-format robustness (only 2.8 points lost on unseen formats). The deployment analysis shows the accuracy-efficiency trade-off is manageable through distillation and quantization. The principal caveats are the synthetic nature of the dataset and untested temporal and multilingual robustness; validating these results on live production traffic and addressing boundary detection via span-based methods remain the most immediate open questions raised by the study.