---
title: 'FinSim-2: Financial Hypernym Classification Task'
url: https://www.emergentmind.com/topics/finsim-2
type: topic
---

# FinSim-2: Financial Hypernym Classification Task

Searching arXiv for FinSim-2 papers and related task descriptions.
FinSim-2 is a shared task in financial-domain semantic classification centered on hypernym detection: given a financial concept label or term, a system must identify the most relevant top-level concept, or hypernym, from a predefined label set in an external ontology. In the task formulations described in the literature, systems may return a ranked list of candidate labels, and performance is evaluated by both top-1 correctness and the position of the true label in that ranking. Published approaches situate FinSim-2 at the intersection of domain NLP, ontology-based reasoning, and knowledge-graph-enhanced classification, with representative systems drawing on the Financial Industry Business Ontology (FIBO), Word2vec, BERT, WordNet, Wikidata, and WebIsALOD [2106.09230].

## 1. Task definition and semantic scope

FinSim-2 is described as a hypernym detection challenge within the financial services domain. The core problem is formulated as multi-class classification: for each input concept label, the system must assign the most relevant top-level hypernym from a fixed inventory of labels. One description states that the task is to classify concepts from the financial domain into the most relevant hypernym concept in an external ontology, specifically the Financial Industry Business Ontology [2106.09230]. Another describes the task as classifying financial terms into their most relevant hypernym, or top-level concept, from an external ontology [2007.11201].

A more explicit FinSim-2 label inventory consists of ten predefined and mutually exclusive categories: Equity Index, Credit Index, Bonds, Swap, Option, Funds, Future, MMIs (Money Market Instruments), Stocks, and Forward [2103.01576]. The task permits ranked outputs rather than only single-label predictions, which is important because the official metrics include both accuracy and mean rank. This ranked-output requirement encourages systems that combine hard classification with candidate ordering, for example by merging ontology-derived predictions with probability distributions from supervised models [2106.09230].

The task has a clear semantic orientation: it is not merely lexical matching, but top-level concept assignment under domain-specific constraints. At the same time, several system reports note that lexical surface structure remains highly informative. In particular, many terms contain their label as a substring, which creates a class of comparatively easy instances and motivates rule-based handling in some systems [2007.11201]. This dual character—semantic classification with strong lexical cues—shapes much of the methodological diversity in the FinSim-2 literature.

## 2. Data configurations and annotation regimes

Published descriptions of the task report two distinct data configurations. One report describes a training set of 100 terms, each paired with its correct hypernym label across 8 possible classes, together with 150 financial prospectus PDFs in English that serve as unlabeled corpus data for embedding training [2007.11201]. A later FinSim-2 system paper describes a training set of 614 hyponym-hypernym pairs labeled with one of ten classes, plus a test set of 212 entries [2103.01576].

This discrepancy indicates that the literature spans closely related task settings rather than a single invariant benchmark. A plausible implication is that FinSim-2 should be understood as a family of shared-task formulations around financial hypernym assignment, with continuity at the level of problem structure even when dataset size and label inventory differ.

Several recurring data characteristics are reported. The labeled sets are small, which constrains the use of data-hungry supervised methods [2007.11201]. The class distribution can be imbalanced; for the 614-example training set, Equity Index appears 286 times whereas Forward appears only 9 times [2103.01576]. The corpora used for domain adaptation may also be noisy: PDF conversion introduces noise and token or sentence boundary detection issues during conversion to text [2007.11201]. Another reported coverage problem is that some terms in the labeled data do not appear in the provided corpus, which has direct consequences for embedding quality when systems rely on corpus-trained vector representations [2007.11201].

These properties are not incidental. They explain why many successful systems favor hybrid designs: rule-based components for overt lexical regularities, ontology traversal for explainable structure, and relatively simple classifiers or shallow neural models to exploit whatever supervised signal is available.

## 3. Ontologies, label spaces, and external knowledge sources

A defining feature of FinSim-2 is its reliance on external knowledge resources. In the ontology-centered formulation, the target space is grounded in FIBO, and each financial label is assumed to correspond to a node in the ontology graph [2106.09230]. The ontology is transformed into a NetworkX directed, unweighted graph, technically a NetworkX MultiGraph, after which classification becomes a problem of mapping an input concept into that graph and traversing upward to locate an admissible shared-task label.

The JSI system operationalizes ontology mapping in several stages. If a concept exists in FIBO as a node with the same name, direct mapping is used. For multi-word concepts or concepts absent from FIBO, the system splits them into component words, beginning with the last word, assumed to be the noun or head, and attempts singular or plural matching against ontology nodes. If that fails, it searches for synonyms using WordNet. Two labels, Equity Index and Credit Index, receive manual treatment because they lacked direct FIBO representations: Equity Index was manually added as the parent of Index in FIBO, and unmappable concepts default to Credit Index [2106.09230].

A different knowledge-centric line of work, represented by FinMatcher, uses three publicly available knowledge graphs—WordNet, Wikidata, and WebIsALOD—to derive both explicit and latent features for each label-class pair [2103.01576]. Here the external resources are not used primarily for symbolic ontology reasoning, but for feature construction. Explicit features include word overlap, Wikidata hypernym lookup, WordNet hypernym lookup, and WebIsALOD hypernym lookup. Latent features are generated via pretrained RDF2vec embeddings for WebIsALOD nodes, with cosine similarity used to measure semantic relatedness between linked label and class entities [2103.01576].

The contrast between these approaches is instructive. In the ontology-graph approach, background knowledge drives prediction directly through graph search. In the knowledge-graph feature approach, background knowledge is converted into numerical signals for a classifier. Both paradigms treat external structure as indispensable, but they place it at different points in the inference pipeline.

## 4. Representation learning and classification strategies

The FinSim-2 literature exhibits three broad modeling families: embedding-based term-label similarity, ontology-augmented symbolic classification, and feature-based supervised learning with knowledge-graph signals.

The IITK system combines context-independent and context-dependent embeddings. It trains Word2vec embeddings from scratch on the financial prospectus corpus and explores vector sizes from 50 to 500. Each term is represented as the average of its constituent word vectors, and label embeddings are constructed similarly. The same work also uses BERT Base Uncased, with 12 layers, 12 attention heads, and 110 million parameters, fine-tuned on the financial PDF corpus; for each word in a term, the embedding from the last hidden layer is extracted and then averaged to obtain the term representation [2007.11201].

Its preprocessing pipeline consists of removal of punctuation, stop words, and special characters; lowercasing; lemmatization and tokenization using NLTK; embedding lookup for tokens; and construction of the term vector as the average of token vectors [2007.11201]. Classification is then split by a domain rule. Terms containing exactly one label are placed in Group 1 and handled by rule-based direct mapping. Group 2 contains terms where the label appeared twice or the label was missing; these are passed to either unsupervised distance-based methods or simple supervised classifiers. The unsupervised variants compute cosine similarity, minimum distance, or absolute vector distance between a term embedding and each label embedding, assigning the label with maximum cosine similarity. The supervised variants use Bernoulli Naive Bayes and one-vs-rest 8-class logistic regression on average term embeddings [2007.11201].

The JSI system takes a different route. It performs ontology-based classification by mapping a concept into FIBO and then generalizing upward through parent nodes until a valid shared-task label is found; this is described as essentially a breadth-first search up the hierarchy with stop conditions. To supplement this symbolic prediction, the system trains a Random Forest classifier on Word2vec vector representations of concepts. For each concept, the classifier predicts a probability distribution over all possible labels, and the ontology-based label is then moved to the top of the ranked list from Random Forest [2106.09230].

FinMatcher constructs per-class signal vectors from explicit and latent knowledge-graph features and concatenates them into a single feature vector,
$$
S = s_1 \Vert s_2 \Vert s_3 \Vert s_4 \Vert s_5,
$$
where each $s_i \in \mathbb{R}^{10}$ is a length-10 signal vector [2103.01576]. The classifier is a simple feedforward artificial neural network with a single fully connected layer and output size 10, trained with Mean Squared Error for 100 epochs and batch size 25. To address imbalance, it uses SMOTE to upsample minority classes to at least one third of the majority-class count in the training split [2103.01576].

Across these systems, a consistent theme is restraint in model complexity relative to data scale. This is explicit in the IITK paper, which states that the small dataset precludes use of complex models [2007.11201]. Even when neural models are used, as in FinMatcher, the architecture remains shallow and heavily dependent on manually designed knowledge signals rather than large end-to-end fine-tuning [2103.01576].

## 5. Evaluation protocol and reported performance

Two evaluation metrics recur across the reported systems: accuracy and mean rank. Accuracy is the fraction of terms correctly assigned to their true hypernym, or, equivalently, correct top-1 predictions [2007.11201]. Mean rank, also described as average label rank, measures the average position of the correct label in the returned ranked list; lower values are better [2106.09230].

The IITK submission reports that its system ranks 1st based on both metrics, namely mean rank and accuracy [2007.11201]. The paper attributes this performance to a hybrid strategy combining rule-based mapping, custom-trained Word2vec, fine-tuned BERT, and simple classifiers.

The JSI paper reports several internal and official results. On training data, the ontology-based method achieves 0.87 accuracy, corresponding to 535 out of 614 correctly classified instances, outperforming the provided distance-based classifier at 0.62 and slightly exceeding logistic regression at 0.86. Its Random Forest classifier attains 0.85 accuracy on a 10% held-out set. The merged method yields a slight boost in average label rank, improving from 1.37 for Random Forest alone to 1.19 when the ontology-based prediction is placed first. On the official test set, the system reaches accuracy 0.811, ranking 15th out of 18, and average label rank 1.316, ranking 12th of 18 [2106.09230].

FinMatcher reports 81.1% accuracy and mean rank 1.415 on test data. In five-fold cross-validation on the training data, the submission entry reaches 86.7% accuracy and mean rank 1.43 [2103.01576]. Its ablation study shows that word overlap is particularly important: removing it drops accuracy from 86.69 to 60.51 and worsens mean rank from 1.432 to 2.007. Removing Wikidata, WordNet, WebIsALOD hypernym, or RDF2vec features causes only modest degradation, suggesting partial redundancy among those knowledge signals [2103.01576].

These results collectively indicate that FinSim-2 rewards systems that optimize not only class assignment but also ranking quality. They also show that no single modeling family dominates all settings: first-place performance is reported for a hybrid embedding-and-rule system in one configuration, while ontology-centric and knowledge-graph-centric systems achieve competitive but different trade-offs in other settings.

## 6. Methodological themes, bottlenecks, and interpretive issues

One major methodological theme is hybridization. The IITK system explicitly divides the test data into rule-based and classifier-based subsets [2007.11201]. The JSI system combines ontology-based prediction with a Random Forest ranked list [2106.09230]. FinMatcher integrates lexical overlap, symbolic hypernym lookup, and latent graph embeddings in a single classifier [2103.01576]. This convergence suggests that FinSim-2 is not well served by purely distributional or purely symbolic methods in isolation.

A second theme is the importance of lexical structure. Word overlap is reported as critical in FinMatcher, and the IITK report notes that many terms contain their label as a substring [2103.01576]. A plausible implication is that a substantial fraction of FinSim-2 instances can be approached through label-aware lexical heuristics before deeper semantic machinery is applied. This does not reduce the task to string matching, but it does mean that robust systems exploit lexical transparency when present.

A third theme is mapping as bottleneck. In the ontology-driven JSI system, errors are typically due to failure in mapping, including ambiguous multi-word expressions and lack of domain synonyms [2106.09230]. The paper explicitly suggests that better word-to-ontology matching and improved head noun detection could further improve performance. This is a distinct challenge from downstream classification: even a well-structured ontology cannot help if the concept cannot be aligned to the graph.

A fourth issue is data scarcity and imbalance. Small labeled sets limit model complexity [2007.11201], while class skew motivates explicit imbalance handling such as SMOTE [2103.01576]. Combined with corpus noise from PDF conversion and coverage gaps between labeled terms and unlabeled corpora, these constraints explain why many systems emphasize feature engineering, ontology use, and controlled model capacity rather than large supervised architectures.

## 7. Position within financial NLP and related research directions

Within financial NLP, FinSim-2 occupies a specialized niche: semantic categorization of domain concepts against a structured hypernym inventory. Its significance lies in the fact that the target labels are not arbitrary classes but top-level financial concepts, often grounded in FIBO or analogous ontology structures [2106.09230]. This gives the task both practical and methodological relevance for semantic indexing, ontology population, and explainable concept classification.

The published systems illustrate three related research directions. One is domain-adaptive embedding learning from financial prospectus text, including both Word2vec trained from scratch and BERT fine-tuned on the provided corpus [2007.11201]. Another is explainable ontology reasoning through graph transformation and upward traversal in FIBO [2106.09230]. A third is the combination of explicit and latent knowledge-graph features, including hypernym reachability and RDF2vec similarity, within a classifier [2103.01576].

The literature also surfaces a recurring misconception: that ontologies eliminate the need for empirical modeling altogether. The JSI paper notes that ontology-based classification does not require training data for that part of the system, but it also shows that mapping quality is a practical limitation and supplements ontology reasoning with a learned Random Forest ranker [2106.09230]. Conversely, the strong ablation effect of word overlap in FinMatcher cautions against assuming that sophisticated graph features will necessarily dominate simpler lexical signals [2103.01576].

Taken together, the available work presents FinSim-2 as a benchmark family where structured finance knowledge, lexical regularities, and compact statistical models interact under severe data constraints. This suggests that future progress is likely to depend less on scaling model size than on better concept-to-ontology alignment, more robust handling of multi-word financial expressions, and more systematic integration of ranking-oriented inference with domain-specific background knowledge.

Source: https://www.emergentmind.com/topics/finsim-2