---
title: 'Furina: Multilingual, Security & Roleplay'
url: https://www.emergentmind.com/topics/furina
type: topic
---

# Furina: Multilingual, Security & Roleplay

Furina is a polysemous term in recent arXiv literature rather than a single established concept. In contemporary usage it denotes, in distinct and unrelated research programs, a multilingual pretrained language model derived from Glot500-m to mitigate the script barrier in cross-lingual representation learning, a backbone used for zero-shot cross-lingual Semantic Textual Relatedness in SemEval-2024, a jailbreak attack based on refusal instability in large language models and multimodal large language models, and a role-playing evaluation framework comprising FURINA-Builder and FURINA-Bench. It should also be distinguished from **furin**, the host serine protease central to several SARS-CoV-2 docking studies, whose name is orthographically similar but conceptually unrelated [2401.06620] [2404.02570] [2605.26158] [2510.06800] [2107.12799].

## 1. Furina as a multilingual pretrained language model

In multilingual NLP, **Furina** is the model produced by fine-tuning **Glot500-m** with the **TransliCo** framework, whose explicit goal is to address the **script barrier**: the tendency of multilingual pretrained language models to place languages written in different scripts into different representational subspaces, thereby weakening cross-lingual transfer [2401.06620].

The source model, Glot500-m, is a RoBERTa/XLM-R–style transformer continued-pretrained on **Glot500-c**, a **1.5B-sentence corpus covering 511 languages, 30 scripts, 534 language–script pairs**. Furina is obtained by selecting **5% of sentences from each language–script** in Glot500-c, yielding approximately **75M sentence pairs** consisting of an original sentence and its **Latin transliteration** generated with **Uroman**. Fine-tuning uses two objectives: standard **Masked Language Modeling (MLM)** on both original and transliterated text, and **Transliteration Contrastive Modeling (TCM)** at sentence level. The overall training objective is

$$
\mathcal{L}_{\text{training}} = \lambda_1 \mathcal{L}_{\text{MLM}} + \lambda_2 \mathcal{L}_{\text{MLM}}^{\text{trans}} + \lambda_3 \mathcal{L}_{\text{TCM}},
$$

with $\lambda_1 = \lambda_2 = \lambda_3 = 1$. TCM uses **mean-pooled 8th-layer token embeddings** and an InfoNCE-style loss with **cosine similarity** and **temperature $\tau = 1$**. The training setup updates the full encoder and MLM head; no layers are frozen.

The model’s design premise is that transliteration increases lexical sharing across scripts while the contrastive term explicitly aligns a sentence with its transliteration. This yields a model that continues to support original scripts at inference time, rather than requiring universal romanization. A plausible implication is that Furina occupies a middle position between script-preserving multilingual encoders and romanization-only approaches: it uses Latin script as an alignment bridge without forcing a single-script deployment regime.

## 2. Representation geometry and zero-shot transfer behavior

Furina is presented not merely as a training recipe but as a representational intervention. PCA visualizations of **8th-layer sentence representations** on SR-B data show that **Glot500-m** forms clear script-specific clusters, whereas **Furina** yields a much more intermixed geometry with substantial overlap among scripts such as Latin, Cyrillic, Arabic, and Hebrew [2401.06620].

The empirical evaluation spans **sentence retrieval** on **SR-B** and **SR-T**, **text classification** on **Taxi1500**, **named entity recognition** on **WikiANN**, and **part-of-speech tagging** on **UD v2.11**. The largest gains appear on cross-script, sequence-level transfer. On **SR-B**, Furina reaches **58.1** average top-10 retrieval accuracy versus **47.2** for Glot500-m. On **Taxi1500**, Furina reaches **61.0** macro F1 versus **54.3** for Glot500-m. On **NER**, Furina reaches **62.8** versus **61.6**. On **POS**, Furina reaches **71.9** versus **71.8**.

| Task | Glot500-m | Furina |
|---|---:|---:|
| SR-B | 47.2 | 58.1 |
| SR-T | 70.7 | 68.8 |
| Taxi1500 | 54.3 | 61.0 |
| NER | 61.6 | 62.8 |
| POS | 71.8 | 71.9 |

The gains are not uniform across scripts. Furina improves Latin, Cyrillic, Arabic, and Devanagari group performance on several tasks, but it degrades substantially on some **Han (Hani)** settings, especially on **SR-T** and **POS**. The paper attributes these failures to **domain mismatch** and to the fact that transliteration of logographic scripts such as Chinese can be lossy and semantically ambiguous. This establishes an important limitation: transliteration-based alignment is effective when phonological or orthographic regularities survive romanization, but not universally.

A local **Indic** case study strengthens the model’s central claim. Fine-tuning on **16.5M sentence pairs** from **12 Indic languages** improves average **NER** on IndicGLUE from **82.2** to **83.5** and **Wikipedia Section Title Prediction** from **85.0** to **85.3**. Because these languages are closely related yet distributed across Bengali, Devanagari, Gujarati, Gurmukhi, Sinhala, and Oriya scripts, the case study isolates cross-script alignment as the primary variable rather than language-family distance alone.

## 3. Furina in zero-shot cross-lingual Semantic Textual Relatedness

In SemEval-2024 Task 1, Track C, Furina appears as one of two backbone encoders—alongside **XLM-R**—for zero-shot **Semantic Textual Relatedness (STR)**. In this context it is described as a **multilingual transformer encoder** derived from XLM-R, specialized for cross-script alignment, covering **511 low-resource languages**, and fine-tuned on **Glot500-m** using **5%** of pretraining sentences in original script plus their **Latin transliterations** [2404.02570].

The task predicts a real-valued relatedness score $y \in [0,1]$ for a sentence pair $(s_1,s_2)$. Training uses **Mean Squared Error**,

$$
\mathcal{L}_\text{MSE} = \frac{1}{N} \sum_{i=1}^{N} ( y_i - \hat{y}_i )^2,
$$

and evaluation uses **Spearman’s rank correlation**. The study evaluates three source-language selection regimes: **single-source transfer**, **English plus two nearest neighbors (kNN)**, and **multi-source transfer** with either all languages or languages from the same family.

Average results show substantial sensitivity to source selection.

| Furina configuration | Average Spearman |
|---|---:|
| Furina\_eng | 0.62 |
| Furina\_kNN | 0.67 |
| Furina\_MS-All | 0.71 |
| Furina\_MS-Fam | 0.71 |

These results encode several distinct findings. First, donor choice is decisive in single-source transfer: the **best possible single-source choice (MAX)** reaches **0.71**, while the **worst possible choice (MIN)** falls to **0.36**. Second, multi-source training improves Furina over the English-only baseline by **+0.09** on average. Third, family-restricted multi-source transfer can mitigate **language interference**: for **Spanish**, **Furina\_MS-Fam = 0.72** whereas **Furina\_MS-All = 0.59**.

The model is especially notable on **Kinyarwanda (kin, C8)**. Kinyarwanda’s nearest neighbors under the cell-state similarity scheme are **Spanish (esp)** and **Hausa (hau)**, so the submitted system is **Furina\_eng+esp+hau**. That system achieves **0.68** Spearman and ranks **1st for C8 (Kinyarwanda)** on the official leaderboard. The paper attributes part of this strength to a coverage asymmetry: unlike XLM-R, Furina had seen **Kinyarwanda** during pretraining. By contrast, **machine translation augmentation** and **transliteration with Uroman** are not uniformly helpful for STR; for example, transliteration reduces Hindi from **0.72** to **0.67** under the Furina kNN setup. The broader conclusion is that script-robust pretraining does not imply that explicit romanization at task time will improve fine-grained semantic regression.

## 4. Furina as a refusal-instability jailbreak attack

In LLM safety research, **Furina** names a different object entirely: **Fragmented Uncertainty-Driven Refusal Instability Attack**. This work rejects the assumption that safety alignment behaves as a near-binary threshold. Instead it defines a **compliance probability**

$$
T_\theta(x) := \mathbb{E}_{Y\sim p_\theta(\cdot\mid x)}[C(Y)],
$$

and partitions input space into **stable refusal**, **stable compliance**, and an **instability band** in which refusals are stochastic and sensitive to small perturbations [2605.26158].

The paper introduces a multi-metric diagnostic framework. External signals include **token-level entropy** $H_{\text{tok}}$ and **semantic entropy** $H_{\text{sem}}$; internal signals include **HiddenDetect** ($HD_{\max}$) and **Refusal Direction** ($RD_{\max}$). The empirical signature of instability is a decoupling phenomenon: **high external uncertainty and high attack success rate, but decreased internal safety activation**. On **LLaMA-2-7B** over semantic rewrite levels, **ASR** rises from **0.01** to **0.42**, **$H_{\text{tok}}$** rises from **0.345** to **0.435**, **$H_{\text{sem}}$** rises from **0.088** to **0.147**, while **$RD_{\max}$** falls from **0.6770** to **0.0830**.

The attack mechanism operationalizes this diagnosis through **fragmented, scene-anchored prompts**. A harmful task is decomposed by an auxiliary model into benign-looking probes and a metaphorical scene description. For multimodal targets, the scene can be realized as a **typographic image** or a **stable-diffusion scene**. The target model answers the local probes, and an auxiliary model then synthesizes the final answer. The method is explicitly described as requiring **no model-specific optimization**, no gradients, and no suffix search.

On **HarmBench**, Furina exceeds strong single-turn and multi-turn baselines. Examples include **92.5** ASR on **LLaMA-3-8B** versus **79.0** for **ActorBreaker**, **94.0** on **GPT-4o-mini** versus **82.0**, **93.5** on **Gemini-2.5-Flash** versus **85.5**, and **83.5** on **Claude-Haiku-4.5** versus **65.0**. On **MM-SafetyBench**, Furina remains competitive, with **Typo** mode reaching **93.81** on **GPT-4o-mini** and **77.20** on **Claude-Haiku-4.5**. Ablations show that **semantic probes** and **synthesis** are the critical components: removing probes collapses several category scores from the **91.41–100.00** range to near zero. Classical per-turn defenses perform poorly; **LlamaGuard** intercepts only **1/200** samples and reduces ASR only from **94.0** to **93.5** on GPT-4o-mini.

## 5. FURINA as a role-playing benchmark ecosystem

A third major usage appears in evaluation research, where **FURINA** denotes a role-playing framework consisting of **FURINA-Builder** and **FURINA-Bench**. Here the goal is not representation alignment or jailbreaking but scalable construction of customizable role-playing benchmarks for LLMs [2510.06800].

**FURINA-Builder** models a test character as

$$
c_{\mathrm{test}} = \{(k_i, v_i, \tau_i)\}_{i=1}^n,\quad \tau_i \in \{\text{public}, \text{private}\},
$$

and inserts that character into a scenario pool extracted from books. The pool contains **6,556 scenario fragments** from **80 Chinese novels** and **100 English books**. A **director model** manages multi-party dialogue, a **scene-character model** plays non-test characters, a **source model** and a **base model** generate candidate test-character replies, and a **judge model** selects the better reply and the most appropriate evaluation dimension. Dimension balancing uses **Dynamically Weighted Random Selection**, with weights

$$
w_i = c_{\max} - c_i + 1,\qquad
P(d_i) = \frac{w_i}{\sum_j w_j}.
$$

The instantiated benchmark, **FURINA-Bench**, contains **20 test characters**, **1,471 unique roles**, **1,459** multi-party dialogues, and **7,181** dimension-tagged evaluation instances, with an average of approximately **19.8** turns per conversation. The five evaluation dimensions are **Context Reliance (CR)**, **Factual Recall (FR)**, **Reflective Reasoning (RR)**, **Conversational Ability (CA)**, and **Preference Alignment (PA)**. Scoring uses pairwise comparison against **GPT-4.1** with a reward mapping $f:\{1,2,3,4,5\}\rightarrow\{3,1,0.5,0,0\}$ and a normalized overall performance score in $[0,1]$.

Empirically, the benchmark surfaces three findings. First, **o3** achieves the best **English** overall score at approximately **43.98/100**, while **DeepSeek-R1** achieves the best **Chinese** overall score at approximately **73.38**. Second, **established characters** consistently outperform **synthesized characters** across all models, suggesting strong dependence on pretraining exposure. Third, reasoning improves role-playing performance but also increases **role-playing hallucinations**. The paper identifies an empirical **Pareto frontier** between role-playing performance and reliability, with reliability defined as the inverse hallucination rate:

$$
\text{Reliability score} = \frac{100}{\text{hallucination rate (\%)}}.
$$

This formulation makes FURINA unusual among evaluation suites: it is simultaneously a benchmark and a pipeline for continuously rebuilding benchmarks as interaction styles, models, and application requirements evolve.

## 6. Distinction from furin and other naming confusions

The most common confusion is between **Furina/FURINA** and **furin**. The latter is a biological entity: a **794-residue serine endoprotease encoded by the FURIN gene**, a member of the subtilisin-like proprotein convertase family, ubiquitously expressed in mammalian tissues and involved in cleavage of precursor proteins at motifs of the form $\text{Arg–X–(Arg/Lys)–Arg}$. In SARS-CoV-2 research it is studied because cleavage of spike at the **S1/S2** junction enhances viral entry, infectivity, and tissue tropism [2107.12799] [2106.05440].

This biochemical literature is unrelated to the NLP and LLM usages of Furina. One docking study identifies **saquinavir**, **nelfinavir**, and **atazanavir** as top candidate furin inhibitors with cumulative inhibitory indices **2.52**, **2.16**, and **2.13**. Another in silico study reports **mozenavir** binding furin with **−12.04 kcal/mol**, stronger than several other SARS-CoV-2-related targets. These are statements about host protease inhibition, not about multilingual language models, jailbreaks, or role-playing benchmarks.

The coexistence of these meanings has methodological consequences for literature retrieval. This suggests that accurate search and citation practice requires explicit domain qualifiers—such as **TransliCo**, **SemEval STR**, **refusal instability**, **role-playing benchmark**, or **serine protease**—because the token “Furina” alone does not identify a unique research object. In present arXiv usage, the term is best understood as a family of unrelated proper names that happen to coincide orthographically while belonging to different technical subfields.

Source: https://www.emergentmind.com/topics/furina