TIPA: Typologically Informed Parameter Aggregation

• TIPA is a training-free method that combines language adapter parameters based on typological similarity to enable zero-shot cross-lingual transfer for low-resource languages.
• It leverages structured URIEL+ feature vectors to compute similarity weights, ensuring that typologically similar source adapters contribute more effectively.
• Integrating with the MAD-X framework, TIPA achieves significant gains across over 230 languages on tasks including NER, POS tagging, QA, and topic classification.

Typologically Informed Parameter Aggregation (TIPA) is a training-free algorithmic approach for proxy language adapter construction in massively multilingual transformer models. TIPA leverages typological similarity, derived from structured language feature sets, to combine the parameters of existing adapters and enable zero-shot cross-lingual transfer, especially for ^^^^1^^^^ and unseen languages. Its integration into the MAD-X modular adapter framework achieves significant gains over baselines in diverse natural language processing tasks across over 230 languages (Accou et al., 23 Jan 2026).

1. Formal Model and Algorithmic Foundations

TIPA operates over a frozen multilingual transformer $M$ (e.g., XLM-RoBERTa), supplemented by a pool of $N$ pre-trained language adapters $\{A^{(1)},...,A^{(N)}\}$, each fine-tuned on a distinct source language $\ell_i$. Adapter parameters $A^{(i),(L)}$ at transformer layer $L$ include weight and bias matrices.

For a target language $\ell_{\mathrm{tgt}}$ with no dedicated adapter, TIPA constructs the proxy adapter $A^{(\mathrm{tgt})}$ at layer $L$ via a weighted sum:

$$A^{(\mathrm{tgt}),(L)} = \sum_{i=1}^N w_i^{(\mathrm{tgt})} A^{(i),(L)}$$

where weights $w_i^{(\mathrm{tgt})}$ encode typological similarity between $\ell_{\mathrm{tgt}}$ and each source language $\ell_i$. The weights derive from the normalized URIEL+ distance function $d(\ell_{\mathrm{tgt}},\ell_i)\in [0,1]$: $s_i = 1 - d(\ell_{\mathrm{tgt}},\ell_i)$

$$w_i^{(\mathrm{tgt})} = \frac{\exp(s_i)}{\sum_{j=1}^N \exp(s_j)} = \frac{\exp(1 - d(\ell_{\mathrm{tgt}},\ell_i))}{\sum_{j=1}^N \exp(1 - d(\ell_{\mathrm{tgt}},\ell_j))}$$

By this mechanism, TIPA aggregates parameters such that typologically proximate source adapters contribute more.

2. Computation of Typological Similarity

TIPA exploits structured feature vectors sourced from the URIEL+ database (Khan et al., 2025), encompassing syntactic, morphological, phonological, genetic, and geographic attributes. Each language $\ell$ is represented by a normalized vector $\mathbf{x}_\ell$. The default similarity calculation uses Euclidean distance in the featural space: $$d(\ell_a,\ell_b) = \|\mathbf{x}_{\ell_a} - \mathbf{x}_{\ell_b}\|_2$$ This distance is linearly rescaled to $[0,1]$ across all source adapters for each target. Additionally, ablations restricted to morphological-only and syntactic-only features examine the impact of typology subspaces. No additional clustering or dimension reduction occurs beyond URIEL+'s preprocessing (e.g., PCA for phonology).

3. Integration with MAD-X Modular Adapters

Within the MAD-X framework, two distinct adapter types exist:

• Task adapter $T$: fine-tuned on labeled English data.
• Language adapter $L$: trained via monolingual LM on each source $\ell_i$.

TIPA substitutes the standard language adapter at inference with the proxy $A^{(\mathrm{tgt})}$, preserving the MAD-X architecture. The inference pipeline is:

1. Compute typological similarity weights $w_i$ for $\ell_{\mathrm{tgt}}$.
2. Aggregate adapter parameters using the weighted sum.
3. Inject $A^{(\ell_{\mathrm{tgt}})}$ into the MAD-X stack replacing the target or closest adapter.
4. Predict labels for the target language using the composition $M(A_{\mathrm{proxy}}(T(x_{\mathrm{input}})))$.

Provided code illustrates the aggregation and inference steps:

for i in 1..N: s_i = 1 - d(ℓ_tgt, ℓ_i) for i in 1..N: w_i = softmax_i({s_j}_{j=1..N}) for each layer L: A_proxy[L].W = sum_{i=1}^N w_i * A^(i)[L].W A_proxy[L].b = sum_{i=1}^N w_i * A^(i)[L].b y_hat = M(A_proxy(T(x_input)))

4. Zero-Shot Cross-Lingual Transfer Protocol

The TIPA procedure enforces a strict zero-shot regime:

• The task adapter is always trained on English.
• For each $\ell_{\mathrm{tgt}}$, a proxy adapter $A^{(\ell_{\mathrm{tgt}})}$ is constructed post hoc without further training.
• The proxy substitutes the language adapter at inference, and predictions are generated for $\ell_{\mathrm{tgt}}$ test instances. This protocol ensures that the system never encounters $\ell_{\mathrm{tgt}}$ data during fine-tuning, explicitly addressing low-resource and unseen language evaluation.

5. Empirical Evaluation and Performance

TIPA is assessed on five multilingual NLP tasks covering 234 languages:

• Named Entity Recognition: WikiAnn (134 languages)
• POS Tagging: Universal Dependencies (80 languages)
• COPA: XCOPA (11 languages)
• QA: XQuAD (12 languages)
• Topic Classification: SIB-200 (176 languages)

The following baselines are compared:

• English-only fine-tuning
• MAD-X with actual/closest adapters
• "No Train but Gain" (English+closest adapter, Klimaszewski et al., 2025)
• Uniform averaging across all source adapters

Table 3 demonstrates that TIPA (featural weighting) achieves the highest aggregate metric across all tasks, significantly outperforming uniform averaging (+6.7% gain, $p<0.01$) and English-only fine-tuning (up to +10–15% on token-level tasks). The greatest improvements are noted for languages without any dedicated adapter (Table 6). Figure 1 highlights that token-level tasks (NER, POS) benefit strongly from typological weighting, and higher-order semantic tasks (COPA, QA, SIB) remain competitive, occasionally surpassing MAD-X baselines for resource-rich languages.

6. Ablations and Analytical Findings

Examination of typology-feature ablations (Table 7) reveals:

• Featural distance (all URIEL+ features) yields the strongest aggregate results across tasks.
• Morphological distance alone is optimal for NER and POS ($p\leq 0.05$ vs. featural).
• Syntactic distance performs best for SIB topic classification (+0.5% gain over featural, $p\leq 0.01$).

Two source-adapter pruning strategies are evaluated:

• Retaining top-$k$ (with $k=5$) nearest adapters
• Including adapters with similarity $\geq 0.33$

Both pruning approaches yield modest additional gains ($p\leq 0.03$), especially when applied with syntactic distance for SIB. No universally optimal pruning parameter is identified, suggesting that task-specific tuning may be required.

7. Limitations and Directions for Future Research

TIPA's effectiveness is bounded by several factors:

• The approach does not mitigate issues caused by the multilingual transformer's inability to process previously unseen scripts or tokens.
• Performance is sensitive to the breadth and quality of the available source adapter pool.
• Parameter choices (feature subsets, $k$ for pruning, thresholds) are set heuristically and not exhaustively optimized.
• Evaluation is limited to 234 languages, which is a small fraction of the world's ~7000 languages, implying that "low-resource" remains a skewed sample.
• Architecture specificity is evident: preliminary attempts to port TIPA to other models and PEFT approaches (e.g., Gemma, Qwen, LoRA) are less successful, indicating that future investigations must verify TIPA's generalizability across backbone architectures.

TIPA offers a parameter-efficient, training-free methodology for generating language adapters by leveraging structured typological priors. Its architecture-agnostic weighting scheme, grounded in URIEL+ feature distances, supports robust zero-shot cross-lingual transfer in scenarios devoid of labeled or monolingual target-language training data (Accou et al., 23 Jan 2026).