---
title: 'TRepLiNa: Dual Frameworks in Translation & RL'
url: https://www.emergentmind.com/topics/treplina
type: topic
---

# TRepLiNa: Dual Frameworks in Translation & RL

TRepLiNa is an overloaded research term that, in 2025 arXiv literature, denotes two unrelated methods. In low-resource machine translation, it names a training recipe for Aya‑23 8B that combines Centered Kernel Alignment (CKA) with REPINA to align internal representations across languages while constraining drift from the pretrained model [2510.06249]. In reinforcement learning, the same label is used as a shorthand for True Online TD‑Replan(\(\lambda\)), a planning-through-replay extension of true online TD(\(\lambda\)) for linear value prediction [2501.19027]. For disambiguation, this entry uses **“MT‑TRepLiNa”** (*Editor’s term*) for the translation method and **“RL‑TRepLiNa”** (*Editor’s term*) for the reinforcement-learning method.

## 1. Dual designation and scope

The acronym collision is substantive rather than superficial: the two methods address different problem classes, use different mathematical objects, and operate in different experimental regimes.

| Usage | Domain | Core mechanism |
|---|---|---|
| TRepLiNa | Low-resource machine translation | Layer-wise CKA + REPINA alignment on Aya‑23 8B |
| TRepLiNa | Reinforcement learning prediction | True Online TD‑Replan(\(\lambda\)) with sequential online replay |

MT‑TRepLiNa is introduced in the context of the **2025 Multimodal Models for Low-Resource Contexts and Social Impact (MMLoSo) Language Challenge**, with LRL→HRL translation for **Mundari→Hindi**, **Santali→English**, and **Bhili→Hindi** [2510.06249]. RL‑TRepLiNa is introduced for **standard RL prediction (policy evaluation)** with **linear function approximation**, and is evaluated on a **17-state random walk** and a **myoelectric control domain** [2501.19027].

A common misconception is to treat TRepLiNa as a single framework. The available evidence indicates the opposite: the term names two independent constructions that share only an acronym.

## 2. MT‑TRepLiNa: low-resource machine translation objective

MT‑TRepLiNa is a training recipe for improving low‑resource machine translation by **explicitly aligning internal representations** of a multilingual LLM across languages, while **constraining the model not to drift too far from its pretrained state** [2510.06249]. It is implemented on top of **Aya‑23 8B**, a **decoder-only transformer LLM with ~8B parameters**, pretrained on **23 languages**, including **English and Hindi**, but **not** on **Mundari, Bhili, Santali**.

The motivating problem is low-resource LRL→HRL transfer where the model’s **tokenizer and embeddings may be suboptimal**, there is **little or no direct supervision during pretraining**, and the MT data is **~20k parallel sentence pairs per direction**, split into **~95% train / 5% dev**. The central observation, drawn from prior Aya‑23 8B analysis by **Trinley et al. (2025)**, is that the model shows **strong neuron overlap in the embedding layer across related languages**, but this **overlap drops sharply in mid and high layers**. The resulting hypothesis is that selectively increasing cross-lingual similarity at layers where alignment is weakest may improve transfer from unseen LRLs to well-supported pivot languages [2510.06249].

The method combines two ingredients. **Representation alignment (CKA)** encourages hidden states for parallel LRL and HRL sentences to become similar at a chosen layer. **REPINA** anchors the adapted representations to the pretrained model so that the HRL side does not drift excessively. This is presented as attractive in **data-scarce settings** because it **reuses** structure already learned for HRLs, **avoids overfitting and catastrophic drift**, and **uses parallel LRL–HRL pairs efficiently** [2510.06249].

The translation setup is **causal language modeling** with prompts such as:

```text
Translate to Hindi:
<Mundari source sentence>
```

The paper compares **zero-shot prompting**, **few-shot prompting** with **1, 3, or 5** in-context examples, and **full fine-tuning (with QLoRA)**. TRepLiNa is used in the full fine-tuning regime [2510.06249].

## 3. MT‑TRepLiNa: architecture, losses, and layer selection

The fine-tuning substrate is **QLoRA**, combining **4-bit NF4** quantization with **LoRA** adapters applied to **`q, k, v, o, gate, up, down`** projections, with hyperparameters \(r = 16\), \(\alpha = 32\), and dropout \(= 0.05\) [2510.06249]. Only the low-rank adapters are trained; the base weights remain frozen. The setup uses **bf16 or fp16**, **mixed-precision training**, **gradient checkpointing**, and **gradient accumulation**, with **global batch size ~16 via micro-batch size 1 + accumulation**. The paper states that this is **parameter-efficient**, updating only **~0.77% of parameters** from **Zhang et al. 2023**, and **compute-efficient**, with 8B models fitting in **40–80 GB GPUs even with hidden state retrieval**.

For each parallel pair \((x^{(A)}, x^{(B)})\), where \(A\) is the LRL and \(B\) the HRL pivot, the method applies alignment at a **single layer** \(\ell\). The combined objective is:

$$
L = L_{\text{MT}} + \lambda L_{\text{CKA}} + \mu L_{\text{REPINA}}.
$$

Here \(L_{\text{MT}}\) is a **label-smoothed causal LM loss** with \(\epsilon = 0.1\). The **linear CKA similarity** at layer \(\ell\) is:

$$
\mathrm{CKA}(H^{(A)}_{\ell}, H^{(B)}_{\ell}) \;=\; \frac{\| X^\top Y \|_F^2}{\sqrt{\| X^\top X \|_F^2}\,\sqrt{\| Y^\top Y \|_F^2}},
$$

with the loss

$$
L_{\text{CKA}} = 1 - \mathrm{CKA}(H^{(A)}_{\ell}, H^{(B)}_{\ell}).
$$

The paper uses **mean-centered** token-wise hidden states, **masks by pad**, **truncates to the minimum length across both languages**, and **flattens across the batch** before computing CKA. It explicitly uses **linear CKA** rather than more complex kernels such as **RBF** [2510.06249].

The REPINA term anchors current representations to a reference from the pretrained model with adapters disabled:

$$
L_{\text{REPINA}}\!\bigl(H^{(A)}_{\text{pre}\,\ell},H^{(A)}_{\ell}\bigr) \;=\; \bigl\|\,H^{(A)}_{\text{pre}\,\ell} - \tilde{\phi}\!\bigl(H^{(A)}_{\ell}\bigr)\,\bigr\|_2^2,
$$

where the anchoring function \(\tilde{\phi}(\cdot)\) is a **stop-gradient identity mapping**, implemented by detaching the reference representation from the computation graph. The paper distinguishes REPINA from standard weight decay by emphasizing that it regularizes **representations** \(H_\ell\) rather than **parameter values** \(\|\theta\|_2^2\) [2510.06249].

Layer choice is central. Aya‑23 8B is treated as having **32** transformer blocks, and the layer sweep covers

$$
\ell \in \{1, 2, 5, 10, 15, 20, 25, 30, 31, 32\}.
$$

The reported finding is that **mid layers**—especially **layer 15**—are best for TRepLiNa in LRL→HRL translation, whereas **CKA-only** peaks at **layer 10** [2510.06249]. Early layers are described as already somewhat aligned, and very high layers as strongly tied to language-specific generation. This suggests that the effective alignment region is a semantic abstraction zone rather than the embedding layer or the output-proximal layers.

## 4. MT‑TRepLiNa: experimental results, ablations, and limitations

The evaluation uses the MMLoSo weighted composite metric

$$
\text{Score} = 0.6 \times \text{BLEU} + 0.4 \times \text{chrF}.
$$

The paper also reports **BLEU** and **chrF++** separately [2510.06249]. On full training, the composite results are:

| Language pair | TRepLiNa (Ours) | REPINA-only | NoAlign |
|---|---:|---:|---:|
| Bhili→Hindi | 47.96 | 48.02 | 48.01 |
| Mundari→Hindi | 34.24 | 33.45 | 33.36 |
| Santali→English | 33.02 | 32.28 | 32.14 |

Prompting baselines are reported as **extremely low**, with composite scores in the **~1–5** range, while full fine-tuning raises performance into the **30–48** range [2510.06249]. For **Mundari→Hindi**, TRepLiNa improves over both **REPINA-only** and **NoAlign** by about **0.8–0.9 composite points**. For **Santali→English**, the gain is similarly about **0.9 composite points**. For **Bhili→Hindi**, **REPINA-only slightly wins**, with only very small differences among the three fine-tuning variants.

The paper’s interpretation is that TRepLiNa is particularly helpful for **more distant language pairs** such as **Mundari–Hindi** and **Santali–English**, whereas for a **very closely related pair** such as **Bhili–Hindi**, stronger CKA pressure may **over-align** and slightly harm useful language-specific cues [2510.06249]. This is presented as a pair-dependent trade-off rather than a universal advantage.

Several ablations refine the picture. In the **1k pairs, 1 epoch** layer sweep, the method uses \(\lambda = 0.05\) and \(\mu = 0.05\), except that for some **Santali** runs better results are reported with \(\mu = 0.01\) **to avoid over-anchoring**. In the **~20k pairs, up to 5 epochs** regime, the paper lowers the CKA weight to \(\lambda = 0.01\) and keeps \(\mu = 0.05\), arguing that stronger CKA can become over-regularizing in longer training [2510.06249].

A notable highlight concerns **Santali→English**, where TRepLiNa achieves **BLEU 25.24**. The paper compares this with **Billah et al. (2024)**, which reported **BLEU 11.13** on its validation set, noting an absolute difference of **+14.11 BLEU** and about **2.27× relative**, while also cautioning that **the datasets may not be identical** [2510.06249].

The limitations are explicit. The paper explores **only CKA** as the similarity objective; it does not test **cosine similarity**, **contrastive InfoNCE**, or **newer resonance-based objectives**. It keeps \(\lambda\) and \(\mu\) fixed rather than scheduled, applies CKA and REPINA at the **same layer**, and evaluates only **LRL→HRL** in the main experiments. It also notes **tokenization/truncation issues** for **Santali**, no **human evaluation**, no **domain transfer experiments**, and incomplete language-pair coverage within MMLoSo [2510.06249].

## 5. RL‑TRepLiNa: planning-through-replay extension of true online TD(\(\lambda\))

In reinforcement learning, TRepLiNa refers to **True Online TD‑Replan(\(\lambda\))**, sometimes written as **True Online TD(\(\lambda\))-Replan(\(\acute{\lambda}\))** and abbreviated as **TD‑Replan(\(\lambda\))** when \(\acute{\lambda}=\lambda\) [2501.19027]. The setting is **value prediction with linear function approximation** in an MDP \((\mathcal{S},\mathcal{A},p,r,\gamma)\), with value function

$$
V(s \mid \boldsymbol{\theta}) = \boldsymbol{\theta}^\top \boldsymbol{\phi}(s),
$$

where \(\boldsymbol{\phi}(s)\in\mathbb{R}^n\) is the feature vector and \(\boldsymbol{\theta}\in\mathbb{R}^n\) the learned weight vector. The paper is explicitly about **prediction**, not **policy gradient** or **actor-critic** [2501.19027].

The motivating limitation is that standard **TD(\(\lambda\))** and **true online TD(\(\lambda\))** do not include **explicit experience replay or planning**, while **conventional replay** and **Dyna-style planning** generally operate with **TD(0)** targets or model-based simulation. RL‑TRepLiNa reinterprets planning as **replaying previously observed experience with current weights**, **sequentially**, in the order the trajectory was observed, and **online**, after each real interaction [2501.19027].

The point of departure is true online TD(\(\lambda\)), which uses the TD error

$$
\delta_t = R_{t+1} + \gamma\,\boldsymbol{\theta}_t^\top \boldsymbol{\phi}_{t+1} - \boldsymbol{\theta}_t^\top \boldsymbol{\phi}_t,
$$

the eligibility trace

$$
\boldsymbol{e}_t = \gamma\lambda \boldsymbol{e}_{t-1} + \alpha_t\boldsymbol{\phi}_t\left(1 - \boldsymbol{e}_{t-1}^\top\boldsymbol{\phi}_t\right),
$$

and exact forward/backward-view equivalence for linear on-policy prediction. RL‑TRepLiNa adds an explicit replay mechanism that is mathematically exact with respect to replaying past experience and using multi-step interim \(\lambda\)-returns as targets [2501.19027].

The replay-depth parameter \(\acute{\lambda}\in[0,1]\) is the distinguishing addition. When \(\acute{\lambda}=0\), the method reduces exactly to true online TD(\(\lambda\)); when \(\acute{\lambda}=1\), it performs **full replay of all past experience at every step**; intermediate values give **partial replay** [2501.19027]. When \(\acute{\lambda}=\lambda\), the same symbol controls both the **depth of the \(\lambda\)-return target** and the **density / intensity of replay**.

## 6. RL‑TRepLiNa: bundled replay equations, complexity, and empirical behavior

The incremental formulation maintains the usual trace \(\boldsymbol{e}_t\) together with an **augmented replay trace** \(\bar{\boldsymbol{e}}_t\) and a **bundled transformation matrix** \(\bar{\boldsymbol{A}}_t\). In the full-replay form, the weight update is

$$
\boldsymbol{\theta}_{t+1} = \bar{\boldsymbol{A}}_t\boldsymbol{\theta}_t + \bar{\boldsymbol{e}}_t.
$$

In the partial-replay form, the update becomes

$$
\boldsymbol{\theta}_{t+1} = \bar{\boldsymbol{A}}_t\left(\acute{\lambda}\boldsymbol{\theta}_t + (1-\acute{\lambda})\boldsymbol{\theta}_0\right) + \bar{\boldsymbol{e}}_t.
$$

The paper states two main theoretical properties. **Theorem 1** shows equivalence between the explicit forward replay algorithm and the incremental backward-view update using \(\bar{\boldsymbol{A}}_t\) and \(\bar{\boldsymbol{e}}_t\). **Theorem 2** shows that fixing the weights so as to remove replay reduces the method exactly to standard linear true online TD(\(\lambda\)) [2501.19027].

The computational trade-off is explicit. True online TD(\(\lambda\)) is \(O(n)\) per step, but RL‑TRepLiNa stores and updates the \(n\times n\) matrix \(\bar{\boldsymbol{A}}_t\), making the method **quadratic in complexity with respect to the number of features** and requiring **\(O(n^2)\) time and memory per step** [2501.19027]. The paper compares this cost profile to **TD(0)-Replan(1)** and **Dyna Planning**, which also have quadratic complexity in the relevant linear setting.

The empirical evaluation uses two benchmarks. The first is a **17-state random walk** with **simple binary one-hot features**, **\(\gamma = 1\)**, and performance measured by **RMSE** against the analytically correct value function, averaged over **20 trials** and shown over the first **10 episodes**. The second is a **myoelectric control domain** based on **16 sEMG channels**, with prediction of future cursor position, using both **raw sEMG features** and **deep features from a sparse autoencoder**, and results averaged over **66 trials** [2501.19027].

Across both domains, the paper reports that RL‑TRepLiNa **outperforms true online TD(\(\lambda\))**, **TD(0)-Replan**, and **Dyna Planning** in the tested settings [2501.19027]. On the random walk task, **True Online TD‑Replan(1) dominates True Online TD(\(\lambda\)) for all \(\lambda\) values in this domain** and is described as **less sensitive to step size**. In the myoelectric domain, the advantage is especially pronounced with **deep autoencoder features**, where the method is reported to **clearly outperform true online TD(\(\lambda\)) for all \(\lambda\)** in the fine step-size range, while **Dyna Planning diverges or performs very poorly**. The paper also reports that increasing \(\acute{\lambda}\) improves performance and stability up to about **0.8**, while \(\acute{\lambda}=1\) may require smaller \(\alpha\) [2501.19027].

A plausible implication is that the replay mechanism is most valuable when interactions are expensive and the final linear layer is moderate in dimension, whereas the \(O(n^2)\) cost can become prohibitive for very large \(n\).

## 7. Comparative interpretation and disambiguation

The two TRepLiNa methods share a family resemblance only at a very high level: both attempt to improve learning by imposing structure on intermediate computations rather than relying solely on a base objective. MT‑TRepLiNa adds **single-layer representation alignment + anchoring** to standard QLoRA fine-tuning, while RL‑TRepLiNa adds **sequential online replay** to true online TD(\(\lambda\)) [2510.06249; 2501.19027].

Their mathematical primitives are entirely different. MT‑TRepLiNa is expressed in terms of hidden-state matrices \(H_\ell\), **linear CKA**, and **representation-space regularization**. RL‑TRepLiNa is expressed in terms of feature vectors \(\boldsymbol{\phi}_t\), weight vectors \(\boldsymbol{\theta}_t\), eligibility traces, and the bundled matrix \(\bar{\boldsymbol{A}}_t\). Their computational profiles also differ: the translation method is described as **relatively low-cost** because CKA is applied at **one layer** and REPINA reference passes are periodic, whereas the reinforcement-learning method is explicitly **quadratic in complexity** in the number of features [2510.06249; 2501.19027].

The most important disambiguation principle is therefore bibliographic and conceptual. In machine translation, TRepLiNa refers to **layer-wise CKA+REPINA alignment** for **Aya‑23 8B** on **MMLoSo** LRL→HRL translation. In reinforcement learning, TRepLiNa refers to **True Online TD‑Replan(\(\lambda\))**, a model-free replay-based extension of **true online TD(\(\lambda\))**. Any technical discussion of “TRepLiNa” requires the domain to be specified explicitly.

Source: https://www.emergentmind.com/topics/treplina