TRepLiNa: Dual Frameworks in Translation & RL

• TRepLiNa is an overloaded term denoting two distinct research methods in low-resource machine translation and reinforcement learning.
• MT‑TRepLiNa employs layer-wise CKA alignment with REPINA anchoring on Aya‑23 8B to enhance LRL→HRL translation by constraining drift.
• RL‑TRepLiNa extends true online TD(λ) with sequential replay, providing improved value prediction in tasks like random walks and myoelectric control.

TRepLiNa is an overloaded research term that, in 2025 arXiv literature, denotes two unrelated methods. In low-resource machine translation, it names a training recipe for Aya‑23 8B that combines Centered Kernel Alignment (CKA) with REPINA to align internal representations across languages while constraining drift from the pretrained model (Nakai et al., 3 Oct 2025). In reinforcement learning, the same label is used as a shorthand for True Online TD‑Replan($\lambda$), a planning-through-replay extension of true online TD($\lambda$) for linear value prediction (Altahhan, 31 Jan 2025). For disambiguation, this entry uses “MT‑TRepLiNa” (Editor’s term) for the translation method and “RL‑TRepLiNa” (Editor’s term) for the reinforcement-learning method.

1. Dual designation and scope

The acronym collision is substantive rather than superficial: the two methods address different problem classes, use different mathematical objects, and operate in different experimental regimes.

Usage	Domain	Core mechanism
TRepLiNa	Low-resource machine translation	Layer-wise CKA + REPINA alignment on Aya‑23 8B
TRepLiNa	Reinforcement learning prediction	True Online TD‑Replan($\lambda$) with sequential online replay

MT‑TRepLiNa is introduced in the context of the 2025 Multimodal Models for Low-Resource Contexts and Social Impact (MMLoSo) Language Challenge, with LRL→HRL translation for Mundari→Hindi, Santali→English, and Bhili→Hindi (Nakai et al., 3 Oct 2025). RL‑TRepLiNa is introduced for standard RL prediction (policy evaluation) with linear function approximation, and is evaluated on a 17-state random walk and a myoelectric control domain (Altahhan, 31 Jan 2025).

A common misconception is to treat TRepLiNa as a single framework. The available evidence indicates the opposite: the term names two independent constructions that share only an acronym.

2. MT‑TRepLiNa: low-resource machine translation objective

MT‑TRepLiNa is a training recipe for improving low‑resource machine translation by explicitly aligning internal representations of a multilingual LLM across languages, while constraining the model not to drift too far from its pretrained state (Nakai et al., 3 Oct 2025). It is implemented on top of Aya‑23 8B, a decoder-only transformer LLM with ~8B parameters, pretrained on 23 languages, including English and Hindi, but not on Mundari, Bhili, Santali.

The motivating problem is low-resource LRL→HRL transfer where the model’s tokenizer and embeddings may be suboptimal, there is little or no direct supervision during pretraining, and the MT data is ~20k parallel sentence pairs per direction, split into ~95% train / 5% dev. The central observation, drawn from prior Aya‑23 8B analysis by Trinley et al. (2025), is that the model shows strong neuron overlap in the embedding layer across related languages, but this overlap drops sharply in mid and high layers. The resulting hypothesis is that selectively increasing cross-lingual similarity at layers where alignment is weakest may improve transfer from unseen LRLs to well-supported pivot languages (Nakai et al., 3 Oct 2025).

The method combines two ingredients. Representation alignment (CKA) encourages hidden states for parallel LRL and HRL sentences to become similar at a chosen layer. REPINA anchors the adapted representations to the pretrained model so that the HRL side does not drift excessively. This is presented as attractive in data-scarce settings because it reuses structure already learned for HRLs, avoids overfitting and catastrophic drift, and uses parallel LRL–HRL pairs efficiently (Nakai et al., 3 Oct 2025).

The translation setup is causal language modeling with prompts such as:

$B$2

The paper compares zero-shot prompting, few-shot prompting with 1, 3, or 5 in-context examples, and full fine-tuning (with QLoRA). TRepLiNa is used in the full fine-tuning regime (Nakai et al., 3 Oct 2025).

3. MT‑TRepLiNa: architecture, losses, and layer selection

The fine-tuning substrate is QLoRA, combining 4-bit NF4 quantization with LoRA adapters applied to q, k, v, o, gate, up, down projections, with hyperparameters $r = 16$, $\alpha = 32$, and dropout $= 0.05$ (Nakai et al., 3 Oct 2025). Only the low-rank adapters are trained; the base weights remain frozen. The setup uses bf16 or fp16, mixed-precision training, gradient checkpointing, and gradient accumulation, with global batch size ~16 via micro-batch size 1 + accumulation. The paper states that this is parameter-efficient, updating only ~0.77% of parameters from Zhang et al. 2023, and compute-efficient, with 8B models fitting in 40–80 GB GPUs even with hidden state retrieval.

For each parallel pair $(x^{(A)}, x^{(B)})$, where $A$ is the LRL and $B$ the HRL pivot, the method applies alignment at a single layer $\ell$. The combined objective is:

$\lambda$0

Here $\lambda$1 is a label-smoothed causal LM loss with $\lambda$2. The linear CKA similarity at layer $\lambda$3 is:

$\lambda$4

with the loss

$\lambda$5

The paper uses mean-centered token-wise hidden states, masks by pad, truncates to the minimum length across both languages, and flattens across the batch before computing CKA. It explicitly uses linear CKA rather than more complex kernels such as RBF (Nakai et al., 3 Oct 2025).

The REPINA term anchors current representations to a reference from the pretrained model with adapters disabled:

$\lambda$6

where the anchoring function $\lambda$7 is a stop-gradient identity mapping, implemented by detaching the reference representation from the computation graph. The paper distinguishes REPINA from standard weight decay by emphasizing that it regularizes representations $\lambda$8 rather than parameter values $\lambda$9 (Nakai et al., 3 Oct 2025).

Layer choice is central. Aya‑23 8B is treated as having 32 transformer blocks, and the layer sweep covers

$\lambda$0

The reported finding is that mid layers—especially layer 15—are best for TRepLiNa in LRL→HRL translation, whereas CKA-only peaks at layer 10 (Nakai et al., 3 Oct 2025). Early layers are described as already somewhat aligned, and very high layers as strongly tied to language-specific generation. This suggests that the effective alignment region is a semantic abstraction zone rather than the embedding layer or the output-proximal layers.

4. MT‑TRepLiNa: experimental results, ablations, and limitations

The evaluation uses the MMLoSo weighted composite metric

$\lambda$1

The paper also reports BLEU and chrF++ separately (Nakai et al., 3 Oct 2025). On full training, the composite results are:

Language pair	TRepLiNa (Ours)	REPINA-only	NoAlign
Bhili→Hindi	47.96	48.02	48.01
Mundari→Hindi	34.24	33.45	33.36
Santali→English	33.02	32.28	32.14

Prompting baselines are reported as extremely low, with composite scores in the ~1–5 range, while full fine-tuning raises performance into the 30–48 range (Nakai et al., 3 Oct 2025). For Mundari→Hindi, TRepLiNa improves over both REPINA-only and NoAlign by about 0.8–0.9 composite points. For Santali→English, the gain is similarly about 0.9 composite points. For Bhili→Hindi, REPINA-only slightly wins, with only very small differences among the three fine-tuning variants.

The paper’s interpretation is that TRepLiNa is particularly helpful for more distant language pairs such as Mundari–Hindi and Santali–English, whereas for a very closely related pair such as Bhili–Hindi, stronger CKA pressure may over-align and slightly harm useful language-specific cues (Nakai et al., 3 Oct 2025). This is presented as a pair-dependent trade-off rather than a universal advantage.

Several ablations refine the picture. In the 1k pairs, 1 epoch layer sweep, the method uses $\lambda$2 and $\lambda$3, except that for some Santali runs better results are reported with $\lambda$4 to avoid over-anchoring. In the ~20k pairs, up to 5 epochs regime, the paper lowers the CKA weight to $\lambda$5 and keeps $\lambda$6, arguing that stronger CKA can become over-regularizing in longer training (Nakai et al., 3 Oct 2025).

A notable highlight concerns Santali→English, where TRepLiNa achieves BLEU 25.24. The paper compares this with Billah et al. (2024), which reported BLEU 11.13 on its validation set, noting an absolute difference of +14.11 BLEU and about 2.27× relative, while also cautioning that the datasets may not be identical (Nakai et al., 3 Oct 2025).

The limitations are explicit. The paper explores only CKA as the similarity objective; it does not test cosine similarity, contrastive InfoNCE, or newer resonance-based objectives. It keeps $\lambda$7 and $\lambda$8 fixed rather than scheduled, applies CKA and REPINA at the same layer, and evaluates only LRL→HRL in the main experiments. It also notes tokenization/truncation issues for Santali, no human evaluation, no domain transfer experiments, and incomplete language-pair coverage within MMLoSo (Nakai et al., 3 Oct 2025).

5. RL‑TRepLiNa: planning-through-replay extension of true online TD($\lambda$9)

In reinforcement learning, TRepLiNa refers to True Online TD‑Replan($r = 16$0), sometimes written as True Online TD($r = 16$1)-Replan($r = 16$2) and abbreviated as TD‑Replan($r = 16$3) when $r = 16$4 (Altahhan, 31 Jan 2025). The setting is value prediction with linear function approximation in an MDP $r = 16$5, with value function

$r = 16$6

where $r = 16$7 is the feature vector and $r = 16$8 the learned weight vector. The paper is explicitly about prediction, not policy gradient or actor-critic (Altahhan, 31 Jan 2025).

The motivating limitation is that standard TD($r = 16$9) and true online TD($\alpha = 32$0) do not include explicit experience replay or planning, while conventional replay and Dyna-style planning generally operate with TD(0) targets or model-based simulation. RL‑TRepLiNa reinterprets planning as replaying previously observed experience with current weights, sequentially, in the order the trajectory was observed, and online, after each real interaction (Altahhan, 31 Jan 2025).

The point of departure is true online TD($\alpha = 32$1), which uses the TD error

$\alpha = 32$2

the eligibility trace

$\alpha = 32$3

and exact forward/backward-view equivalence for linear on-policy prediction. RL‑TRepLiNa adds an explicit replay mechanism that is mathematically exact with respect to replaying past experience and using multi-step interim $\alpha = 32$4-returns as targets (Altahhan, 31 Jan 2025).

The replay-depth parameter $\alpha = 32$5 is the distinguishing addition. When $\alpha = 32$6, the method reduces exactly to true online TD($\alpha = 32$7); when $\alpha = 32$8, it performs full replay of all past experience at every step; intermediate values give partial replay (Altahhan, 31 Jan 2025). When $\alpha = 32$9, the same symbol controls both the depth of the $= 0.05$0-return target and the density / intensity of replay.

6. RL‑TRepLiNa: bundled replay equations, complexity, and empirical behavior

The incremental formulation maintains the usual trace $= 0.05$1 together with an augmented replay trace $= 0.05$2 and a bundled transformation matrix $= 0.05$3. In the full-replay form, the weight update is

$= 0.05$4

In the partial-replay form, the update becomes

$= 0.05$5

The paper states two main theoretical properties. Theorem 1 shows equivalence between the explicit forward replay algorithm and the incremental backward-view update using $= 0.05$6 and $= 0.05$7. Theorem 2 shows that fixing the weights so as to remove replay reduces the method exactly to standard linear true online TD($= 0.05$8) (Altahhan, 31 Jan 2025).

The computational trade-off is explicit. True online TD($= 0.05$9) is $(x^{(A)}, x^{(B)})$0 per step, but RL‑TRepLiNa stores and updates the $(x^{(A)}, x^{(B)})$1 matrix $(x^{(A)}, x^{(B)})$2, making the method quadratic in complexity with respect to the number of features and requiring $(x^{(A)}, x^{(B)})$3 time and memory per step (Altahhan, 31 Jan 2025). The paper compares this cost profile to TD(0)-Replan(1) and Dyna Planning, which also have quadratic complexity in the relevant linear setting.

The empirical evaluation uses two benchmarks. The first is a 17-state random walk with simple binary one-hot features, $(x^{(A)}, x^{(B)})$4, and performance measured by RMSE against the analytically correct value function, averaged over 20 trials and shown over the first 10 episodes. The second is a myoelectric control domain based on 16 sEMG channels, with prediction of future cursor position, using both raw sEMG features and deep features from a sparse autoencoder, and results averaged over 66 trials (Altahhan, 31 Jan 2025).

Across both domains, the paper reports that RL‑TRepLiNa outperforms true online TD($(x^{(A)}, x^{(B)})$5), TD(0)-Replan, and Dyna Planning in the tested settings (Altahhan, 31 Jan 2025). On the random walk task, True Online TD‑Replan(1) dominates True Online TD($(x^{(A)}, x^{(B)})$6) for all $(x^{(A)}, x^{(B)})$7 values in this domain and is described as less sensitive to step size. In the myoelectric domain, the advantage is especially pronounced with deep autoencoder features, where the method is reported to clearly outperform true online TD($(x^{(A)}, x^{(B)})$8) for all $(x^{(A)}, x^{(B)})$9 in the fine step-size range, while Dyna Planning diverges or performs very poorly. The paper also reports that increasing $A$0 improves performance and stability up to about 0.8, while $A$1 may require smaller $A$2 (Altahhan, 31 Jan 2025).

A plausible implication is that the replay mechanism is most valuable when interactions are expensive and the final linear layer is moderate in dimension, whereas the $A$3 cost can become prohibitive for very large $A$4.

7. Comparative interpretation and disambiguation

The two TRepLiNa methods share a family resemblance only at a very high level: both attempt to improve learning by imposing structure on intermediate computations rather than relying solely on a base objective. MT‑TRepLiNa adds single-layer representation alignment + anchoring to standard QLoRA fine-tuning, while RL‑TRepLiNa adds sequential online replay to true online TD($A$5) (Nakai et al., 3 Oct 2025, Altahhan, 31 Jan 2025).

Their mathematical primitives are entirely different. MT‑TRepLiNa is expressed in terms of hidden-state matrices $A$6, linear CKA, and representation-space regularization. RL‑TRepLiNa is expressed in terms of feature vectors $A$7, weight vectors $A$8, eligibility traces, and the bundled matrix $A$9. Their computational profiles also differ: the translation method is described as relatively low-cost because CKA is applied at one layer and REPINA reference passes are periodic, whereas the reinforcement-learning method is explicitly quadratic in complexity in the number of features (Nakai et al., 3 Oct 2025, Altahhan, 31 Jan 2025).

The most important disambiguation principle is therefore bibliographic and conceptual. In machine translation, TRepLiNa refers to layer-wise CKA+REPINA alignment for Aya‑23 8B on MMLoSo LRL→HRL translation. In reinforcement learning, TRepLiNa refers to True Online TD‑Replan($B$0), a model-free replay-based extension of true online TD($B$1). Any technical discussion of “TRepLiNa” requires the domain to be specified explicitly.