TF2: TinyFabulist Translation Framework

• TinyFabulist Translation Framework (TF2) is a specialized pipeline for English–Romanian literary translation of moral fables, integrating bilingual datasets, parameter-efficient fine-tuning, and narrative-aware evaluation.
• It employs a two-stage fine-tuning process with LoRA adapters to adapt open-weight language models, achieving near-proprietary translation quality while dramatically reducing costs.
• TF2’s reproducible design offers complete datasets, fine-tuned models, and evaluation scripts, advancing research in domain-specific, low-resource machine translation.

Searching arXiv for the cited TF2 and related framework papers to ground the article in current records. TinyFabulist Translation Framework (TF2) is an end-to-end, reproducible pipeline for low-cost, high-quality literary translation targeted at the English→Romanian pair. It is centered on the creation and open release of bilingual literary datasets, parameter-efficient fine-tuning of open-weight LLMs, and a narrative-aware evaluation protocol that combines corpus-level BLEU with a five-dimension LLM-based rubric. TF2 was introduced together with the datasets DS-TF2-EN-RO-15K and DS-TF2-EN-RO-3M and the fine-tuned models TF2-1B, TF2-4B, and TF2-12B, with full prompts, scripts, and evaluation assets released for reproducibility (Nadas et al., 9 Sep 2025).

1. Scope, motivation, and design objective

TF2 addresses a specific gap in machine translation research: the scarcity of open, genre-specific parallel corpora and fine-tuned open models for creative narrative translation in low-resource settings. Its stated scope is English→Romanian literary translation for moral fables, a domain characterized by short narratives, explicit morals, stylistic regularity, and sensitivity to fluency, coherence, and cultural adaptation. In contrast to general-purpose MT systems, TF2 is organized around a narrow literary genre and a cost-aware workflow (Nadas et al., 9 Sep 2025).

The framework builds on DS-TF1-EN-3M, described as the largest open collection of synthetic English fables, by extending that monolingual resource into bilingual English–Romanian data. Its novelty relative to TF1 is specified along four axes: bilingual EN–RO data, a multi-stage fine-tuning strategy, automated rubric-based evaluation tailored to literature, and a cost-aware pipeline. The reported emphasis is on parameter-efficient adaptation through LoRA and quantized deployment, with the explicit aim of approaching the quality of large proprietary systems while remaining open and accessible.

A recurrent misconception is to interpret TF2 as a universal MT framework. The reported contribution is narrower. The paper repeatedly situates TF2 in the domain of moral fables and English→Romanian literary translation, and it lists domain specificity as a limitation. The strongest claims therefore concern literary adequacy, fluency, and deployment cost within that task, rather than general multilingual translation performance across arbitrary genres or language pairs (Nadas et al., 9 Sep 2025).

2. Data assets, provenance, and staged corpus construction

TF2 organizes data creation into staged corpus construction. The two central bilingual datasets are DS-TF2-EN-RO-15K and DS-TF2-EN-RO-3M, both released under the MIT License. The first serves as the benchmark and fine-tuning corpus; the second is a large synthetic corpus created locally with the best TF2 model rather than external APIs (Nadas et al., 9 Sep 2025).

Resource	Size	Role
DS-TF1-EN-3M	3M English fables	Source pool
DS-TF2-EN-RO-15K	15,000 EN–RO pairs	Fine-tuning and evaluation
DS-TF2-EN-RO-3M	$\sim 3{,}000{,}000$ EN–RO pairs	Mass-translated large-scale corpus

DS-TF2-EN-RO-15K contains 15,000 English–Romanian pairs with splits of 12,000 train, 1,500 validation, and 1,500 test. The genre is short moral fables of 1–3 paragraphs, each ending with an explicit moral. The English sources are sampled from DS-TF1-EN-3M. Romanian targets are generated by the best-performing LLM from Stage 1, and in the reported experiments GPT-o3 emerges as the top reference translator; its outputs serve as “silver-standard” references.

Candidate systems for that reference-translator selection are scored on five rubric dimensions over 100 randomly sampled fables per model. The aggregation rule is given as

$$\text{Avg.\ Score}=\frac{\text{Accuracy}+\text{Fluency}+\text{Coherence}+\text{Style}+\text{Cultural/Pragmatic}}{5}.$$

The 15K corpus is serialized as JSONL with fields including fable, translated_fable, pipeline_stage, source_lang, target_lang, prompt_hash, llm_name, translation_model, and generation_timestamp. This provenance scheme is intended to support deduplication, traceability, and auditability.

DS-TF2-EN-RO-3M is then constructed by translating the entire DS-TF1-EN-3M English corpus using the best fine-tuned TF2 model or models from Stage 3. Its average length is reported as approximately 450 tokens per language, or approximately 900 per pair. Because this stage uses local TF2 models rather than paid APIs, it establishes the framework’s cost profile and its claim of scalable open generation (Nadas et al., 9 Sep 2025).

3. Model family and two-stage fine-tuning procedure

The best-performing TF2 configuration is TF2-12B, whose backbone is Gemma-3-12B-it. The paper also reports TF2-4B and TF2-1B variants. The defining training modification is a two-stage fine-tuning process: first, instruction tuning for domain adaptation to moral fables using the 15K EN–RO corpus; second, adapter compression through LoRA for efficient deployment (Nadas et al., 9 Sep 2025).

The Stage (i) instruction-tuning setup structures inputs with a standardized instruction such as “Translate the following fable from English to Romanian:”. Label masking is used so that only the target-side Romanian tokens contribute to the loss. In prose, the reported training objective is standard token-level cross-entropy over target tokens, with the English prompt and source provided as context but masked from loss. This architecture therefore treats literary translation as an instruction-following generation task with target-side supervision.

Stage (ii) injects LoRA adapters into both attention and MLP projections. The specified target modules are q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj. The reported configuration uses rank $r = 32$, scaling $\alpha = 32$, and dropout $p = 0.05$ on adapter paths. The backbone remains frozen while the low-rank matrices are trained, and the update is merged into the base weights at inference.

Optimization details are explicit. The framework uses AdamW, a cosine scheduler, gradient accumulation, and mixed precision in FP16 or bfloat16. Early stopping is based on validation loss. Tooling is reported as PyTorch 2.7.1, HF Transformers 4.54.0, PEFT 0.11, and vLLM 0.7.3 with PagedAttention and FlashAttention. Quantized deployment is integral rather than auxiliary: the models are exported as 8-bit variants, including W8A8 through llmcompressor, AWQ/GPTQ variants, and GGUF artifacts for lightweight inference.

The end-to-end pipeline is divided into four stages. S1 evaluates 13 candidate systems on 100 sampled fables and selects both the strongest reference translator and the open backbone. S2 creates the 15K parallel corpus. S3 fine-tunes the open models with masked target loss and LoRA. S4 uses the best fine-tuned TF2 model to mass-translate the 3M English corpus into Romanian. This sequencing is central to the framework’s reproducibility claim because each stage publishes artifacts rather than only final checkpoints (Nadas et al., 9 Sep 2025).

4. Evaluation methodology and reported empirical performance

TF2 combines corpus-level BLEU with a five-dimension LLM-based rubric. BLEU is reported on a normalized $0$–$1$ scale, where $0.0926$ corresponds to $9.26$ points. The standard BLEU definition is given as

$$\mathrm{BLEU} \;=\; \mathrm{BP} \cdot \exp\!\left( \sum_{n=1}^{N} w_n \log p_n \right),$$

with

$$\text{Avg.\ Score}=\frac{\text{Accuracy}+\text{Fluency}+\text{Coherence}+\text{Style}+\text{Cultural/Pragmatic}}{5}.$$0

where $$\text{Avg.\ Score}=\frac{\text{Accuracy}+\text{Fluency}+\text{Coherence}+\text{Style}+\text{Cultural/Pragmatic}}{5}.$$1 is candidate length, $$\text{Avg.\ Score}=\frac{\text{Accuracy}+\text{Fluency}+\text{Coherence}+\text{Style}+\text{Cultural/Pragmatic}}{5}.$$2 is reference length, $$\text{Avg.\ Score}=\frac{\text{Accuracy}+\text{Fluency}+\text{Coherence}+\text{Style}+\text{Cultural/Pragmatic}}{5}.$$3 are modified n-gram precisions, and $$\text{Avg.\ Score}=\frac{\text{Accuracy}+\text{Fluency}+\text{Coherence}+\text{Style}+\text{Cultural/Pragmatic}}{5}.$$4 are weights. The rubric dimensions are Accuracy, Fluency, Coherence, Style, and Cultural/Pragmatic adaptation, each scored from 1 to 5 using a fixed evaluation prompt (Nadas et al., 9 Sep 2025).

A central methodological point is that TF2 treats BLEU as insufficient for literary translation. The paper’s evaluation design therefore addresses a common criticism of literary MT benchmarking: paraphrastic adequacy and stylistic quality are not well captured by n-gram overlap. This does not eliminate evaluator bias, but it changes the target of evaluation from sentence-level lexical fidelity alone to narrative adequacy.

The reported results place TF2-12B near the top proprietary systems on the rubric. GPT-o3 scores 4.92 average, GPT-4.1 scores 4.89, and Gemma-3-12B-it scores 4.43. TF2-12B reaches 4.83 at $$\text{Avg.\ Score}=\frac{\text{Accuracy}+\text{Fluency}+\text{Coherence}+\text{Style}+\text{Cultural/Pragmatic}}{5}.$$5, while TF2-12B-quant reaches 4.82 at $$\text{Avg.\ Score}=\frac{\text{Accuracy}+\text{Fluency}+\text{Coherence}+\text{Style}+\text{Cultural/Pragmatic}}{5}.$$6. TF2-4B reaches up to 4.74, and TF2-1B reaches approximately 3.75. Under a cross-family robustness check with Grok-3-mini, TF2-12B-Q scores 4.90 versus 4.92 for GPT-o3, narrowing the gap to 0.02.

The BLEU results are more heterogeneous. TF2-12B improves the Gemma-3-12B-it BLEU score from 0.0214 to 0.0926. TF2-4B improves from 3.81 rubric score to 4.74 and reports BLEU up to 0.1290 for the quantized $$\text{Avg.\ Score}=\frac{\text{Accuracy}+\text{Fluency}+\text{Coherence}+\text{Style}+\text{Cultural/Pragmatic}}{5}.$$7 variant. TF2-1B improves rubric quality substantially relative to Gemma-3-1B-it, but its BLEU behavior depends on variant: 0.0543 for the standard model and 0.2180 for the distilled variant. No statistical significance tests are reported.

The paper also reports a temperature ablation. Lower temperatures, specifically $$\text{Avg.\ Score}=\frac{\text{Accuracy}+\text{Fluency}+\text{Coherence}+\text{Style}+\text{Cultural/Pragmatic}}{5}.$$8 and $$\text{Avg.\ Score}=\frac{\text{Accuracy}+\text{Fluency}+\text{Coherence}+\text{Style}+\text{Cultural/Pragmatic}}{5}.$$9, produce the best scores, while $r = 32$0 degrades Accuracy and Cultural/Pragmatic adaptation most strongly. The default for mass translation is therefore $r = 32$1. Qualitative analysis attributes TF2-12B’s gains over the untuned backbone to more stable species fidelity, more idiomatic Romanian, and more coherent moral phrasing, while the untuned Gemma-3-12B-it is reported to exhibit species corruption, awkward phrasing, and occasional grammar errors (Nadas et al., 9 Sep 2025).

5. Efficiency, deployment profile, reproducibility, and limitations

One of TF2’s main claims is cost efficiency. The reported mid-case token budget for translating 3M fables is approximately 2.7B tokens. The cost model is written as

$r = 32$2

Using this accounting, the paper estimates approximately $r = 32$324,300 for GPT-o3 with medium reasoning, approximately $r = 32$4350 for the fine-tuned TF2 pipeline. The corresponding summary claim is that TF2 reduces generation costs by 97–99% relative to the listed baselines (Nadas et al., 9 Sep 2025).

The deployment profile is equally specific. Fine-tuning is reported as feasible on modest hardware for the smaller variants: TF2-1B on $r = 32$5L40S in approximately 1 hour, TF2-4B on $r = 32$6L40S in approximately 2–3 hours, and TF2-12B on $r = 32$7H100 in approximately 2 hours. Large-scale inference for 3M fables uses an $r = 32$8H100 cluster for approximately 31 hours through vLLM with PagedAttention and FlashAttention. GGUF exports are provided for CPU/GPU lightweight serving.

Reproducibility is treated as a first-class design goal. The framework releases datasets, LoRA-adapted checkpoints, quantized variants, evaluation JSON prompts, scoring scripts, data-generation code, quantization/export tooling, and metadata fields such as prompt_hash. The stated environment includes Manjaro Linux 6.12 and versioned library dependencies. Specific random seeds are not listed, but the paper argues that the dataset metadata supports auditing.

The main limitations are explicit. The references are synthetic rather than human translations, so BLEU may penalize acceptable paraphrases. A single LLM-as-judge can introduce stylistic bias, although a cross-family judge check reduces this concern without eliminating it. The domain is fables, so transfer to poetry, historical prose, or more idiomatic genres is not established. The judged evaluation sample is 100 items per model, even though BLEU is computed on the full test set. These points constrain how the near-parity claim should be interpreted: it is a strong result for a narrowly defined literary task, not a universal statement about open models across literary translation as a whole (Nadas et al., 9 Sep 2025).

6. TF2 within adjacent translation research and proposed extension paths

The TF2 paper is specific to English→Romanian literary translation, but the surrounding implementation-oriented summaries position TF2 as compatible with several broader MT paradigms. One such direction is hierarchical multilingual translation. The HNMT framework uses a typological language family tree to determine which encoder and decoder layers are language-specific, family-shared, or universal, with a reported average BLEU of 6.25 versus 5.19 for a many-to-many bilingual baseline across 41 languages and 758 translation tasks. The stated rationale is to encourage positive transfer among typologically similar languages while reducing negative transfer among distant languages. In TF2-oriented guidance, this is presented as a way to extend the framework beyond a single language pair while preserving controlled parameter sharing (Azpiazu et al., 2020).

A second extension path concerns text translation inside images. AnyTrans defines TATI, “Translate AnyText in the Image,” as translation plus style-preserving reinsertion of translated text into the original image. Its TF2-oriented module design includes PP-OCR for detection and recognition, tagged-sequence aggregation across boxes, Qwen-family LLMs or VLMs for global-context translation, StrokeNet for fine-grained erasure, AnyText for multilingual text editing, and Stable Diffusion Inpainting with ControlNet or T2I-Adapter for harmonization. Reported results include I2T BLEU up to 48.7 for zh→en and I2I BLEU up to 31.9, as well as a 2.9 I2I BLEU drop on zh→en when the box-resize step is removed. This suggests a plausible TF2 expansion from text-only literary translation to multimodal translation-and-fusion workflows, although that expansion is not part of the core TF2 paper (Qian et al., 2024).

A third research connection is the “train global, tailor local” strategy for endangered languages. That line of work reports that adapting a large multilingual model to a specific limited text or domain first, and only then specializing it to a new severely low-resource language with about 1,000 seed sentences, improves average chrF from 21.9 to 50.7. The scheduling principle is domain-first, language-second, often using active learning for seed selection and multi-stage schedules such as $r = 32$9. This is not the TF2 training regime described for English→Romanian fables, but it is directly relevant as a methodological template if TF2 is adapted to other low-resource literary or humanitarian domains (Zhou et al., 2023).

A fourth connection is the Factored Transformer, which enriches a Transformer with linguistic factors such as lemmas, PoS tags, morphology, dependency relations, and semantic synsets. The best configuration in that work is a 1-encoder summation variant with lemmas, yielding 37.46 BLEU versus 36.67 on IWSLT16 German→English and 4.27 versus 3.06 on FLoRes English→Nepali. TF2-oriented notes recommend starting with embedding-level factor fusion using lemmas and summation rather than concatenation. A plausible implication is that such factor injection could be useful if TF2 is extended toward morphologically richer target languages or settings in which literary adequacy depends on stronger lexical normalization (Armengol-Estapé et al., 2020).

Taken together, these adjacent lines of work frame TF2 not only as a released English→Romanian literary translation pipeline, but also as a potential host framework for hierarchical multilingual sharing, text-in-image translation, domain-first low-resource adaptation, and factor-augmented Transformer modeling. The factual record, however, remains clear: the core reported TF2 contribution is the open, reproducible literary translation pipeline for English→Romanian moral fables, with released datasets, LoRA-adapted models, rubric-based evaluation, and a sharply reduced cost profile (Nadas et al., 9 Sep 2025).