Meno-Lite-0.1: Bilingual RAG Model
- The paper demonstrates that deliberate bilingual domain adaptation and skill-focused training enable a 7B-parameter model to achieve robust RAG, extraction, and multi-hop reasoning despite its limited scale.
- Meno-Lite-0.1 is a bilingual model defined by enhanced tokenization efficiency and tested on ultra-long contexts, optimizing retrieval-augmented generation with clear context grounding.
- Its integration in a heterogeneous ensemble highlights a cost-effective, specialized contribution that boosts overall system faithfulness by abstaining on unsupported queries.
Searching arXiv for papers mentioning Meno-Lite-0.1 and related Meno/MENO systems to ground the article. arXiv search query: "Meno-Lite-0.1 OR Meno and Friends OR Ablation and the Meno OR MENO" Meno-Lite-0.1 is a 7B-parameter open-weight bilingual LLM, introduced as a domain-adapted component of RaguTeam’s SemEval-2026 Task 8 system for faithful multi-turn response generation with reference passages. It is derived from Qwen2.5-7B-Instruct through a sequence of Russian-focused adaptations and subsequent skill-oriented training, and it is deliberately optimized for context-grounded workloads—retrieval-augmented generation, information extraction, multi-hop reasoning, and knowledge-graph construction—rather than broad parametric factual recall. In the reported system, Meno-Lite-0.1 serves as the small, inexpensive member of a heterogeneous seven-model ensemble whose outputs are ranked by a GPT-4o-mini judge for faithfulness (Bondarenko et al., 6 May 2026).
1. Definition and research position
Meno-Lite-0.1 is characterized as a bilingual Russian–English model with strong Russian performance and solid English capability on RAG-style tasks. Its stated design hypothesis is that large parameter counts are primarily needed for world knowledge, whereas “language skills” such as comprehension, extraction, inference, summarization, grounding, and abstention can reach a usable plateau in the 7B regime if they are deliberately cultivated. The model is therefore positioned as an empirical test of skill-focused domain adaptation under tight deployment constraints, including suitability for a single consumer GPU (Bondarenko et al., 6 May 2026).
Within the SemEval-2026 Task 8 setting, the model’s function is not to dominate the ensemble by raw aggregate score. Instead, it contributes candidate generations with a distinctive behavioral profile: extraction-oriented, grounding-sensitive, and relatively conservative on unsupported queries. This profile makes it useful as a diversity source in a judge-orchestrated ensemble whose overall design emphasizes heterogeneity in model families, scales, training pipelines, and prompting strategies (Bondarenko et al., 6 May 2026).
A central implication of this positioning is that Meno-Lite-0.1 should be understood less as a general-purpose frontier model than as a specialized RAG-centric artifact. Its importance in the literature derives from the conjunction of three properties: open weights, explicit bilingual domain adaptation, and nontrivial competitiveness on context-grounded benchmarks despite the 7B scale (Bondarenko et al., 6 May 2026).
2. Lineage and technical profile
The paper gives a precise lineage: This chain places the model within the Qwen family while making clear that Meno-Lite-0.1 is not a direct off-the-shelf derivative of the base backbone, but the endpoint of successive linguistic and task-oriented adaptations (Bondarenko et al., 6 May 2026).
The reported parameter count is 7B. The model inherits Ruadapt’s extended vocabulary, with tokenizer efficiency reported as 3.77 chars/token on Russian and 4.13 chars/token on English. The Russian figure is described as a 47% improvement over the original Qwen2.5 value of 2.57 chars/token, and the English figure is described as on par with the best models in its class. The intended systems consequence is shorter sequences for the same textual content, with lower compute and latency in Russian-heavy workloads (Bondarenko et al., 6 May 2026).
Long-context behavior is evaluated up to 128k tokens. On LIBRA passkey retrieval, Meno-Lite-0.1 scores 1.0 at 4k–64k and 0.98 at 128k. At the same time, the paper explicitly notes that complex reasoning degrades at contexts beyond 32K tokens, which places a practical distinction between retrieval robustness and higher-order compositional reasoning under ultra-long contexts (Bondarenko et al., 6 May 2026).
No specialized heads, adapters, LoRA components, PEFT methods, or quantization-aware finetuning are described. Training is summarized as continued pretraining plus supervised fine-tuning, and the paper does not provide a separate architecture diagram beyond the inherited Qwen2.5 backbone assumptions (Bondarenko et al., 6 May 2026).
3. Training regimen and domain adaptation
The training program comprises a continued pretraining stage and a supervised fine-tuning stage. Continued pretraining uses 1.3B tokens in a balanced Russian–English corpus. The reported data sources are FineWeb-Edu for English, a RuLM subset filtered for FineWeb-Edu similarity using gte-multilingual-base embeddings, Russian educational PDFs denoted FinePDFs-edu, and RuREBus unlabeled text from Russian e-government documents. The substantial English share is explicitly intended to prevent catastrophic forgetting of English-language capability during Russian-focused adaptation (Bondarenko et al., 6 May 2026).
The supervised fine-tuning stage uses 50M tokens and is designed to reinforce extraction, normalization, summarization, and multi-hop QA. Russian SFT sources include NEREL instructions augmented with GPT-4o-mini-generated entity definitions and normalizations, LightRAG query logs from Habr, and manually created self-cognition and alignment instructions in Russian. English SFT sources include MultiHopRAG dialogues and MTRAGEval training dialogues, placing the model in direct contact with the target task distribution of the shared task (Bondarenko et al., 6 May 2026).
This training setup establishes a clear bias toward grounded generation. The intended behaviors are answer-only-from-documents, explicit abstention when support is absent, and effective use of dialogue history for cross-turn reference resolution. The paper repeatedly contrasts this emphasis with broad factual memorization and presents weaker world-knowledge performance as a deliberate trade-off rather than an accidental deficiency (Bondarenko et al., 6 May 2026).
The article’s technical specificity ends at the dataset and stage level. Learning rates, optimizer choice, batch size, epoch counts, warmup schedules, and training hardware for Meno-Lite-0.1 itself are not disclosed. This absence is significant for replication: the model’s data composition is documented, but its training recipe is not fully reconstructible from the paper alone (Bondarenko et al., 6 May 2026).
4. Prompting and role in the judge-orchestrated ensemble
RaguTeam’s winning Task B submission is a heterogeneous ensemble of seven LLMs with two prompting groups and GPT-4o-mini as the ranking judge. Group 1 uses a carefully engineered system prompt and contains Gemini-3-Pro-Preview, GLM-4.6, Llama-3.3-70B-Instruct, and Qwen3-235B-A22B. Group 2 uses a simpler system prompt plus four category-aware few-shot exemplars and contains Claude 4.5 Haiku, Qwen2.5-32B-Instruct, and Meno-Lite-0.1 (Bondarenko et al., 6 May 2026).
For Group 2, exemplars are selected by embedding all training instances with gte-multilingual-base and choosing medoids: one from the empty-context category, one from the empty-dialogue-history category, and two from the non-empty-context-and-history category. The few-shot prompt is designed to demonstrate several target behaviors: declining to answer when documents contain no relevant information, grounding yes/no responses in specific document snippets, and resolving anaphora through dialogue history, such as mapping “it” back to a previously established referent (Bondarenko et al., 6 May 2026).
Meno-Lite-0.1 participates by generating one candidate per instance under this few-shot regime. GPT-4o-mini then scores all candidates in the interval for faithfulness with respect to the retrieved passages, and the highest-scoring answer becomes the final system output. The model is selected in only 2 of 424 evaluated instances, but those selected instances have high metric scores, with HM around 0.707–0.72. This establishes a pattern of low selection frequency but genuine complementarity: the model is rarely the best candidate, yet when it is selected, it is often because it remains faithful in cases where stronger models deviate (Bondarenko et al., 6 May 2026).
This role is consistent with the ensemble’s broader philosophy. Diversity is treated as operationally useful even when the marginal contribution of a particular model is small in aggregate. Meno-Lite-0.1 functions as a low-cost generator of grounded, abstention-aware alternatives that a judge can exploit opportunistically (Bondarenko et al., 6 May 2026).
5. Empirical performance and cost–performance profile
On MTRAGEval Task B as a standalone generator, Meno-Lite-0.1 records conditioned metrics of RB_\text{idk}= 0.50_\ell , RL[0,1]$0 $[0,1]$1, and HM$[0,1]$2 $[0,1]$3. By category, its conditioned HM$[0,1]$4 values are as follows (Bondarenko et al., 6 May 2026):
| Category | HM$[0,1]$5 |
|---|---|
| Answerable (ANS) | 0.681 |
| Unanswerable (UNA) | 0.540 |
| Underspecified (UND) | 0.320 |
| Partially answerable (PAR) | 0.490 |
These figures show a markedly asymmetric profile. On answerable instances, the model reaches 0.681 HM$[0,1]$6 and is reported to exceed the organizer baseline. On unanswerable instances, the score of 0.540 corresponds to a 54% IDK rate, substantially better calibrated than Qwen2.5-32B-Instruct, whose corresponding figure is 0.237. On underspecified queries, however, the score drops to 0.320, aligning with the paper’s broader observation that most models struggle to ask for clarification when pragmatic context is insufficient (Bondarenko et al., 6 May 2026).
Inside the full ensemble, the official MTRAGEval judges assign the system a conditioned HM$[0,1]$7 of 0.7827, compared with 0.6390 for the strongest baseline, gpt-oss-120b. Meno-Lite-0.1’s direct numerical contribution to that gain is small because of its low selection count, and removal ablations change answerable and underspecified HM$[0,1]$8 by only 0.2 and 0.4 percentage points, respectively. The paper therefore treats its main value not as dominant ensemble throughput but as a cost-effective standalone model and a useful domain-adaptation case study (Bondarenko et al., 6 May 2026).
The cost–performance argument becomes stronger on Russian-oriented external benchmarks. On MERA, Meno-Lite-0.1 obtains an overall score of 0.555, matching Llama-3.3-70B-Instruct at 0.555 and substantially improving over the base Qwen2.5-7B-Instruct at 0.482. On NEREL-bench, it reaches HM $[0,1]$9, outperforming Qwen2.5-32B-Instruct at 0.416 and Gemma-3-27B-IT at 0.396. On LIBRA, it achieves passkey retrieval of 1.0 from 4k to 64k and 0.98 at 128k, scores 0.484 on LibrusecMHQA, and records 0.542 on ruQasper at 8k context, reported as the highest among all tested models. These results support the paper’s narrower thesis that long-context grounding, structured extraction, and multi-hop reasoning need not require large-scale parametric knowledge when the training objective is aligned with context-grounded use cases (Bondarenko et al., 6 May 2026).
6. Limitations, reproducibility, and nomenclatural ambiguity
The paper states several limitations explicitly. Meno-Lite-0.1 is not intended for broad world-knowledge tasks without retrieval, a point reflected in its lower CheGeKa scores on MERA. Its function-calling performance on BFCL Russian is 58.9%, versus 76.1% for Qwen2.5-7B-Instruct. Complex reasoning degrades beyond 32K tokens even though retrieval remains strong up to 128k. On MTRAGEval, it trails the strongest large models on answerable questions and remains weak on underspecified turns, where the broader system analysis suggests that asking clarification questions is still inadequately learned (Bondarenko et al., 6 May 2026).
Reproducibility is partial. The model is described as open weight and is reported as available on Hugging Face at https://huggingface.co/bond005/meno-lite-0.1, while system code is reported as available at https://github.com/RaguTeam/ragu_mtrag_semeval. The repository is said to contain the inference pipeline, prompt templates, vLLM setups for open models, and judging scripts with GPT-4o-mini. By contrast, full training reproduction is not supported by the paper: hyperparameters, optimizer details, and training scripts for Meno-Lite-0.1 are not fully disclosed (Bondarenko et al., 6 May 2026).
The name also requires disambiguation. A plausible secondary usage appears in “Ablation and the Meno: Tools for Empirical Metamathematics,” which describes a simple Lean 4 autoformalizer named Meno; that paper does not explicitly use the string “Meno-Lite-0.1,” but its authors’ described system is characterized as a small-scale, early-stage prototype, making the label an interpretive shorthand rather than an official release name (Fan et al., 24 Apr 2026). Separate papers titled “MENO” introduce “MeanFlow-Enhanced Neural Operators for Dynamical Systems” and “Hybrid Matrix Exponential-based Neural Operator for Stiff ODEs,” both of which are unrelated scientific machine learning frameworks sharing only the acronymic surface form (Yang et al., 8 Apr 2026, Zanardi et al., 18 Jul 2025).
Taken together, the literature establishes Meno-Lite-0.1 primarily as a specialized 7B bilingual RAG model rather than a general family name. Its significance lies in demonstrating that targeted bilingual adaptation, efficient tokenization, and skill-focused supervised data can make a small open model materially competitive on grounded generation, structured extraction, and abstention-sensitive workloads, even when much larger models remain superior as general-purpose ensemble workhorses (Bondarenko et al., 6 May 2026).