LFM2 Technical Report

Abstract: We present LFM2, a family of Liquid Foundation Models designed for efficient on-device deployment and strong task capabilities. Using hardware-in-the-loop architecture search under edge latency and memory constraints, we obtain a compact hybrid backbone that combines gated short convolutions with a small number of grouped query attention blocks, delivering up to 2x faster prefill and decode on CPUs compared to similarly sized models. The LFM2 family covers 350M-8.3B parameters, including dense models (350M, 700M, 1.2B, 2.6B) and a mixture-of-experts variant (8.3B total, 1.5B active), all with 32K context length. LFM2's training pipeline includes a tempered, decoupled Top-K knowledge distillation objective that avoids support mismatch; curriculum learning with difficulty-ordered data; and a three-stage post-training recipe of supervised fine-tuning, length-normalized preference optimization, and model merging. Pre-trained on 10-12T tokens, LFM2 models achieve strong results across diverse benchmarks; for example, LFM2-2.6B reaches 79.56% on IFEval and 82.41% on GSM8K. We further build multimodal and retrieval variants: LFM2-VL for vision-language tasks, LFM2-Audio for speech, and LFM2-ColBERT for retrieval. LFM2-VL supports tunable accuracy-latency tradeoffs via token-efficient visual processing, while LFM2-Audio separates audio input and output pathways to enable real-time speech-to-speech interaction competitive with models 3x larger. LFM2-ColBERT provides a low-latency encoder for queries and documents, enabling high-performance retrieval across multiple languages. All models are released with open weights and deployment packages for ExecuTorch, llama.cpp, and vLLM, making LFM2 a practical base for edge applications that need fast, memory-efficient inference and strong task capabilities.

• The paper demonstrates a co-designed hybrid architecture that combines gated convolutions with selective attention to optimize performance on resource-constrained devices.
• It introduces a novel decoupled Top-K knowledge distillation strategy that ensures efficient training and robust cross-lingual transfer with minimal overhead.
• The report validates significant improvements in latency, throughput, and multimodal performance, establishing a new baseline for practical, edge-focused foundation models.

LFM2: An Edge-First Foundation Model Suite

Introduction and Motivation

"LFM2 Technical Report" (2511.23404) introduces LFM2, a comprehensive family of "Liquid Foundation Models" (LFMs) co-designed for efficient on-device deployment and high downstream performance. The driving motivation is to fill the capability-efficiency gap for small, edge-focused models, a regime under-served by prior open research focused primarily on datacenter-scale or larger models. LFM2 addresses stringent latency, memory, and energy constraints and supports a broad context window (32K tokens), multimodality (text, vision, audio), and information retrieval, covering 350M–8.3B parameter scales.

Figure 1: The LFM2 portfolio spans multiple scales, modalities, and deployment targets, with a taxonomy from the architecture level to highly specialized downstream domains.

Architecture: Minimal Hybrid for Edge Deployment

LFM2 employs a hybrid backbone, combining gated short convolution blocks and a minority of grouped-query attention layers (GQA), selected via hardware-in-the-loop search optimizing for the latency–quality–memory Pareto frontier on CPUs and NPUs. The Mixture-of-Experts (MoE) variant activates only a sparse subset of SwiGLU experts per token—yielding higher quality per active parameter and demonstrating 3–4B-class accuracy at 1.5B active parameters.

Figure 2: LFM2 architecture consists of interleaved gated short convolution blocks, GQA layers, and SwiGLU-based position-wise feed-forward modules; for the MoE variant, experts are implemented as SwiGLU blocks.

The search systematically excluded the inclusion of SSM or linear-attention operators (Mamba, S4, etc.), as these did not yield improvements in either quality or device-level efficiency once a few attention blocks were present. Instead, gated short convolutions dominate, responsible for efficient local mixing and cache behavior.

This edge-first approach is validated by strong profiling: on mobile and laptop-class CPUs, LFM2 yields up to 2× improvements in prefill and decode throughput relative to baselines such as Qwen3, Gemma 3, and Granite 4.0, across parameter scales, all while maintaining or improving accuracy on hard downstream evaluations.

Training Methodology: Data, Distillation, and Post-Training Innovations

Large-scale multilingual and code-heavy corpora (10–12T tokens for each model, 32K context) form the base. Pre-training integrates a novel decoupled, tempered Top-K knowledge distillation objective: rather than standard KL, this objective avoids support mismatch and instability by decomposing over Top-K logits, weighting KL over the teacher’s Top-K set and matching total probability mass, applying temperature only within the conditional Top-K KL. This enables cost-effective transfer from stronger teachers with minimal logit storage or bandwidth overhead.

Post-training leverages curriculum ordering (via multi-model ensembles to estimate example difficulty), supervised fine-tuning on a high-quality, semantically and stylistically filtered mixture, direct preference optimization with length normalization (through generalized DPO/APO losses), and a comprehensive model merging pipeline employing a range of parameter-space techniques (model soups, TIES, DARE, DELLA). The aggregate strengthens instruction following, multi-turn conversational robustness, RAG, and multilingual transfer.

Modal and Domain Extensions

Vision-Language: LFM2-VL

Augmentation of the language backbone with SigLIP2 encoders (NaFlex variant for flexible input resolutions) enables VLMs at 450M–3B scale, using lightweight PixelUnshuffle+MLP connectors for efficient visual tokenization and competitive accuracy/latency tradeoff across device constraints.

Audio: LFM2-Audio

The LFM2 backbone is extended with a continuous audio encoder (FastConformer-based) and a discrete audio output path (Mimi RVQ codes, with lightweight detokenizer), supporting real-time speech-to-speech and highly competitive ASR—all at edge inference speeds. Importantly, the architecture distinguishes between continuous input representations and codebook token outputs, which is more robust relative to pure tokenized audio approaches.

Retrieval: LFM2-ColBERT

A late-interaction retriever (ColBERT-style) is instantiated at 350M scale with additional projection layers, trained via knowledge distillation from strong cross-encoders. This yields first-class NDCG@10 on multilingual/cross-lingual retrieval (e.g., 0.661 mono-lingual, 0.490–0.564 in Arabic, Japanese, Korean, and strong cross-lingual generalization), at throughput matching much smaller baselines while retaining contextual expressivity.

Numerical Results and Claims

LFM2-2.6B achieves 79.56% on IFEval and 82.41% on GSM8K. LFM2-8B-A1B achieves 84.38% GSM8K, 74.2% on MATH 500, and 3–4B-class generalization at 1.5B decode cost. In multilingual transfer, LFM2-1.2B retains 94.4% of English GSM8K accuracy on MGSM, indicating robust cross-lingual training signal transfer. Across benchmarks, LFM2 models consistently lead or are on the Pareto frontier for accuracy versus throughput on mobile CPUs.

LFM2-VL-3B achieves 76.55% (SEEDBench), 71.37% (RealWorldQA), and 51.83% (MM-IFEval), outperforming or matching larger competitors, and demonstrates graceful degradation under aggressive vision token compression—enabling tunable accuracy–latency profiles. In multilingual VLM evaluation (translated MMBench/MMMB), LFM2-VL-3B exhibits 75.84 and 81.52 averages, respectively, outperforming VLMs below 4B parameters.

LFM2-Audio-1.5B scores 3.78 (AlpacaEval, VoiceBench), 51.2 (BBH), and attains ASR benchmarks (e.g., 2.03 WER on LibriSpeech Clean) on par with 3–5B baselines. LFM2-ColBERT-350M attains 0.661 NDCG@10 average on multilingual NanoBEIR, with competitive or better retrieval encoding throughput compared to baselines at <50% parameter count.

Practical and Theoretical Implications

LFM2 concretely demonstrates that edge-first foundation models, with co-designed architectures and training protocols, can approach or surpass larger models in several critical downstream tasks when deployed in latency/memory-restricted settings. The minimal hybrid design also provides theoretical and empirical evidence that for small and efficient models, the gains promoted by complex SSM or linear attention variants are predominantly realized by their short-range convolutional parts and a handful of attention layers—contradicting recent arguments for SSM-rich architectures in efficiency-critical settings.

From a deployment standpoint, LFM2's open weights, support for leading edge inference runtimes (ExecuTorch, llama.cpp, vLLM), and aggressive quantization make it a practical baseline for industry and open communities deploying LLMs, VLMs, and speech models directly on resource-constrained hardware without large-batch acceleration or remote datacenters.

The decoupled Top-K distillation objective generalizes and stabilizes knowledge transfer for compact models, and the length-normalized preference alignment family extends DPO/APO to be more robust for length-variant tasks and small models. The model merging pipeline, leveraging parameter arithmetic and interference-reducing algorithms, is shown to be effective in the low-capacity regime, where catastrophic forgetting and domain interference present severe bottlenecks.

Limitations and Future Directions

LFM2 remains parameter-limited relative to frontier models and is tuned for single-batch, on-device settings. Its architecture and quantization profiles may not be optimal for all NPUs or large-batch accelerators, and its coverage of non-speech audio, low-resource languages, and fine-grained visual grounding is incomplete. LFM2-VL and LFM2-Audio are not yet trained for video or more complex multimodal grounding. LFM2-ColBERT, while robust in European cross-lingual scenarios, could be improved with targeted fine-tuning for challenging language pairs.

Future work will extend architecture optimization across broader hardware and accelerator classes, explore lightweight video and multimodal extensions, improve few-shot composition and generalization tasks, and investigate in-depth hardware–model co-evolution, especially as mobile NPU/TPU stacks mature and expand.

Conclusion

LFM2 marks a significant advance in co-designing model architectures, training regimes, and deployment recipes for edge-centric, high-capability foundation models. Its family of models covers dense, mixture-of-experts, vision–language, audio, and retrieval, all released with open weights and device-optimized runtimes. Strong benchmarking and efficiency substantiate LFM2 as a technical reference for small, practical, real-world LLM deployment, and its modular extensions demonstrate that competitive multimodal and information retrieval capabilities are realizable on-device within tight compute and memory budgets. The released tools and recipes will likely serve as strong baselines and experimental platforms for both applied practitioners and methodologically focused research in efficient, accessible AI.