Soofi S: A Sovereign Foundation Model for German and English
This presentation examines Soofi S 30B-A3B, a fully open-source, bilingual foundation model optimized for German and English. The talk explores how deliberate data curation, architectural efficiency through Mixture-of-Experts and Mamba-Transformer hybrid design, and unprecedented transparency in training recipes enable high performance on technical tasks while maintaining exceptional inference throughput. We highlight the model's contributions to sovereign AI standards and its position at the frontier of reproducible, auditable foundation model research.Script
Most foundation models are black boxes trained on undisclosed data, making them unsuitable for organizations that need transparency and sovereignty. Soofi S breaks that pattern by delivering a fully auditable, bilingual model built for both German and English with complete data provenance and open training recipes.
The architecture combines 52 layers interleaving Mamba-2 recurrence, sparse Mixture-of-Experts routing, and selective Grouped-Query Attention. This design activates only 3.2 billion parameters per forward pass from a total capacity of 31.6 billion, keeping memory and compute costs remarkably low while preserving model expressiveness.
Training followed a three-phase curriculum spanning 27 trillion tokens: broad multilingual pretraining, high-density annealing with code and math emphasis, and long-context extension to 1 million tokens. German content was deliberately upweighted from 7 percent in phase one to over 15 percent in annealing, addressing the chronic underrepresentation of non-English data in foundation models.
On English aggregate benchmarks, Soofi S outperforms every fully open model, including dense competitors with far more active parameters. But the true standout is German performance: Soofi S achieves unmatched scores across regional technical benchmarks, German code generation, and grade-school math, widening margins that no other open-source model approaches.
Because only 6 of its 52 layers maintain a key-value cache, Soofi S sustains nearly flat throughput as context grows from 4,000 to 256,000 tokens. Dense baselines collapse under memory pressure at long context, but Soofi S delivers over 9 times higher tokens per second per GPU at 40,000 tokens with batch size 32.
By releasing not just weights but full data accounting, intermediate checkpoints, and complete training recipes, Soofi S establishes a reproducibility standard for sovereign AI. If transparency and efficiency at long context matter to your work, explore this model and others like it at EmergentMind.com.
