EdgeRunner 20B: Military-Grade Edge LLM

• EdgeRunner 20B is a domain-specialized large language model built on the GPT-OSS-20B architecture, optimized for military tasks with quantization for efficient edge deployment.
• The model is fine-tuned on 1.6 million high-quality military Q-A records using a rigorous data curation pipeline to ensure secure and accurate inference in air-gapped settings.
• Performance on specialized military benchmarks shows that EdgeRunner 20B can match or exceed GPT-5 in key areas, offering a viable, cost-effective solution for mission-critical applications.

EdgeRunner 20B is a domain-specialized LLM derived from the gpt-oss-20b architecture, engineered to achieve military task parity with GPT-5 while supporting air-gapped edge deployment. Optimized through instruction fine-tuning on a corpus of 1.6 million high-quality, military-specific question–answer records, EdgeRunner 20B integrates post-training quantization and a rigorous data curation pipeline to deliver efficient, accurate, and secure inference suitable for data-sensitive, disconnected environments (Fitzgerald et al., 30 Oct 2025).

1. Model Architecture and Quantization

EdgeRunner 20B employs the GPT-OSS-20B backbone, a 20-billion-parameter, decoder-only transformer, retaining the original number of layers, hidden size, and attention heads. Fine-tuning was conducted in bfloat16 precision, followed by post-training weight-only quantization to the MXFP4 format using Nvidia’s convert_oai_mxfp4_weight_only.py. This approach mirrors the expert-layer quantization protocol of the parent model and is designed to maximize efficiency on heterogeneous hardware while minimizing the memory and compute footprint.

No structural modifications were made to the model architecture, thereby ensuring compatibility with existing open-source transformer implementations and enabling deployment on commodity and specialized edge hardware.

2. Fine-Tuning Pipeline, Data Curation, and Objectives

The fine-tuning procedure utilized a standard autoregressive cross-entropy loss:

$$L(\theta) = -\sum_{(x, y) \in D} \sum_{t=1}^{T} \log p_\theta(y_t | x, y_{<t})$$

where $D$ is the curated instruction-tuning dataset. The pipeline comprised several automated stages:

1. Ingestion and chunking of documents with the vLLM engine.
2. Generation of high-quality summaries for each chunk via model-in-the-loop summarization.
3. Synthesis of diverse question–answer pairs conditioned on summaries.
4. Automated triage by a separate model, classifying output as PASS, FIX, or FAIL. Pairs marked as FIX were auto-rewritten; FAIL pairs were discarded.
5. Aggregation of PASS and FIX pairs into the final training corpus.

The dataset comprises 1.6 million Q-A records sourced from both publicly available and proprietary military documentation, spanning Army, Joint Staff, DoD, and logistics material. Preprocessing included context relevance filtering, deduplication through exact and fuzzy match removals, and exclusion of all data overlapping with evaluation sets. No additional synthetic data augmentation was performed beyond the Q-A generation pipeline.

Training was distributed using a SLURM-based pipeline leveraging PyTorch c10d and NCCL over InfiniBand, with the Axolotl library orchestrating supervised fine-tuning (SFT) on global batch sizes of 1,024–1,536 and sequence lengths of 4,096 tokens.

3. Military Benchmark Design and Performance

EdgeRunner 20B introduces four new military Q-A benchmarks, each composed of freeform open-ended queries with SME (subject matter expert) vetting and LLM-graded outputs using a W8A8-quantized Atla Selene 1 model:

Test Set	Classification	Size	Description
combat-arms	Silver	180	Infantry/combat-arms SME-vetted Q&A pairs
combat-medic	Silver	446	Battlefield medicine SME-vetted Q&A pairs
cyber	Gold	142	From-scratch cyber operations, SME-produced
mil-bench-5k	Silver	5,000	SME-vetted, general military doctrine topics

Benchmarks were evaluated under three reasoning effort settings—low, medium, and high—scaling the depth of chain-of-thought and sampling steps. Relative error (%, with standard error) against GPT-5 at medium reasoning showed:

• On combat-arms at high reasoning, EdgeRunner achieved significant improvement (–26.7% ± 14.46; p=0.023).
• On mil-bench-5k at medium (–5.41% ± 1.78; p=0.003) and high reasoning (p=0.005), EdgeRunner outperformed GPT-5.
• GPT-5 significantly outperformed EdgeRunner on combat-medic at high reasoning (p=0.002) and on mil-bench-5k under low reasoning (p=0.002).
• All other differences were not statistically significant at $α=0.05$.

4. General-Purpose and In-Domain Regression Analysis

Comparative analysis relative to gpt-oss-20b and GPT-5 assessed model stability across standard academic benchmarks (ARC-C, GPQA Diamond, GSM8k, IFEval, MMLU Pro, TruthfulQA):

Task	EdgeRunner 20B rel. error (%) ± SE	Significant regression vs. gpt-oss-20b?
ARC-C (medium)	–27.8 ± 25.5	No (p=0.2184)
GPQA Diamond (med)	–3.43 ± 8.83	No (p=0.7027)
GSM8k (low)	+34.06 ± 19.53	Yes (p=0.0369)
IFEval (medium)	–2.83 ± 11.35	No (p=0.8057)
MMLU Pro (medium)	–1.31 ± 4.24	No (p=0.7585)
TruthfulQA (medium)	–4.41 ± 7.77	No (p=0.5791)

This profile indicates no significant regression against the base model on general-purpose metrics except GSM8k under low reasoning effort. All other general benchmarks retained parity or minor improvements, suggesting specialization did not compromise broad domain utility.

5. Hyperparameter Choices and Ablation

Hyperparameters were optimized via ablation:

• Optimal learning rate: $1 \times 10^{-6}$ (balance of in-domain gains/minimal regression).
• Global batch size: 1,024–1,536 (sequences of 4,096 tokens).
• Epochs: 5–10 recommended; 1 for most runs due to compute constraints.
• Chat formatting: Adoption of the Alpaca SYSTEM/USER/ASSISTANT template yielded minor performance gains over the gpt-oss template.
• Synthetic reasoning chain augmentation was found to degrade in-domain knowledge injection performance.

A plausible implication is that structural simplicity and careful template selection maximize domain-specific performance without incurring generalization loss.

6. Cost, Throughput, and Edge Deployment

Cloud-based inference costs were modeled as $c_{in} = $1.25$per$10^{6}inputtokensandinput tokens andc_{out} = $10$per$10^{6}$$output tokens. Example annualized usage scenarios per user (25 queries/day, 250 days):</p> <div class='overflow-x-auto max-w-full my-4'><table class='table border-collapse w-full' style='table-layout: fixed'><thead><tr> <th>Scenario</th> <th>Input Tokens</th> <th>Output Tokens</th> <th>Annual Cost</th> </tr> </thead><tbody><tr> <td>Chatbot ($$20/mo) | – | – | $240 Q&A API 10,890 2,625 $249 Agentic (15 calls/interaction) 627,660 21,910 $6,273 Proactive (1/min 24/7) 10,000 1,500 $14,454

Edge-hosted models incur no incremental usage costs post hardware acquisition.

Throughput benchmarking using llama.cpp and Artificial Analysis:

Model/Hardware	Prompt Tokens/s	Generation Tokens/s
GPT-5 API (cloud)	–	155
gpt-oss-20b (RTX 5090)	10,944	262
gpt-oss-20b (M4 Max)	1,659	95
gpt-oss-20b (M3 Air)	365	23

EdgeRunner 20B, quantized similarly to gpt-oss-20b, inherits these efficiency characteristics.

Edge deployment enables fully local inference, eliminating the risks of public network transmission of classified data and supporting redundancy and resilience in contested or air-gapped environments.

7. Significance in Data-Sensitive and Military Contexts

EdgeRunner 20B demonstrates that small, locally-hosted models can match or exceed the performance of state-of-the-art cloud LLMs (e.g., GPT-5) on military tasks when evaluated under statistically rigorous conditions. This outcome is achieved without significant regression on general-purpose NLP benchmarks and at a fraction of the operational cost when deployed on edge devices.

These results suggest the viability of domain-specialized, quantized transformers for mission-critical, data-sensitive applications, particularly in air-gapped or disconnected settings where public cloud inference is infeasible or non-compliant with information security mandates (Fitzgerald et al., 30 Oct 2025).