Phi-3.5 Mini Instruct Overview

• Phi-3.5 Mini Instruct is an instruction-tuned, decoder-only LLM with 3.8B parameters, designed for robust performance and portability.
• It employs a dual-phase pretraining strategy with extensive data curation, including synthetic examples for enhanced multi-step reasoning, mathematics, and code synthesis.
• The model’s supervised fine-tuning and direct preference optimization ensure strong alignment and safety, enabling resource-efficient deployment using low-bit quantization.

Phi-3.5 Mini Instruct is an instruction-tuned, decoder-only LLM within the Phi-3.5 series developed to optimize instruction-following, safety, and resource efficiency while maintaining high performance in academic benchmarks and practical deployments. At 3.8 billion parameters, it exemplifies a design philosophy emphasizing high data quality, robust alignment, and portability, setting a new baseline for small but capable Transformer-based LLMs in both research and edge-device applications (Abdin et al., 2024).

1. Model Architecture and Parameterization

Phi-3.5 Mini Instruct employs a decoder-only Transformer architecture mirroring that of Phi-3-Mini with 32 layers, each featuring 32 attention heads and a hidden dimension of 3072. The model uses rotary positional embeddings (RoPE) for position encoding and a vocabulary size of 32,064 tokens, identical to Llama-2’s tokenization scheme. The default context window is 4,096 tokens, but mid-training application of the LongRoPE method extends context capability to 128,000 tokens for tasks requiring extended sequence understanding. Training was conducted in bfloat16 precision across all layers for numerical efficiency at scale (Abdin et al., 2024).

2. Data Curation and Pretraining Regime

The pretraining dataset comprises 3.3 trillion tokens sourced in two sequential phases. The initial phase utilizes a broad web crawl filtered using “educational level” criteria and LLM-based heuristics to remove trivial, topical, or low-quality documents. The second phase refines this with a curated web subset and a proportion of synthetic data generated by larger LLMs, emphasizing examples in multi-step reasoning, mathematics, and code synthesis. This hybrid approach ensures exposure to rich, diverse linguistic patterns and advanced reasoning tasks. All filtering leverages automated LLM scoring to further optimize the training corpus for conceptual depth and generalization capabilities (Abdin et al., 2024).

3. Supervised Instruction Tuning and Alignment

Post-pretraining, Phi-3.5 Mini Instruct undergoes two-stage supervised alignment:

1. Supervised Fine-Tuning (SFT): Instruction–response pairs, highly curated and multi-domain (math, logical reasoning, code, dialogue, model-identity, safety), are used for SFT. The chat-style prompt format relies on <|user|> and <|assistant|> special tokens. The cross-entropy loss employed is:

$$\mathcal{L}_{CE} = -\sum_{t} \log p_\theta(y_t \mid y_{< t}, x)$$

2. Direct Preference Optimization (DPO): This stage operates on preference pairs ("preferred" vs. "rejected"), including those that probe model robustness and safety boundaries. The pairwise logistic DPO objective magnifies the score gap in favor of preferred completions. This method, rooted in preference-based alignment, follows Ouyang et al. (2022) and incorporates safety triggers and off-distribution examples to optimize real-world deployment fidelity.

Safety alignment utilizes public helpful/harmless datasets (e.g., Bai et al. 2022; Ji et al. 2023) alongside independent adversarial red-teaming. Model outputs are subjected to iterative adversarial multi-turn probing, and instances of harm are incorporated into subsequent DPO rounds, resulting in measurable reductions in harmful response rates according to internal and GPT-4 simulated benchmarks (Abdin et al., 2024).

4. Quantitative Evaluations and Benchmarking

With 3.8 billion parameters and a 128K context window, Phi-3.5 Mini Instruct attains the following on representative tasks (all 5-shot except as noted):

Benchmark	Score	Comparison Models*
MMLU	69.0%	Llama 3.1-8B: 61.0%
HellaSwag	69.4%	Mixtral-8x7B: ~48%
ARC-Challenge (10-shot)	84.6%
GSM-8K (8-shot CoT)	86.2%
HumanEval (0-shot)	61.5%
MBPP (3-shot)	68.6%
TruthfulQA (10-shot)	64.0%

*Scores noted only if present in data for comparison.

The model achieves an overall average across 30+ public benchmarks of 61.1%, matching or outperforming open-source comparators such as Llama 3.1-8B and Mixtral-8x7B, and approaching the performance of Gemini-1.5-Flash and GPT-4o-mini. In multilingual benchmarks, Phi-3.5 Mini Instruct records 55.4% on MMLU-multilingual and 47.9% on MGSM (0-shot CoT). Code understanding and generation tasks, such as RepoQA (128K context) and HumanEval, place it among the most capable sub-4B models (Abdin et al., 2024).

5. Resource Efficiency and Deployment

Phi-3.5 Mini Instruct has been explicitly designed for portability and on-device use. Employing 4-bit quantization, the model weight footprint is approximately 1.8 GB. On an iPhone 14 (A16 Bionic), this achieves inference speeds exceeding 12 tokens/second during native, device-only computation. The complete pipeline does not involve structured pruning; instead, optimization is achieved through low-bit quantization and block-sparse attention layers (in related models) for efficient memory and speed characteristics (Abdin et al., 2024).

6. Data-Optimal Scaling and Empirical Trends

The Phi series, encompassing models such as Phi-1.5 (1.5B), Phi-2 (2.7B), Phi-3-Mini (3.8B), Phi-3-Small (7B), and Phi-3-Medium (14B), explores data-optimal scaling. Empirical evaluation of log-MMLU-error versus log-model-size demonstrates that combining heavily filtered web data with targeted synthetic corpora enables the models to sit near the empirical compute-optimal frontier as described by:

$L(N, D) \approx A N^\alpha D^\beta$

where $N$ is model size and $D$ is dataset size. Notably, scaling from 2.7B to 3.8B parameters yields substantial accuracy gains, while increased scale from 7B to 14B yields diminishing returns, suggesting the need for further data curation at larger scales (Abdin et al., 2024).

7. Relationship to SlimMoE and Phi-mini-MoE-Instruct Variants

Distinct from SlimMoE-based Phi-mini-MoE-instruct and Phi-tiny-MoE-instruct models (Li et al., 23 Jun 2025), Phi-3.5 Mini Instruct does not employ Mixture of Experts (MoE) or expert slimming. While the SlimMoE pipeline compresses larger, expert-based models via structured, multi-stage neuron-level pruning and staged knowledge distillation, resulting in higher parameter efficiency and fine-tunability on single-GPU hardware, Phi-3.5 Mini Instruct retains a dense, single-expert topology. SlimMoE-derived models demonstrate superior parameter efficiency due to their ability to leverage expert pruning and distillation but at the cost of more complex training and architecture modification pipelines (Li et al., 23 Jun 2025). This suggests different design tradeoffs for deployments sensitive to memory, compute, or ease of modification.

---

References:

• (Abdin et al., 2024) Phi-3 Technical Report: A Highly Capable LLM Locally on Your Phone.
• (Li et al., 23 Jun 2025) SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation.