Phi-4-mini
Phi-4-mini Overview
Phi-4-mini is a lightweight 3.8B-parameter open model from Microsoft’s Phi-4 family, built on synthetic “textbook-quality” data and filtered public web text focused on reasoning-dense content. It supports a 128K token context window and a 200K vocabulary, and was post-trained with supervised fine-tuning (SFT) and direct preference optimization (DPO) for precise instruction adherence and robust safety. Compared with Phi-3.5-mini it adopts a new efficient architecture: larger 200K vocabulary, grouped-query attention (GQA), and shared input/output embeddings. Long-awaited function calling (tool use) is now supported. Licensed under MIT, it targets memory/compute-constrained environments, latency-bound scenarios, and strong reasoning (especially math and logic).
Specs at a Glance
| Item | Value |
|---|---|
| Parameters | 3.8B (dense decoder-only Transformer, BF16) |
| Context length | 128K tokens |
| Vocabulary | 200,064 tokens (multilingual-friendly) |
| License | MIT (commercial use OK) |
| Training | 5T tokens · 512 A100-80G · 21 days |
| Weights size | ~7.7GB BF16 (HF) · 2.5GB Q4 (Ollama) |
| Function calling | Yes (<|tool|> chat template, tools in JSON) |
| Languages | 20+ listed (en, ko, ja, zh, de, fr…) — English-centric training |
How to Use
- Ollama (easiest):
ollama run phi4-mini— 2.5GB Q4 download, 128K context, function calling supported. Requires Ollama 0.5.13+. REST: POST tohttp://localhost:11434/api/chat; Python/JS via the officialollamalibraries. - Transformers:
transformers>=4.49.0, load withAutoModelForCausalLM.from_pretrained("microsoft/Phi-4-mini-instruct", trust_remote_code=True, torch_dtype="auto", device_map="auto"). - vLLM (serving):
vllm>=0.7.3—LLM(model="microsoft/Phi-4-mini-instruct", trust_remote_code=True)thenllm.chat(messages, sampling_params). - LM Studio / GGUF: Q4_K_M is only ~2.2GB — runs on any 4GB GPU, 8GB Mac, or CPU-only PC.
- Prompt format: chat template
<|system|>…<|end|><|user|>…<|end|><|assistant|>; for tool calling put the JSON tool spec inside<|tool|>…<|/tool|>in the system message.
Benchmarks (official, 3.8B vs peers)
| Benchmark | Phi-4-mini | Phi-3.5-mini | Llama-3.2-3B | Qwen2.5-7B | GPT-4o-mini |
|---|---|---|---|---|---|
| MMLU (5-shot) | 67.3 | 65.5 | 61.8 | 72.6 | 77.2 |
| MMLU-Pro (0-shot CoT) | 52.8 | 47.4 | 39.2 | 56.2 | 62.8 |
| BigBench Hard (0-shot CoT) | 70.4 | 63.1 | 55.4 | 72.4 | 80.4 |
| GSM8K (8-shot CoT) | 88.6 | 76.9 | 75.6 | 88.7 | 91.3 |
| MATH (0-shot CoT) | 64.0 | 49.8 | 46.7 | 60.4 | 70.2 |
| MGSM (0-shot CoT) | 63.9 | 49.6 | 44.6 | 64.5 | 81.7 |
| Multilingual MMLU (5-shot) | 49.3 | 51.8 | 48.1 | 64.4 | 72.9 |
At 3.8B it rivals 7-8B-class models on math and reasoning (GSM8K 88.6, MATH 64.0 beats Qwen2.5-7B on MATH) — but it is still size-limited for factual knowledge storage.
On-Device Performance (review roundup)
| Hardware | Setup | Speed | Source |
|---|---|---|---|
| RTX 4090 | GGUF Q4 (Ollama/llama.cpp) | ~300 tok/s | community benchmarks |
| Core i9-14900K + Arc B580 | 4-bit OpenVINO | >90 tok/s | Intel / r/LocalLLaMA |
| Core Ultra 9 288V (AI PC iGPU) | OpenVINO, on-device | interactive real-time | Intel official |
| CPU-only (modern desktop) | Q4 llama.cpp | ~15–25 tok/s | community benchmarks |
| Raspberry Pi 5 | Q4 quantized | ~5–8 tok/s | community benchmarks |
| Intel Xeon 6 (2S, batched) | BF16, 1K in/1K out | 1955 tok/s aggregate | Intel official |
Mobile deployment paths: ONNX Runtime (Microsoft’s own, NPU-optimized), NexaAI’s NPU/mobile build on Hugging Face, MLC-LLM (Android/iOS), MediaPipe, and WebLLM (WebGPU browser). MLPerf Client adopted Phi-4-mini as a mandatory benchmark model, so per-device scores are easy to find. In Ollama expect 200–350 tok/s on modern GPUs — fast enough for IDE autocomplete (Continue.dev) and real-time chatbots.
Honest Limits
- Factual knowledge: 3.8B simply cannot store much world knowledge — pair it with RAG/search to curb hallucinations.
- Multilingual gap: ~92% English training data; Multilingual MMLU 49.3 trails Qwen2.5-3B (55.9). Korean/Japanese quality is usable but clearly below English.
- Code: training data is Python-heavy (typing, math, random, itertools…); verify API usage for other languages.
- Long chats: can drift/repeat over very long sessions; cap conversation turns in production.
Links
Ollama: https://ollama.com/library/phi4-mini · Hugging Face: https://huggingface.co/microsoft/Phi-4-mini-instruct · Technical Report: arXiv 2503.01743 · Azure AI Foundry / Phi Cookbook also available.