LLM Prefill throughput on LLM Prompt Length 2048 (pp2048)
4,447Prefill Throughput (tok/s)MLX
Evaluation Results
| Method | Links | |
|---|---|---|
| MLXModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 4,447 | |
| MLXModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 4,439 | |
| llama.cppModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 4,075 | |
| llama.cppModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 4,045 | |
| BaseRTModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 3,680 | |
| BaseRTModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 3,680 | |
| BaseRTModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 3,648 | |
| BaseRTModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 3,648 | |
| MLXModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 2,783 | |
| MLXModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 2,714 | |
| llama.cppModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 2,520 | |
| llama.cppModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 2,489 | |
| BaseRTModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 2,453 | |
| BaseRTModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 2,453 | |
| BaseRTModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 2,418 | |
| BaseRTModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 2,418 | |
| llama.cppModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 1,209 | |
| BaseRTModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 1,169 | |
| BaseRTModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 1,153 | |
| llama.cppModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 971 | |
| MLXModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 934 | |
| MLXModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 900 | |
| llama.cppModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 864 | |
| llama.cppModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 840 | |
| BaseRTModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 835 | |
| BaseRTModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 835 | |
| BaseRTModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 792 | |
| BaseRTModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 792 | |
| BaseRTModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 721 | |
| BaseRTModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 721 | |
| MLXModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 720 | |
| MLXModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 644 | |
| BaseRTModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 626 | |
| BaseRTModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 626 | |
| llama.cppModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 601 | |
| llama.cppModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 558 |