LLM Prefill throughput on LLM Prompt Length 1024
4,748Prefill Throughput (tok/s)MLX
Evaluation Results
| Method | Links | |
|---|---|---|
| MLXModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 4,748 | |
| MLXModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 4,742 | |
| llama.cppModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 4,638 | |
| llama.cppModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 4,605 | |
| BaseRTModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 4,399 | |
| BaseRTModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 4,399 | |
| BaseRTModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 4,354 | |
| BaseRTModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 4,354 | |
| MLXModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 2,851 | |
| MLXModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 2,828 | |
| llama.cppModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 2,671 | |
| llama.cppModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 2,643 | |
| BaseRTModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 2,577 | |
| BaseRTModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 2,577 | |
| BaseRTModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 2,531 | |
| BaseRTModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 2,531 | |
| llama.cppModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 1,268 | |
| BaseRTModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 1,198 | |
| BaseRTModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 1,179 | |
| llama.cppModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 977 | |
| MLXModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 942 | |
| llama.cppModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 896 | |
| MLXModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 894 | |
| BaseRTModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 888 | |
| BaseRTModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 888 | |
| llama.cppModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 887 | |
| BaseRTModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 874 | |
| BaseRTModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 874 | |
| BaseRTModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 810 | |
| BaseRTModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 810 | |
| MLXModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 726 | |
| BaseRTModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 681 | |
| BaseRTModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 681 | |
| MLXModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 664 | |
| llama.cppModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 652 | |
| llama.cppModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 586 |