LLM Prefill throughput on LLM Prompt Length 256 (pp256)
5,063Prefill Throughput (tok/s)uzu
Evaluation Results
| Method | Links | |
|---|---|---|
| uzuModel=Qwen3 0.6B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 5,063 | |
| llama.cppModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 4,987 | |
| llama.cppModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 4,914 | |
| uzuModel=Qwen3 0.6B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 4,887 | |
| BaseRTModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 4,496 | |
| BaseRTModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 4,496 | |
| BaseRTModel=Qwen3 0.6B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 4,496 | |
| BaseRTModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 4,434 | |
| BaseRTModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 4,434 | |
| BaseRTModel=Qwen3 0.6B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 4,434 | |
| MLXModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 4,263 | |
| MLXModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 4,255 | |
| llama.cppModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 2,714 | |
| MLXModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 2,698 | |
| llama.cppModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 2,673 | |
| MLXModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 2,620 | |
| uzuModel=Llama 3.2 1B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 2,604 | |
| BaseRTModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 2,577 | |
| BaseRTModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 2,577 | |
| BaseRTModel=Llama 3.2 1B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 2,577 | |
| uzuModel=Llama 3.2 1B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 2,545 | |
| BaseRTModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 2,517 | |
| BaseRTModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 2,517 | |
| BaseRTModel=Llama 3.2 1B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 2,517 | |
| llama.cppModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 1,281 | |
| BaseRTModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 1,193 | |
| BaseRTModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 1,167 | |
| llama.cppModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 976 | |
| llama.cppModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 928 | |
| uzuModel=Llama 3.2 3B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 923 | |
| llama.cppModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 916 | |
| BaseRTModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 906 | |
| BaseRTModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 906 | |
| BaseRTModel=Llama 3.2 3B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 906 | |
| MLXModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 901 | |
| MLXModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 897 | |
| uzuModel=Llama 3.2 3B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 895 | |
| BaseRTModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 888 | |
| BaseRTModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 888 | |
| BaseRTModel=Llama 3.2 3B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 888 | |
| BaseRTModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 809 | |
| BaseRTModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 809 | |
| BaseRTModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 704 | |
| BaseRTModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 704 | |
| MLXModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 576 | |
| MLXModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 571 | |
| llama.cppModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 550 | |
| llama.cppModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 525 |