LLM Prefill throughput on LLM Prompt Length 512 (pp512)
5,128Throughput (tok/s)uzu
Evaluation Results
| Method | Links | |
|---|---|---|
| uzuModel=Qwen3 0.6B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 5,128 | |
| uzuModel=Qwen3 0.6B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 5,001 | |
| llama.cppModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 4,993 | |
| llama.cppModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 4,933 | |
| BaseRTModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 4,690 | |
| BaseRTModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 4,690 | |
| BaseRTModel=Qwen3 0.6B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 4,690 | |
| MLXModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 4,658 | |
| MLXModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 4,654 | |
| BaseRTModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 4,623 | |
| BaseRTModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 4,623 | |
| BaseRTModel=Qwen3 0.6B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 4,623 | |
| MLXModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 2,817 | |
| MLXModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 2,766 | |
| llama.cppModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 2,728 | |
| llama.cppModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 2,702 | |
| uzuModel=Llama 3.2 1B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 2,642 | |
| BaseRTModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 2,615 | |
| BaseRTModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 2,615 | |
| BaseRTModel=Llama 3.2 1B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 2,615 | |
| uzuModel=Llama 3.2 1B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 2,597 | |
| BaseRTModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 2,564 | |
| BaseRTModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 2,564 | |
| BaseRTModel=Llama 3.2 1B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 2,564 | |
| llama.cppModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 1,292 | |
| BaseRTModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 1,208 | |
| BaseRTModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 1,187 | |
| llama.cppModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 983 | |
| MLXModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 943 | |
| llama.cppModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 932 | |
| uzuModel=Llama 3.2 3B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 929 | |
| MLXModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 919 | |
| uzuModel=Llama 3.2 3B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 905 | |
| BaseRTModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 899 | |
| BaseRTModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 899 | |
| BaseRTModel=Llama 3.2 3B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 899 | |
| BaseRTModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 893 | |
| BaseRTModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 893 | |
| BaseRTModel=Llama 3.2 3B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 893 | |
| llama.cppModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 871 | |
| BaseRTModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 838 | |
| BaseRTModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 838 | |
| BaseRTModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 721 | |
| BaseRTModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 721 | |
| llama.cppModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 668 | |
| MLXModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 664 | |
| MLXModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 639 | |
| llama.cppModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 609 |