LLM Prefill throughput on LLM Prompt Length 128 (pp128)
4,796Throughput (tok/s)llama.cpp
Evaluation Results
| Method | Links | |
|---|---|---|
| llama.cppModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 4,796 | |
| uzuModel=Qwen3 0.6B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 4,684 | |
| llama.cppModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 4,537 | |
| uzuModel=Qwen3 0.6B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 4,466 | |
| BaseRTModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 4,331 | |
| BaseRTModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 4,331 | |
| BaseRTModel=Qwen3 0.6B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 4,331 | |
| BaseRTModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 4,288 | |
| BaseRTModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 4,288 | |
| BaseRTModel=Qwen3 0.6B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 4,288 | |
| MLXModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 3,843 | |
| MLXModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 3,742 | |
| llama.cppModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 2,612 | |
| llama.cppModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 2,555 | |
| uzuModel=Llama 3.2 1B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 2,519 | |
| BaseRTModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 2,490 | |
| BaseRTModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 2,490 | |
| BaseRTModel=Llama 3.2 1B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 2,490 | |
| MLXModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 2,451 | |
| BaseRTModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 2,438 | |
| BaseRTModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 2,438 | |
| BaseRTModel=Llama 3.2 1B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 2,438 | |
| uzuModel=Llama 3.2 1B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 2,426 | |
| MLXModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 2,342 | |
| llama.cppModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 1,238 | |
| BaseRTModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 1,146 | |
| BaseRTModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 1,125 | |
| llama.cppModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 919 | |
| llama.cppModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 916 | |
| llama.cppModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 907 | |
| uzuModel=Llama 3.2 3B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 903 | |
| BaseRTModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 888 | |
| BaseRTModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 888 | |
| BaseRTModel=Llama 3.2 3B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07 | 888 | |
| MLXModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 879 | |
| BaseRTModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 866 | |
| BaseRTModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07 | 866 | |
| BaseRTModel=Llama 3.2 3B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 866 | |
| uzuModel=Llama 3.2 3B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07 | 863 | |
| MLXModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07 | 815 | |
| BaseRTModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 738 | |
| BaseRTModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 738 | |
| BaseRTModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 659 | |
| BaseRTModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 659 | |
| MLXModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 464 | |
| MLXModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07 | 415 | |
| llama.cppModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 414 | |
| llama.cppModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07 | 407 |