List your company
ResearchFormerly WizWand

LLM Prefill throughput on LLM Prompt Length 128 (pp128)

4,796Throughput (tok/s)llama.cpp
231.441,416.472,601.53,786.53Jul 1, 2026
Updated 4mo ago

Evaluation results

MethodLinks
llama.cppModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.074,796
uzuModel=Qwen3 0.6B, Quantization=4-bit, Hardware=Apple M4 Pro2026.074,684
llama.cppModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.074,537
uzuModel=Qwen3 0.6B, Quantization=8-bit, Hardware=Apple M4 Pro2026.074,466
BaseRTModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.074,331
BaseRTModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.074,331
BaseRTModel=Qwen3 0.6B, Quantization=4-bit, Hardware=Apple M4 Pro2026.074,331
BaseRTModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.074,288
BaseRTModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.074,288
BaseRTModel=Qwen3 0.6B, Quantization=8-bit, Hardware=Apple M4 Pro2026.074,288
MLXModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.073,843
MLXModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.073,742
llama.cppModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.072,612
llama.cppModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.072,555
uzuModel=Llama 3.2 1B, Quantization=4-bit, Hardware=Apple M4 Pro2026.072,519
BaseRTModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.072,490
BaseRTModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.072,490
BaseRTModel=Llama 3.2 1B, Quantization=4-bit, Hardware=Apple M4 Pro2026.072,490
MLXModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.072,451
BaseRTModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.072,438
BaseRTModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.072,438
BaseRTModel=Llama 3.2 1B, Quantization=8-bit, Hardware=Apple M4 Pro2026.072,438
uzuModel=Llama 3.2 1B, Quantization=8-bit, Hardware=Apple M4 Pro2026.072,426
MLXModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.072,342
llama.cppModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.071,238
BaseRTModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.071,146
BaseRTModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.071,125
llama.cppModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07919
llama.cppModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.07916
llama.cppModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07907
uzuModel=Llama 3.2 3B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07903
BaseRTModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07888
BaseRTModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07888
BaseRTModel=Llama 3.2 3B, Quantization=4-bit, Hardware=Apple M4 Pro2026.07888
MLXModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07879
BaseRTModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07866
BaseRTModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07866
BaseRTModel=Llama 3.2 3B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07866
uzuModel=Llama 3.2 3B, Quantization=8-bit, Hardware=Apple M4 Pro2026.07863
MLXModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07815
BaseRTModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07738
BaseRTModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07738
BaseRTModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07659
BaseRTModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07659
MLXModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07464
MLXModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07415
llama.cppModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07414
llama.cppModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07407