List your company
ResearchFormerly WizWand

LLM Prefill throughput on LLM Prompt Length 1024

4,748Prefill Throughput (tok/s)MLX
419.521,543.262,6673,790.74Jul 1, 2026
Updated 4mo ago

Evaluation results

MethodLinks
MLXModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.074,748
MLXModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.074,742
llama.cppModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.074,638
llama.cppModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.074,605
BaseRTModel=Qwen3 0.6B, Quantization=Q4, Hardware=Apple M4 Pro2026.074,399
BaseRTModel=Qwen2 0.5B, Quant=Q4, Hardware=Apple M4 Pro2026.074,399
BaseRTModel=Qwen3 0.6B, Quantization=Q8, Hardware=Apple M4 Pro2026.074,354
BaseRTModel=Qwen2 0.5B, Quant=Q8, Hardware=Apple M4 Pro2026.074,354
MLXModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.072,851
MLXModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.072,828
llama.cppModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.072,671
llama.cppModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.072,643
BaseRTModel=Llama 3.2 1B, Quantization=Q4, Hardware=Apple M4 Pro2026.072,577
BaseRTModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M4 Pro2026.072,577
BaseRTModel=Llama 3.2 1B, Quantization=Q8, Hardware=Apple M4 Pro2026.072,531
BaseRTModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M4 Pro2026.072,531
llama.cppModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.071,268
BaseRTModel=Gemma 2 2B, Quant=Q4, Hardware=Apple M4 Pro2026.071,198
BaseRTModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.071,179
llama.cppModel=Gemma 2 2B, Quant=Q8, Hardware=Apple M4 Pro2026.07977
MLXModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07942
llama.cppModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07896
MLXModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07894
BaseRTModel=Llama 3.2 3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07888
BaseRTModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07888
llama.cppModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M4 Pro2026.07887
BaseRTModel=Llama 3.2 3B, Quantization=Q8, Hardware=Apple M4 Pro2026.07874
BaseRTModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M4 Pro2026.07874
BaseRTModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07810
BaseRTModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07810
MLXModel=Qwen3 30B-A3B, Quantization=Q4, Hardware=Apple M4 Pro2026.07726
BaseRTModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07681
BaseRTModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07681
MLXModel=Gemma 4 26B-A4B, Quantization=Q4, Hardware=Apple M4 Pro2026.07664
llama.cppModel=Mistral 7B v0.3, Quant=Q4, Hardware=Apple M4 Pro2026.07652
llama.cppModel=Llama 3.1 8B, Quant=Q4, Hardware=Apple M4 Pro2026.07586