LLM Decode Throughput on Apple M3 base tg128 (test)
223Throughput (tok/s)BaseRT
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| BaseRTModel=Qwen3 0.6B, Quant=Q4, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 223 | 1.24 | 1.22 | |
| MLXModel=Qwen3 0.6B, Quant=Q4, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 182.3 | — | — | |
| llama.cppModel=Qwen3 0.6B, Quant=Q4, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 180.5 | — | — | |
| BaseRTModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 125.6 | 1.18 | 1.12 | |
| BaseRTModel=Qwen3 0.6B, Quant=Q8, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 124.6 | 1.25 | 1.1 | |
| MLXModel=Qwen3 0.6B, Quant=Q8, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 113.6 | — | — | |
| MLXModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 112.6 | — | — | |
| llama.cppModel=Llama 3.2 1B, Quant=Q4, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 106.3 | — | — | |
| llama.cppModel=Qwen3 0.6B, Quant=Q8, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 99.8 | — | — | |
| BaseRTModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 65.1 | 1.13 | 1.04 | |
| MLXModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 62.4 | — | — | |
| BaseRTModel=Gemma 4 E2B, Quant=Q4, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 59 | 1.25 | — | |
| llama.cppModel=Llama 3.2 1B, Quant=Q8, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 57.5 | — | — | |
| llama.cppModel=Gemma 4 E2B, Quant=Q4, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 47.1 | — | — | |
| BaseRTModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 44.5 | 1.14 | 1.01 | |
| MLXModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 43.9 | — | — | |
| llama.cppModel=Llama 3.2 3B, Quant=Q4, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 39 | — | — | |
| BaseRTModel=Gemma 4 E2B, Quant=Q8, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 35.4 | 1.34 | — | |
| llama.cppModel=Gemma 4 E2B, Quant=Q8, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 26.5 | — | — | |
| BaseRTModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 24.2 | 1.14 | 1.09 | |
| MLXModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 22.2 | — | — | |
| llama.cppModel=Llama 3.2 3B, Quant=Q8, Hardware=Apple M3 base, Target Generation Length=tg1282026.07 | 21.2 | — | — |