LLM Decoding Throughput on Apple M4 Pro tg128
465Throughput (tokens/s)BaseRT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| BaseRTModel=Qwen3 0.6B, Quantization=4-bit2026.07 | 465 | 1.17 | |
| uzuModel=Qwen3 0.6B, Quantization=4-bit2026.07 | 398 | — | |
| BaseRTModel=Qwen3 0.6B, Quantization=8-bit2026.07 | 321 | 1.19 | |
| BaseRTModel=Llama 3.2 1B, Quantization=4-bit2026.07 | 295 | 1.02 | |
| uzuModel=Llama 3.2 1B, Quantization=4-bit2026.07 | 290 | — | |
| uzuModel=Qwen3 0.6B, Quantization=8-bit2026.07 | 271 | — | |
| BaseRTModel=Llama 3.2 1B, Quantization=8-bit2026.07 | 184 | 1.06 | |
| uzuModel=Llama 3.2 1B, Quantization=8-bit2026.07 | 173 | — | |
| uzuModel=Llama 3.2 3B, Quantization=4-bit2026.07 | 119 | — | |
| BaseRTModel=Llama 3.2 3B, Quantization=4-bit2026.07 | 117 | 0.99 | |
| BaseRTModel=Llama 3.2 3B, Quantization=8-bit2026.07 | 71 | 1.04 | |
| uzuModel=Llama 3.2 3B, Quantization=8-bit2026.07 | 68 | — |