Text Generation on Text model inference M4 Max
525.5Throughput (tok/s)vllm-mlx
Evaluation results
| Method | Links | ||
|---|---|---|---|
| vllm-mlxModel=Qwen3-0.6B, Quantization=4-bit2026.01 | 525.5 | 1.87 | |
| vllm-mlxModel=Llama-3.2-1B, Quantization=4-bit2026.01 | 461.9 | 1.39 | |
| vllm-metalModel=Qwen3-0.6B, Quantization=4-bit2026.01 | 365.8 | — | |
| mlx-lmModel=Qwen3-0.6B, Quantization=4-bit2026.01 | 356.2 | — | |
| vllm-metalModel=Llama-3.2-1B, Quantization=4-bit2026.01 | 350.9 | — | |
| mlx-lmModel=Llama-3.2-1B, Quantization=4-bit2026.01 | 347.1 | — | |
| llama.cppModel=Llama-3.2-1B, Quantization=4-bit2026.01 | 331.3 | — | |
| llama.cppModel=Qwen3-0.6B, Quantization=4-bit2026.01 | 281.5 | — | |
| vllm-mlxModel=Llama-3.2-3B, Quantization=4-bit2026.01 | 203.6 | 1.31 | |
| vllm-metalModel=Llama-3.2-3B, Quantization=4-bit2026.01 | 174.3 | — | |
| mlx-lmModel=Llama-3.2-3B, Quantization=4-bit2026.01 | 167.5 | — | |
| vllm-mlxModel=Qwen3-4B, Quantization=4-bit2026.01 | 159 | 1.35 | |
| llama.cppModel=Llama-3.2-3B, Quantization=4-bit2026.01 | 155.8 | — | |
| vllm-mlxModel=Gemma 3-4B, Quantization=4-bit2026.01 | 152.5 | 1.24 | |
| vllm-metalModel=Qwen3-4B, Quantization=4-bit2026.01 | 137.3 | — | |
| mlx-lmModel=Qwen3-4B, Quantization=4-bit2026.01 | 128.9 | — | |
| llama.cppModel=Gemma 3-4B, Quantization=4-bit2026.01 | 123.2 | — | |
| vllm-mlxModel=Nemotron-30B-A3B, Quantization=4-bit2026.01 | 121.8 | 1.43 | |
| llama.cppModel=Qwen3-4B, Quantization=4-bit2026.01 | 118.2 | — | |
| vllm-metalModel=Gemma 3-4B, Quantization=4-bit2026.01 | 117 | — | |
| vllm-metalModel=Qwen3-30B-A3B, Quantization=4-bit2026.01 | 110.3 | — | |
| vllm-mlxModel=Qwen3-30B-A3B, Quantization=4-bit2026.01 | 109.7 | 1.17 | |
| mlx-lmModel=Qwen3-30B-A3B, Quantization=4-bit2026.01 | 107.4 | — | |
| mlx-lmModel=Gemma 3-4B, Quantization=4-bit2026.01 | 105.4 | — | |
| mlx-lmModel=Nemotron-30B-A3B, Quantization=4-bit2026.01 | 101.6 | — | |
| vllm-mlxModel=Qwen3-8B, Quantization=4-bit2026.01 | 93.3 | 1.21 | |
| llama.cppModel=Qwen3-30B-A3B, Quantization=4-bit2026.01 | 89.9 | — | |
| vllm-metalModel=Qwen3-8B, Quantization=4-bit2026.01 | 87.1 | — | |
| llama.cppModel=Nemotron-30B-A3B, Quantization=4-bit2026.01 | 85.1 | — | |
| mlx-lmModel=Qwen3-8B, Quantization=4-bit2026.01 | 79.9 | — | |
| llama.cppModel=Qwen3-8B, Quantization=4-bit2026.01 | 76.9 | — |