Decode Throughput on Qwen3 0.6B
464.5Throughput (tok/s)BaseRT
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| BaseRTQuantization=Q4, Hardware=Apple M4 Pro, Target Length=tg1282026.07 | 464.5 | 1.56 | 1.35 | |
| MLXQuantization=Q4, Hardware=Apple M4 Pro, Target Length=tg1282026.07 | 343.6 | — | — | |
| BaseRTQuantization=Q8, Hardware=Apple M4 Pro, Target Length=tg1282026.07 | 321.2 | 1.46 | 1.26 | |
| llama.cppQuantization=Q4, Hardware=Apple M4 Pro, Target Length=tg1282026.07 | 297.4 | — | — | |
| MLXQuantization=Q8, Hardware=Apple M4 Pro, Target Length=tg1282026.07 | 255.3 | — | — | |
| llama.cppQuantization=Q8, Hardware=Apple M4 Pro, Target Length=tg1282026.07 | 219.8 | — | — |