Decode Throughput on Gemma 4 E2B
127.7Throughput (tok/s)BaseRT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BaseRTQuantization=Q4, Hardware=Apple M4 Pro, Target Length=tg1282026.07 | 127.7 | 1.19 | — | |
| llama.cppQuantization=Q4, Hardware=Apple M4 Pro, Target Length=tg1282026.07 | 107 | — | — | |
| BaseRTQuantization=Q8, Hardware=Apple M4 Pro, Target Length=tg1282026.07 | 84.5 | 1.42 | — | |
| llama.cppQuantization=Q8, Hardware=Apple M4 Pro, Target Length=tg1282026.07 | 59.5 | — | — |