Decoding Throughput on Qwen3 1.7B
316.9Decoding Throughput (tokens/sec)LAQuant
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LAQuantBits=3, Batch size=1, Hardware=RTX A60002026.05 | 316.9 | 2.29 | |
| ParoQuantBits=3, Batch size=1, Hardware=RTX A60002026.05 | 283.7 | 2.05 | |
| LAQuantBits=4, Batch size=1, Hardware=RTX A60002026.05 | 227.3 | 1.64 | |
| ParoQuantBits=4, Batch size=1, Hardware=RTX A60002026.05 | 205.3 | 1.48 | |
| FP16Bits=16, Batch size=1, Hardware=RTX A60002026.05 | 138.4 | — |