Decoding Throughput on Qwen3-8B
196.8Throughput (tokens/sec)LAQuant
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LAQuantBits=3, Batch size=1, Hardware=RTX A60002026.05 | 196.8 | 5.15 | |
| ParoQuantBits=3, Batch size=1, Hardware=RTX A60002026.05 | 177 | 4.63 | |
| LAQuantBits=4, Batch size=1, Hardware=RTX A60002026.05 | 93.8 | 2.45 | |
| ParoQuantBits=4, Batch size=1, Hardware=RTX A60002026.05 | 88.7 | 2.32 | |
| FP16Bits=16, Batch size=1, Hardware=RTX A60002026.05 | 38.2 | — |