Decoding Throughput on Qwen3-4B
231.3Throughput (tokens/sec)LAQuant
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LAQuantBits=3, Batch size=1, Hardware=RTX A60002026.05 | 231.3 | 3.42 | |
| ParoQuantBits=3, Batch size=1, Hardware=RTX A60002026.05 | 203.5 | 3.01 | |
| LAQuantBits=4, Batch size=1, Hardware=RTX A60002026.05 | 135.1 | 2 | |
| ParoQuantBits=4, Batch size=1, Hardware=RTX A60002026.05 | 124.5 | 1.84 | |
| FP16Bits=16, Batch size=1, Hardware=RTX A60002026.05 | 67.6 | — |