Language Model Inference Efficiency on Meta-Llama-3-8B
1,398Throughput (tokens/s)AWQ
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| AWQQuantization=4-bit2026.02 | 1,398 | 0.715 | |
| DACQ HybridQuantization=4-bit2026.02 | 1,022.5 | 0.978 | |
| DACQ LogisticQuantization=4-bit2026.02 | 975.05 | 1.025 | |
| UnquantizedPrecision=FP162026.02 | 857.6 | 1.166 |