Quantization on LLaMA2-13B
37Averaged Quantization Time (s)SingleQuant
Evaluation results
| Method | Links | |
|---|---|---|
| SingleQuantHardware=NVIDIA A800 80GB GPU, Number of trials=302025.11 | 37 | |
| Du QuantHardware=NVIDIA A800 80GB GPU, Number of trials=302025.11 | 95 | |
| OST QuantHardware=NVIDIA A800 80GB GPU, Number of trials=302025.11 | 4,598 | |
| Spin QuantHardware=NVIDIA A800 80GB GPU, Number of trials=302025.11 | 52,561 |