Quantization on LLaMA2-7B
24Averaged Quantization Time (s)SingleQuant
Evaluation results
| Method | Links | |
|---|---|---|
| SingleQuantHardware=NVIDIA A800 80GB GPU, Number of trials=302025.11 | 24 | |
| Du QuantHardware=NVIDIA A800 80GB GPU, Number of trials=302025.11 | 58 | |
| OST QuantHardware=NVIDIA A800 80GB GPU, Number of trials=302025.11 | 2,184 | |
| Spin QuantHardware=NVIDIA A800 80GB GPU, Number of trials=302025.11 | 22,334 |