Quantization on LLaMA3-8B
27Averaged Quantization Time (s)SingleQuant
Evaluation Results
| Method | Links | |
|---|---|---|
| SingleQuantHardware=NVIDIA A800 80GB GPU, Number of trials=302025.11 | 27 | |
| Du QuantHardware=NVIDIA A800 80GB GPU, Number of trials=302025.11 | 60 | |
| OST QuantHardware=NVIDIA A800 80GB GPU, Number of trials=302025.11 | 2,226 | |
| Spin QuantHardware=NVIDIA A800 80GB GPU, Number of trials=302025.11 | 24,849 |