INT2 Quantization on OPT-1.3B
4.49Memory Footprint (GB)OPTQ
Evaluation Results
| Method | Links | |
|---|---|---|
| OPTQTarget=layer-wise reconstruction, Hardware=NVIDIA A100 GPU2024.02 | 4.49 | |
| Z-FOLDTarget=layer-wise reconstruction, Hardware=NVIDIA A100 GPU2024.02 | 4.49 | |
| OPTQTarget=layer-wise reconstruction, Hardware=single NVIDIA A100 GPU2024.02 | 4.49 | |
| Z-FOLDTarget=layer-wise reconstruction, Hardware=single NVIDIA A100 GPU2024.02 | 4.49 | |
| aespaTarget=attention-wise reconstruction, Hardware=NVIDIA A100 GPU2024.02 | 5.47 | |
| aespaTarget=attention-wise reconstruction, Hardware=single NVIDIA A100 GPU2024.02 | 5.47 | |
| OmniQuantTarget=attention-wise reconstruction, Hardware=NVIDIA A100 GPU2024.02 | 5.87 | |
| OmniQuantTarget=attention-wise reconstruction, Hardware=single NVIDIA A100 GPU2024.02 | 5.87 | |
| AffineQuantTarget=attention-wise reconstruction, Hardware=NVIDIA A100 GPU2024.02 | 9.96 | |
| AffineQuantTarget=attention-wise reconstruction, Hardware=single NVIDIA A100 GPU2024.02 | 9.96 | |
| BRECQTarget=attention-wise reconstruction, Hardware=NVIDIA A100 GPU2024.02 | 16.6 | |
| BRECQTarget=attention-wise reconstruction, Hardware=single NVIDIA A100 GPU2024.02 | 16.6 |