Block-wise quantization complexity analysis on OPT Models
0.24GFLOPSaespa
Evaluation results
| Method | Links | |
|---|---|---|
| aespaModel size=125M2024.02 | 0.24 | |
| aespaModel size=350M2024.02 | 0.42 | |
| aespaModel size=1.3B2024.02 | 1.6 | |
| aespaModel size=2.7B2024.02 | 3.2 | |
| Conventional block-wise quantizationModel size=125M, Batch size (B)=42024.02 | 6.7 | |
| Conventional block-wise quantizationModel size=350M, Batch size (B)=42024.02 | 7.5 | |
| Conventional block-wise quantizationModel size=1.3B, Batch size (B)=42024.02 | 11 | |
| aespaModel size=6.7B2024.02 | 13 | |
| Conventional block-wise quantizationModel size=2.7B, Batch size (B)=42024.02 | 15 | |
| aespaModel size=13B2024.02 | 20 | |
| Conventional block-wise quantizationModel size=6.7B, Batch size (B)=42024.02 | 34 | |
| Conventional block-wise quantizationModel size=13B, Batch size (B)=42024.02 | 41 |