General Language Understanding on WikiText-2, MMLU, PIQA, HellaSwag, WinoGrande, and ARC-Challenge
4.88PPLBF16
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| BF16Backbone=LLaMA-2-13B, Precision=BF162024.02 | 4.88 | 55.54 | 79.16 | 60.13 | 72.14 | 48.12 | 63.02 | |
| BitDistillerBackbone=LLaMA-2-13B, Bits=3, Group Size=g1282024.02 | 5.2 | 53.21 | 78.67 | 58.66 | 71.59 | 46.67 | 61.76 | |
| LLM-QATBackbone=LLaMA-2-13B, Bits=3, Group Size=g1282024.02 | 5.32 | 51.6 | 78.29 | 58.45 | 70.56 | 44.62 | 60.7 | |
| GPTQBackbone=LLaMA-2-13B, Bits=3, Group Size=g1282024.02 | 5.41 | 50.63 | 77.26 | 56.84 | 70.72 | 42.83 | 59.66 | |
| AWQBackbone=LLaMA-2-13B, Bits=3, Group Size=g1282024.02 | 5.47 | 49.64 | 77.09 | 57.52 | 70.32 | 43.86 | 59.69 | |
| OmniQuantBackbone=LLaMA-2-13B, Bits=3, Group Size=g1282024.02 | 5.48 | 48.97 | 77.64 | 57.08 | 70.88 | 44.28 | 59.77 | |
| RTNBackbone=LLaMA-2-13B, Bits=3, Group Size=g1282024.02 | 5.52 | 50.74 | 78.35 | 57.75 | 71.11 | 43.86 | 60.36 | |
| BitDistillerBackbone=LLaMA-2-13B, Bits=2, Group Size=g1282024.02 | 6.78 | 37.5 | 75.84 | 51.3 | 65.9 | 37.46 | 53.6 | |
| LLM-QATBackbone=LLaMA-2-13B, Bits=2, Group Size=g1282024.02 | 7.8 | 29.37 | 74.1 | 49.49 | 63.14 | 33.87 | 49.99 | |
| GPTQBackbone=LLaMA-2-13B, Bits=2, Group Size=g1282024.02 | 15.08 | 23.7 | 56.04 | 30.99 | 51.22 | 19.28 | 36.25 | |
| OmniQuantBackbone=LLaMA-2-13B, Bits=2, Group Size=g1282024.02 | 25.69 | 26.09 | 61.81 | 31.92 | 51.38 | 22.27 | 38.69 | |
| RTNBackbone=LLaMA-2-13B, Bits=2, Group Size=g1282024.02 | 109.21 | 24.74 | 57.56 | 32.56 | 50.75 | 21.84 | 37.49 | |
| AWQBackbone=LLaMA-2-13B, Bits=2, Group Size=g1282024.02 | 120,000 | 27.04 | 53.16 | 25.82 | 51.7 | 23.04 | 36.15 |