Language Understanding and Reasoning on MMLU, PIQA, HellaSwag, WinoGrande, ARC-Challenge
46.45MMLU (5s)BF16
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| BF16Format=BF16, Backbone=LLaMA-2-7B2024.02 | 46.45 | 77.86 | 57.14 | 68.35 | 43.34 | 58.63 | |
| BitDistillerQuantization Bits=3 Bits, Group Size=g128, Backbone=LLaMA-2-7B2024.02 | 43.65 | 76.99 | 55.38 | 68.35 | 41.21 | 57.12 | |
| LLM-QATQuantization Bits=3 Bits, Group Size=g128, Backbone=LLaMA-2-7B2024.02 | 41.32 | 77.26 | 54.74 | 68.35 | 40.61 | 56.46 | |
| OmniQuantQuantization Bits=3 Bits, Group Size=g128, Backbone=LLaMA-2-7B2024.02 | 41.22 | 77.47 | 54.41 | 67.09 | 39.08 | 55.85 | |
| AWQQuantization Bits=3 Bits, Group Size=g128, Backbone=LLaMA-2-7B2024.02 | 39.68 | 76.27 | 55.14 | 67.56 | 40.61 | 55.85 | |
| GPTQQuantization Bits=3 Bits, Group Size=g128, Backbone=LLaMA-2-7B2024.02 | 39.57 | 75.46 | 51.68 | 67.16 | 38.39 | 54.45 | |
| RTNQuantization Bits=3 Bits, Group Size=g128, Backbone=LLaMA-2-7B2024.02 | 38.65 | 75.24 | 53.7 | 67.32 | 38.56 | 54.69 | |
| BitDistillerQuantization Bits=2 Bits, Group Size=g128, Backbone=LLaMA-2-7B2024.02 | 29.25 | 73.61 | 48.7 | 61.09 | 33.27 | 49.18 | |
| OmniQuantQuantization Bits=2 Bits, Group Size=g128, Backbone=LLaMA-2-7B2024.02 | 25.42 | 58.92 | 29.2 | 50.83 | 19.45 | 36.76 | |
| AWQQuantization Bits=2 Bits, Group Size=g128, Backbone=LLaMA-2-7B2024.02 | 25.38 | 52.39 | 25.7 | 50.12 | 21.33 | 34.98 | |
| RTNQuantization Bits=2 Bits, Group Size=g128, Backbone=LLaMA-2-7B2024.02 | 24.12 | 53.43 | 26.33 | 49.96 | 21.58 | 35.08 | |
| LLM-QATQuantization Bits=2 Bits, Group Size=g128, Backbone=LLaMA-2-7B2024.02 | 23.62 | 70.08 | 43.79 | 61.64 | 29.09 | 45.64 | |
| GPTQQuantization Bits=2 Bits, Group Size=g128, Backbone=LLaMA-2-7B2024.02 | 23.12 | 49.51 | 25.04 | 49.57 | 22.69 | 33.99 |