Mathematical Reasoning on GSM8K @MetaMath (test)
72.3AccuracyBF16
Evaluation Results
| Method | Links | |
|---|---|---|
| BF16Model Size=13B, Bits=16, Backbone=MetaMath2024.02 | 72.3 | |
| BitDistillerModel Size=13B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 69.69 | |
| GPTQModel Size=13B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 68.75 | |
| AWQModel Size=13B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 68.67 | |
| RTNModel Size=13B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 68.51 | |
| OmniQuantModel Size=13B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 68.28 | |
| LLM-QATModel Size=13B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 66.62 | |
| BF16Model Size=7B, Bits=16, Backbone=MetaMath2024.02 | 66.41 | |
| BitDistillerModel Size=7B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 64.38 | |
| AWQModel Size=7B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 62.34 | |
| GPTQModel Size=7B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 62.11 | |
| OmniQuantModel Size=7B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 61.7 | |
| BitDistillerModel Size=13B, Bits=2, Group Size=128, Backbone=MetaMath2024.02 | 61.33 | |
| LLM-QATModel Size=7B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 60.78 | |
| RTNModel Size=7B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 59.3 | |
| BitDistillerModel Size=7B, Bits=2, Group Size=128, Backbone=MetaMath2024.02 | 51.02 | |
| LLM-QATModel Size=13B, Bits=2, Group Size=128, Backbone=MetaMath2024.02 | 36.64 | |
| BF16Model Size=3B, Bits=16, Backbone=MetaMath2024.02 | 36.4 | |
| BitDistillerModel Size=3B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 32.5 | |
| LLM-QATModel Size=3B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 26.25 | |
| OmniQuantModel Size=3B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 23.67 | |
| LLM-QATModel Size=7B, Bits=2, Group Size=128, Backbone=MetaMath2024.02 | 23.13 | |
| AWQModel Size=3B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 21.87 | |
| RTNModel Size=3B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 17.5 | |
| BitDistillerModel Size=3B, Bits=2, Group Size=128, Backbone=MetaMath2024.02 | 16.09 | |
| GPTQModel Size=13B, Bits=2, Group Size=128, Backbone=MetaMath2024.02 | 11.43 | |
| OmniQuantModel Size=13B, Bits=2, Group Size=128, Backbone=MetaMath2024.02 | 9.45 | |
| RTNModel Size=13B, Bits=2, Group Size=128, Backbone=MetaMath2024.02 | 7.89 | |
| AWQModel Size=13B, Bits=2, Group Size=128, Backbone=MetaMath2024.02 | 7.89 | |
| GPTQModel Size=3B, Bits=3, Group Size=128, Backbone=MetaMath2024.02 | 6.72 | |
| LLM-QATModel Size=3B, Bits=2, Group Size=128, Backbone=MetaMath2024.02 | 6.56 |