Code Generation on HumanEval @WizardCoder (test)
71.95Pass@1BF16
Evaluation results
| Method | Links | |
|---|---|---|
| BF16Model Size=34B, Bits=16, Backbone=WizardCoder2024.02 | 71.95 | |
| BitDistillerModel Size=34B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 69.51 | |
| OmniQuantModel Size=34B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 68.9 | |
| AWQModel Size=34B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 67.07 | |
| LLM-QATModel Size=34B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 66.46 | |
| BitDistillerModel Size=13B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 63.41 | |
| GPTQModel Size=34B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 63.41 | |
| BF16Model Size=13B, Bits=16, Backbone=WizardCoder2024.02 | 62.8 | |
| LLM-QATModel Size=13B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 57.92 | |
| GPTQModel Size=13B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 55.48 | |
| BF16Model Size=7B, Bits=16, Backbone=WizardCoder2024.02 | 54.88 | |
| OmniQuantModel Size=13B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 54.88 | |
| BitDistillerModel Size=7B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 53.66 | |
| AWQModel Size=13B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 53.04 | |
| RTNModel Size=13B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 50 | |
| LLM-QATModel Size=7B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 48.78 | |
| GPTQModel Size=7B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 46.34 | |
| BitDistillerModel Size=34B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 46.34 | |
| AWQModel Size=7B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 45.73 | |
| OmniQuantModel Size=7B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 44.51 | |
| BitDistillerModel Size=13B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 42.07 | |
| BitDistillerModel Size=7B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 36.59 | |
| RTNModel Size=7B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 34.15 | |
| RTNModel Size=34B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 33.54 | |
| LLM-QATModel Size=34B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 29.27 | |
| OmniQuantModel Size=34B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 26.83 | |
| BF16Model Size=3B, Bits=16, Backbone=WizardCoder2024.02 | 23.17 | |
| BitDistillerModel Size=3B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 20.73 | |
| OmniQuantModel Size=13B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 20.12 | |
| LLM-QATModel Size=3B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 18.29 | |
| AWQModel Size=3B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 16.46 | |
| LLM-QATModel Size=13B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 15.21 | |
| LLM-QATModel Size=7B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 14.63 | |
| OmniQuantModel Size=3B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 10.36 | |
| BitDistillerModel Size=3B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 7.31 | |
| GPTQModel Size=3B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 4.3 | |
| RTNModel Size=3B, Bits=3, Group Size=128, Backbone=WizardCoder2024.02 | 4.27 | |
| GPTQModel Size=34B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 3.65 | |
| GPTQModel Size=13B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 1.83 | |
| RTNModel Size=34B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 0.61 | |
| RTNModel Size=3B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 0 | |
| GPTQModel Size=3B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 0 | |
| AWQModel Size=3B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 0 | |
| OmniQuantModel Size=3B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 0 | |
| LLM-QATModel Size=3B, Bits=2, Group Size=128, Backbone=WizardCoder2024.02 | 0 |