Commonsense Reasoning on Commonsense Reasoning LLaMA2-7B
79.68Average AccuracyDoRA
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| DoRA%Params=0.84, %C=66.722024.06 | 79.68 | 71.8 | 83.7 | 83.7 | 68.2 | 82.6 | 82.4 | 89.1 | 76 | |
| SHiRA-SNIP%Params=1.0, %C=1.02024.06 | 79.28 | 70.42 | 81.71 | 83.25 | 68.6 | 80.51 | 81 | 89.78 | 79.01 | |
| LoRA%Params=0.83, %C=66.722024.06 | 77.61 | 69.9 | 79.9 | 79.8 | 64.7 | 82.6 | 81 | 83.6 | 79.5 | |
| FP16Bits=16, Group=-, Evaluation Protocol=Zero-shot2024.07 | 64.86 | — | — | — | — | — | — | — | — | |
| EfficientQATBits=3, Group=128, Evaluation Protocol=Zero-shot2024.07 | 64.02 | — | — | — | — | — | — | — | — | |
| AutoRoundBits=3, Group=128, Evaluation Protocol=Zero-shot2024.07 | 63.72 | — | — | — | — | — | — | — | — | |
| QuIP#Bits=3, Group=-, Evaluation Protocol=Zero-shot2024.07 | 63.52 | — | — | — | — | — | — | — | — | |
| AWQBits=3, Group=128, Evaluation Protocol=Zero-shot2024.07 | 62.82 | — | — | — | — | — | — | — | — | |
| GPTQBits=3, Group=128, Evaluation Protocol=Zero-shot2024.07 | 62.48 | — | — | — | — | — | — | — | — | |
| OmniQBits=3, Group=128, Evaluation Protocol=Zero-shot2024.07 | 62.42 | — | — | — | — | — | — | — | — | |
| RTNBits=3, Group=128, Evaluation Protocol=Zero-shot2024.07 | 62.06 | — | — | — | — | — | — | — | — | |
| QuIP#Bits=2, Group=-, Evaluation Protocol=Zero-shot2024.07 | 60.61 | — | — | — | — | — | — | — | — | |
| EfficientQATBits=2, Group=64, Evaluation Protocol=Zero-shot2024.07 | 60.14 | — | — | — | — | — | — | — | — | |
| EfficientQATBits=2, Group=128, Evaluation Protocol=Zero-shot2024.07 | 59.5 | — | — | — | — | — | — | — | — | |
| AQLMBits=2, Group=2x8, Evaluation Protocol=Zero-shot2024.07 | 57.61 | — | — | — | — | — | — | — | — | |
| DB-LLMBits=2, Group=64, Evaluation Protocol=Zero-shot2024.07 | 56.93 | — | — | — | — | — | — | — | — | |
| AutoRoundBits=2, Group=128, Evaluation Protocol=Zero-shot2024.07 | 54.5 | — | — | — | — | — | — | — | — | |
| OmniQBits=2, Group=128, Evaluation Protocol=Zero-shot2024.07 | 46.98 | — | — | — | — | — | — | — | — |