Common-sense Reasoning on 5 common-sense reasoning tasks (Llama-2-70B)
72.41Average AccuracyFP16
Evaluation results
| Method | Links | |
|---|---|---|
| FP16Bits=16, Group=-, Evaluation Protocol=Zero-shot2024.07 | 72.41 | |
| QuIP#Bits=3, Group=-, Evaluation Protocol=Zero-shot2024.07 | 72.13 | |
| EfficientQATBits=3, Group=128, Evaluation Protocol=Zero-shot2024.07 | 71.76 | |
| GPTQBits=3, Group=128, Evaluation Protocol=Zero-shot2024.07 | 71.47 | |
| AWQBits=3, Group=128, Evaluation Protocol=Zero-shot2024.07 | 71.41 | |
| AutoRoundBits=3, Group=128, Evaluation Protocol=Zero-shot2024.07 | 71.24 | |
| OmniQBits=3, Group=128, Evaluation Protocol=Zero-shot2024.07 | 71.07 | |
| QuIP#Bits=2, Group=-, Evaluation Protocol=Zero-shot2024.07 | 70.91 | |
| RTNBits=3, Group=128, Evaluation Protocol=Zero-shot2024.07 | 70.83 | |
| AQLMBits=2, Group=2x8, Evaluation Protocol=Zero-shot2024.07 | 69.85 | |
| EfficientQATBits=2, Group=64, Evaluation Protocol=Zero-shot2024.07 | 69.48 | |
| EfficientQATBits=2, Group=128, Evaluation Protocol=Zero-shot2024.07 | 68.93 | |
| DB-LLMBits=2, Group=64, Evaluation Protocol=Zero-shot2024.07 | 68.01 | |
| AutoRoundBits=2, Group=128, Evaluation Protocol=Zero-shot2024.07 | 67.7 | |
| OmniQBits=2, Group=128, Evaluation Protocol=Zero-shot2024.07 | 54.87 |