Zero-shot Evaluation on Reasoning Suite (BoolQ, PIQA, Hellaswag, Winogrande, ARC-e, ARC-c, OBQA)
77.7Accuracy (BoolQ)LLaMA2
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| LLaMA2Size=7B, Bit=16, Zero-shot=true2025.08 | 77.7 | 79.1 | 76 | 69.1 | 74.6 | 46.2 | 44.2 | 66.7 | |
| BinaryLLMSize=7B, Bit=1.01, Zero-shot=true2025.08 | 65.5 | 73.2 | 59.5 | 66.4 | 60.3 | 37.2 | 34.5 | 56.7 | |
| MoSSize=7B, Bit=1.01, Zero-shot=true2025.08 | 65 | 71.6 | 59.4 | 56.2 | 41.8 | 30 | — | — | |
| OneBit-LLaMA2Size=7B, Bit=1.01, Zero-shot=true2025.08 | 63.1 | 68.1 | 52.6 | 58.4 | 41.6 | 29.6 | — | — | |
| FBI-LLMSize=7B, Bit=1.01, Zero-shot=true2025.08 | 61.5 | 72.6 | 57.7 | 58.9 | 53 | 29.9 | 36.8 | 52.9 |