Zero-Shot Question Answering on (BoolQ, Lambada, Piqa, OPQA, Winogrande, ARC-E, ARC-C, Hellaswag)
74.86BoolQ AccuracyARB-RC
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ARB-RCModel Size=LLaMA-2-13B, Bits=1.06, Protocol=zero-shot2025.12 | 74.86 | 66.23 | 71.16 | 33 | 62.04 | 57.37 | 33.36 | 50.46 | 56.06 | |
| OUR-RCModel Size=LLaMA-2-13B, Bits=1.06, Protocol=zero-shot2025.12 | 72.02 | 68.99 | 72.58 | 36 | 63.14 | 59.85 | 33.7 | 53.92 | 57.53 | |
| ARB-RCModel Size=LLaMA-2-7B, Bits=1.06, Protocol=zero-shot2025.12 | 67.74 | 51.87 | 65.51 | 29.8 | 59.98 | 46.8 | 28.24 | 48.1 | 49.75 | |
| ARB-RCModel Size=LLaMA-3-8B, Bits=1.06, Protocol=zero-shot2025.12 | 67.58 | 48.85 | 62.73 | 29.2 | 57.38 | 42.97 | 24.57 | 43.63 | 47.11 | |
| OUR-RCModel Size=LLaMA-3-8B, Bits=1.06, Protocol=zero-shot2025.12 | 66.54 | 49.35 | 63.33 | 30.4 | 56.2 | 45.66 | 26.02 | 44.21 | 47.71 | |
| OUR-RCModel Size=LLaMA-2-7B, Bits=1.06, Protocol=zero-shot2025.12 | 66.45 | 52.53 | 68.12 | 29.8 | 56.99 | 30.3 | 51.56 | 49.24 | 50.62 |