Zero-shot Reasoning Suite (PiQA, ARC-E/C, HellaSwag, WinoGrande, BoolQ)
80.7PiQA Accuracy (Zero-shot)FP
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| FPModel=Llama3-8b, W/A=FP, Evaluation Protocol=Zero-shot2026.05 | 80.7 | 77.7 | 53.7 | 79.1 | 73.2 | 81.1 | 74.3 | |
| FPModel=Llama2-13b, W/A=FP, Evaluation Protocol=Zero-shot2026.05 | 80.5 | 77.5 | 49.2 | 79.4 | 72.4 | 80.6 | 73.3 | |
| ResComp-MARRModel=Llama2-13b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 79.38 | 75.17 | 47.78 | 77.11 | 70.96 | 78.35 | 71.46 | |
| FPModel=Llama2-7b, W/A=FP, Evaluation Protocol=Zero-shot2026.05 | 79 | 74.6 | 46.5 | 76 | 68.9 | 77.7 | 70.5 | |
| GPTAQ-MARRModel=Llama2-13b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 78.73 | 73.99 | 46.84 | 77.27 | 70.48 | 77.46 | 70.79 | |
| ResCompModel=Llama2-13b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 78.67 | 74.45 | 47.87 | 77.1 | 70.32 | 80.03 | 71.41 | |
| GPTQModel=Llama2-13b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 78.24 | 74.83 | 46.67 | 76.9 | 69.53 | 79.14 | 70.88 | |
| GPTAQModel=Llama2-13b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 78.24 | 73.91 | 46.84 | 77.1 | 69.46 | 78.23 | 70.63 | |
| GPTAQModel=Llama3-8b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 78.24 | 71.8 | 46.76 | 74.94 | 67.88 | 77.86 | 69.58 | |
| ResComp-MARRModel=Llama3-8b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 78.18 | 74.62 | 46.16 | 74.97 | 67.4 | 76.7 | 69.67 | |
| ResCompModel=Llama3-8b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 78.02 | 72.22 | 47.44 | 74.72 | 67.64 | 76.79 | 69.47 | |
| GPTAQ-MARRModel=Llama3-8b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 77.97 | 73.95 | 47.35 | 75.17 | 68.67 | 76.97 | 70.01 | |
| ResCompModel=Llama2-7b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 77.86 | 70.46 | 42.69 | 73.72 | 65.23 | 73.06 | 67.17 | |
| GPTQModel=Llama2-7b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 77.69 | 71.76 | 42.49 | 73.17 | 65.04 | 75.32 | 67.58 | |
| RTNModel=Llama2-13b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 77.37 | 74.03 | 44.8 | 73.73 | 68.82 | 76.09 | 69.14 | |
| GPTAQ-MARRModel=Llama2-7b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 77.04 | 71.46 | 43.77 | 73.57 | 67.96 | 73.85 | 67.94 | |
| ResComp-MARRModel=Llama2-7b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 76.88 | 69.91 | 42.24 | 73.7 | 66.06 | 74.71 | 67.25 | |
| GPTAQModel=Llama2-7b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 76.5 | 70.37 | 41.98 | 73.52 | 65.9 | 73.64 | 66.98 | |
| GPTQModel=Llama3-8b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 75.73 | 72.18 | 44.28 | 73.81 | 66.69 | 76.15 | 68.14 | |
| RTNModel=Llama2-7b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 74.16 | 62.84 | 36.35 | 66.24 | 61.17 | 69.79 | 61.76 | |
| RTNModel=Llama3-8b, W/A=W4A4, Evaluation Protocol=Zero-shot2026.05 | 74.1 | 65.28 | 40.02 | 70.26 | 65.19 | 74.4 | 64.88 | |
| GPTAQModel=Llama2-13b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 67.03 | 54.83 | 31.08 | 48.54 | 56.31 | 62.01 | 53.3 | |
| GPTAQ-MARRModel=Llama2-13b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 66.38 | 51.39 | 30.38 | 51.93 | 59.04 | 64.16 | 53.88 | |
| ResComp-MARRModel=Llama2-13b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 64.74 | 48.27 | 28.75 | 48.67 | 57.62 | 62.91 | 51.83 | |
| ResCompModel=Llama2-13b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 63.76 | 51.76 | 26.55 | 47.65 | 54.22 | 60.29 | 50.7 | |
| GPTAQModel=Llama2-7b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 62.13 | 44.15 | 25.68 | 42.7 | 52.64 | 58.59 | 47.65 | |
| GPTAQ-MARRModel=Llama2-7b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 61.48 | 43.9 | 25.26 | 43.84 | 57.38 | 61.31 | 48.86 | |
| ResComp-MARRModel=Llama2-7b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 60.94 | 43.56 | 25.17 | 42.81 | 55.49 | 62.05 | 48.34 | |
| GPTQModel=Llama2-13b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 60.94 | 41.29 | 24.49 | 41.35 | 53.35 | 62.05 | 47.24 | |
| ResCompModel=Llama2-7b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 60.79 | 44.17 | 25.02 | 41.79 | 53.41 | 62.06 | 47.87 | |
| GPTAQ-MARRModel=Llama3-8b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 59.79 | 39.35 | 25.26 | 40.58 | 54.14 | 58.9 | 46.34 | |
| GPTAQModel=Llama3-8b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 58.7 | 40.49 | 23.21 | 38.96 | 54.22 | 62.29 | 46.31 | |
| ResCompModel=Llama3-8b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 58.25 | 38.32 | 23.74 | 38.54 | 53.2 | 61.59 | 45.44 | |
| ResComp-MARRModel=Llama3-8b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 57.56 | 37.37 | 23.98 | 38.35 | 55.25 | 61.5 | 45.67 | |
| GPTQModel=Llama2-7b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 56.8 | 33.38 | 21.76 | 31.77 | 52.33 | 49.79 | 40.97 | |
| GPTQModel=Llama3-8b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 53.48 | 32.95 | 21.25 | 30.09 | 50.75 | 46.12 | 39.11 | |
| RTNModel=Llama3-8b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 51.09 | 25.63 | 26.62 | 26.87 | 50.2 | 43.15 | 37.26 | |
| RTNModel=Llama2-7b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 50.76 | 25.59 | 28.58 | 26.38 | 48.54 | 37.83 | 36.28 | |
| RTNModel=Llama2-13b, W/A=W2A4, Evaluation Protocol=Zero-shot2026.05 | 48.59 | 27.19 | 27.13 | 25.02 | 51.78 | 37.83 | 36.26 |