Commonsense Question Answering on CSQA (Multi-Dataset Suite)
84.06PIQAQTALE
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| QTALEModel=LLaMA2-7B, Bits=16, Layer Execution=QTALE, Evaluation Protocol=Zero-shot2026.02 | 84.06 | 88.22 | 78.97 | 81.65 | 65.7 | 81.45 | 82.4 | 80.35 | |
| FullModel=LLaMA2-7B, Bits=16, Layer Execution=Full, Evaluation Protocol=Zero-shot2026.02 | 83.73 | 87.98 | 79.58 | 82.53 | 65.27 | 81.61 | 83 | 80.53 | |
| D-LLMModel=LLaMA2-7B, Bits=16, Layer Execution=D-LLM, Evaluation Protocol=Zero-shot2026.02 | 83.51 | 88.17 | 79.02 | 81.06 | 66.04 | 81.22 | 81.4 | 80.06 | |
| QTALEModel=LLaMA2-7B, Bits=4, Layer Execution=QTALE, Evaluation Protocol=Zero-shot2026.02 | 83.3 | 87.67 | 78.56 | 79.59 | 66.21 | 79.95 | 80.2 | 79.18 | |
| FullModel=LLaMA2-7B, Bits=4, Layer Execution=Full, Evaluation Protocol=Zero-shot2026.02 | 81.34 | 87.67 | 79.53 | 79.97 | 62.37 | 81.14 | 79.4 | 78.77 | |
| D-LLMModel=LLaMA2-7B, Bits=4, Layer Execution=D-LLM, Evaluation Protocol=Zero-shot2026.02 | 81.23 | 86.2 | 77.18 | 79.08 | 62.03 | 78.14 | 77.4 | 77.32 | |
| FullModel=LLaMA3.1-8B, Bits=16, Layer Execution=Full, Evaluation Protocol=Zero-shot2026.02 | 80.04 | 88.19 | 88.9 | 87.58 | 77.03 | 84.21 | 85.2 | 81.28 | |
| D-LLMModel=LLaMA3.1-8B, Bits=16, Layer Execution=D-LLM, Evaluation Protocol=Zero-shot2026.02 | 79.84 | 86.02 | 89.35 | 86.24 | 75.68 | 83.43 | 84.2 | 80.45 | |
| FullModel=LLaMA3.1-8B, Bits=4, Layer Execution=Full, Evaluation Protocol=Zero-shot2026.02 | 79.53 | 86.29 | 87.52 | 86.07 | 75.17 | 83.58 | 83 | 79.67 | |
| D-LLMModel=LLaMA3.1-8B, Bits=4, Layer Execution=D-LLM, Evaluation Protocol=Zero-shot2026.02 | 79.27 | 83.57 | 86.88 | 84.93 | 69.88 | 80.51 | 81 | 77.68 | |
| QTALEModel=LLaMA3.1-8B, Bits=4, Layer Execution=QTALE, Evaluation Protocol=Zero-shot2026.02 | 79.27 | 85.26 | 88.13 | 85.56 | 74.83 | 82.08 | 82.4 | 79.17 | |
| QTALEModel=LLaMA3.1-8B, Bits=16, Layer Execution=QTALE, Evaluation Protocol=Zero-shot2026.02 | 78.81 | 86.18 | 87.37 | 87.16 | 78.16 | 83.43 | 84.8 | 80.54 | |
| FullModel=LLaMA2-7B, Bits=3, Layer Execution=Full, Evaluation Protocol=Zero-shot2026.02 | 78.02 | 83.76 | 74.36 | 71.38 | 55.29 | 74.51 | 68.2 | 72.22 | |
| QTALEModel=LLaMA2-7B, Bits=3, Layer Execution=QTALE, Evaluation Protocol=Zero-shot2026.02 | 77.58 | 83.82 | 73.34 | 73.44 | 57.17 | 74.98 | 69.2 | 72.79 | |
| D-LLMModel=LLaMA2-7B, Bits=3, Layer Execution=D-LLM, Evaluation Protocol=Zero-shot2026.02 | 74.65 | 80.02 | 73.34 | 71.55 | 55.03 | 74.43 | 65 | 70.57 | |
| FullModel=LLaMA3.1-8B, Bits=3, Layer Execution=Full, Evaluation Protocol=Zero-shot2026.02 | 72.11 | 79.65 | 81.58 | 77.61 | 59.9 | 71.27 | 69.6 | 69.54 | |
| QTALEModel=LLaMA3.1-8B, Bits=3, Layer Execution=QTALE, Evaluation Protocol=Zero-shot2026.02 | 69.96 | 77.2 | 78.98 | 77.57 | 61.52 | 75.06 | 71.8 | 69.65 | |
| D-LLMModel=LLaMA3.1-8B, Bits=3, Layer Execution=D-LLM, Evaluation Protocol=Zero-shot2026.02 | 68.99 | 76.88 | 77.33 | 76.09 | 55.29 | 71.19 | 65.2 | 66.81 |