Physical Commonsense Reasoning on PIQA (Accuracy and Performance Gain)
7,497AccuracyHPTQ
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HPTQAvg. Bitwidth=3.125, Zero-shot=true2025.07 | 7,497 | — | |
| Llama 3.1 8BClassifier=Self-labeled2026.01 | 91 | 9.7 | |
| Llama 3.1 8BClassifier=Majority-labeled2026.01 | 90.9 | 9.7 | |
| Mistral Nemo Base 2407Classifier=Self-labeled2026.01 | 89.5 | 7.3 | |
| Mistral Nemo Base 2407Classifier=Majority-labeled2026.01 | 89.2 | 7 | |
| Mistral 7B v0.3Classifier=Self-labeled2026.01 | 88.8 | 7.6 | |
| Mistral 7B v0.3Classifier=Majority-labeled2026.01 | 88.6 | 7.4 | |
| Llama 3.2 1BClassifier=Self-labeled2026.01 | 88.4 | 11.7 | |
| Llama 3.2 1BClassifier=Majority-labeled2026.01 | 88.4 | 11.7 | |
| Qwen3 8B BaseClassifier=Self-labeled2026.01 | 88.2 | 8.8 | |
| Qwen3 8B BaseClassifier=Majority-labeled2026.01 | 87.8 | 8.3 | |
| Qwen3 0.6BClassifier=Self-labeled2026.01 | 82.6 | 15.9 | |
| Qwen3 0.6BClassifier=Majority-labeled2026.01 | 82.5 | 15.8 | |
| SSQR-5%Avg. Bitwidth=5.725, Zero-shot=true2025.07 | 80.52 | — | |
| BF16 BaselineAvg. Bitwidth=16.000, Zero-shot=true2025.07 | 80.14 | — | |
| SSQR-4%Avg. Bitwidth=5.405, Zero-shot=true2025.07 | 79.92 | — | |
| SSQR-1%Avg. Bitwidth=4.445, Zero-shot=true2025.07 | 79.65 | — | |
| SSQR-3%Avg. Bitwidth=5.085, Zero-shot=true2025.07 | 79.54 | — | |
| HPTQAvg. Bitwidth=4.125, Zero-shot=true2025.07 | 79.49 | — | |
| SSQR-2%Avg. Bitwidth=4.765, Zero-shot=true2025.07 | 79.49 | — | |
| GPTQAvg. Bitwidth=4.125, Zero-shot=true2025.07 | 79.27 | — | |
| HPTQAvg. Bitwidth=3.125, Zero-shot=true2025.07 | 79.16 | — | |
| SSQR-3%Avg. Bitwidth=4.085, Zero-shot=true2025.07 | 79.11 | — | |
| SSQR-5%Avg. Bitwidth=4.725, Zero-shot=true2025.07 | 79.05 | — | |
| SSQR-2%Avg. Bitwidth=3.765, Zero-shot=true2025.07 | 78.78 | — | |
| SSQR-4%Avg. Bitwidth=4.405, Zero-shot=true2025.07 | 78.73 | — | |
| SSQR-1%Avg. Bitwidth=3.445, Zero-shot=true2025.07 | 77.97 | — | |
| GPTQAvg. Bitwidth=3.125, Zero-shot=true2025.07 | 77.37 | — | |
| SSQR-3%Avg. Bitwidth=5.085, Zero-shot=true2025.07 | 76.82 | — | |
| HPTQAvg. Bitwidth=4.125, Zero-shot=true2025.07 | 76.55 | — | |
| SSQR-2%Avg. Bitwidth=4.765, Zero-shot=true2025.07 | 76.33 | — | |
| SSQR-4%Avg. Bitwidth=5.405, Zero-shot=true2025.07 | 76.28 | — | |
| BF16 BaselineAvg. Bitwidth=16.000, Zero-shot=true2025.07 | 76.17 | — | |
| SSQR-1%Avg. Bitwidth=4.445, Zero-shot=true2025.07 | 76.12 | — | |
| GPTQAvg. Bitwidth=4.125, Zero-shot=true2025.07 | 75.73 | — | |
| SSQR-5%Avg. Bitwidth=5.725, Zero-shot=true2025.07 | 75.57 | — | |
| SSQR-5%Avg. Bitwidth=4.725, Zero-shot=true2025.07 | 75.52 | — | |
| SSQR-3%Avg. Bitwidth=4.085, Zero-shot=true2025.07 | 75.03 | — | |
| SSQR-4%Avg. Bitwidth=4.405, Zero-shot=true2025.07 | 75.03 | — | |
| SSQR-1%Avg. Bitwidth=3.445, Zero-shot=true2025.07 | 74.92 | — | |
| SSQR-2%Avg. Bitwidth=3.765, Zero-shot=true2025.07 | 74.43 | — | |
| SSQR-5%Avg. Bitwidth=3.725, Zero-shot=true2025.07 | 72.58 | — | |
| GPTQAvg. Bitwidth=3.125, Zero-shot=true2025.07 | 72.58 | — | |
| HPTQAvg. Bitwidth=2.125, Zero-shot=true2025.07 | 69.15 | — | |
| HPTQAvg. Bitwidth=2.125, Zero-shot=true2025.07 | 69.15 | — | |
| SSQR-4%Avg. Bitwidth=3.405, Zero-shot=true2025.07 | 66.49 | — | |
| SSQR-3%Avg. Bitwidth=3.085, Zero-shot=true2025.07 | 65.61 | — | |
| SSQR-5%Avg. Bitwidth=3.725, Zero-shot=true2025.07 | 65.56 | — | |
| SSQR-2%Avg. Bitwidth=2.765, Zero-shot=true2025.07 | 61.04 | — | |
| SSQR-2%Avg. Bitwidth=2.765, Zero-shot=true2025.07 | 60.01 | — | |
| SSQR-3%Avg. Bitwidth=3.085, Zero-shot=true2025.07 | 58.81 | — | |
| SSQR-1%Avg. Bitwidth=2.445, Zero-shot=true2025.07 | 58.49 | — | |
| SSQR-1%Avg. Bitwidth=2.445, Zero-shot=true2025.07 | 58.22 | — | |
| SSQR-4%Avg. Bitwidth=3.405, Zero-shot=true2025.07 | 54.95 | — | |
| GPTQAvg. Bitwidth=2.125, Zero-shot=true2025.07 | 54.46 | — | |
| GPTQAvg. Bitwidth=2.125, Zero-shot=true2025.07 | 53.65 | — |