Commonsense Reasoning on WinoGrande standard (test)
80.2AccuracyLLAMA 2
Evaluation Results
| Method | Links | |
|---|---|---|
| LLAMA 2Size=70B2023.07 | 80.2 | |
| LLAMA 1Size=65B2023.07 | 77 | |
| FalconSize=40B2023.07 | 76.9 | |
| LLAMA 2Size=34B2023.07 | 76.7 | |
| LLAMA 1Size=33B2023.07 | 76 | |
| FP16Model=LLaMA-2-13B, Quantization=None, Calibration=FP162026.04 | 76 | |
| COVERCALModel=LLaMA-2-13B, Quantization=AWQ INT4, Calibration Samples=128, Calibration=COVERCAL2026.04 | 75.6 | |
| Max-ActVarModel=LLaMA-2-13B, Quantization=AWQ INT4, Calibration Samples=128, Calibration=Max-ActVar2026.04 | 75 | |
| RandomModel=LLaMA-2-13B, Quantization=AWQ INT4, Calibration Samples=128, Calibration=Random2026.04 | 74.7 | |
| LLAMA 1Size=13B2023.07 | 73 | |
| LLAMA 2Size=13B2023.07 | 72.8 | |
| MPTSize=30B2023.07 | 71 | |
| LLAMA 1Size=7B2023.07 | 70.1 | |
| LLAMA 2Size=7B2023.07 | 69.2 | |
| Dip-SVDRatio=0.2, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=true, Zero-shot evaluation=true2026.02 | 69.2 | |
| SVD-LLMRatio=0.2, Model=LLaMA-13B, Fine-tuning=true, Mixed-rank strategies=false, Zero-shot evaluation=true2026.02 | 68.4 | |
| SAES-SVDRatio=0.2, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=false, Zero-shot evaluation=true2026.02 | 68.4 | |
| MPTSize=7B2023.07 | 68.3 | |
| AUSteer-HeadModel=Gemma2-27B-it2026.02 | 67.25 | |
| FalconSize=7B2023.07 | 66.3 | |
| AUSteer-FFNModel=Gemma2-27B-it2026.02 | 66.3 | |
| SAES-SVDRatio=0.4, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=false, Zero-shot evaluation=true2026.02 | 65.4 | |
| Dip-SVDRatio=0.4, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=true, Zero-shot evaluation=true2026.02 | 64.4 | |
| Pythia-12BParameters=12B, Zero-shot=true2023.04 | 63.9 | |
| SVD-LLMRatio=0.4, Model=LLaMA-13B, Fine-tuning=true, Mixed-rank strategies=false, Zero-shot evaluation=true2026.02 | 63.9 | |
| VanillaModel=Gemma2-27B-it2026.02 | 63.61 | |
| AUSteer-FFNModel=LLaMA3.1-8B-Instruct2026.02 | 61.56 | |
| Pythia-6.9BParameters=6.9B, Zero-shot=true2023.04 | 60.9 | |
| AUSteer-HeadModel=LLaMA3.1-8B-Instruct2026.02 | 60.38 | |
| AUSteer-HeadModel=LLaMA2-13B-Chat2026.02 | 59.43 | |
| Pythia-2.8BParameters=2.8B, Zero-shot=true2023.04 | 59.4 | |
| AUSteer-FFNModel=LLaMA2-13B-Chat2026.02 | 58.88 | |
| VanillaModel=LLaMA3.1-8B-Instruct2026.02 | 57.77 | |
| Pythia-1.4BParameters=1.4B, Zero-shot=true2023.04 | 57.3 | |
| VanillaModel=LLaMA2-13B-Chat2026.02 | 53.99 | |
| Pythia-410MParameters=410M, Zero-shot=true2023.04 | 53.7 | |
| Pythia-1BParameters=1B, Zero-shot=true2023.04 | 53.7 | |
| Pythia-160MParameters=160M, Zero-shot=true2023.04 | 53.1 | |
| Pythia-70MParameters=70M, Zero-shot=true2023.04 | 52.8 |