List your company
ResearchFormerly WizWand

Commonsense Reasoning on WinoGrande standard (test)

80.2AccuracyLLAMA 2
51.70459.10266.573.898Apr 3, 2023Oct 6, 2023Apr 10, 2024Oct 14, 2024Apr 18, 2025Oct 22, 2025Apr 27, 2026
Updated 6mo ago

Evaluation results

MethodLinks
LLAMA 2Size=70B2023.0780.2
LLAMA 1Size=65B2023.0777
FalconSize=40B2023.0776.9
LLAMA 2Size=34B2023.0776.7
LLAMA 1Size=33B2023.0776
FP16Model=LLaMA-2-13B, Quantization=None, Calibration=FP162026.0476
COVERCALModel=LLaMA-2-13B, Quantization=AWQ INT4, Calibration Samples=128, Calibration=COVERCAL2026.0475.6
Max-ActVarModel=LLaMA-2-13B, Quantization=AWQ INT4, Calibration Samples=128, Calibration=Max-ActVar2026.0475
RandomModel=LLaMA-2-13B, Quantization=AWQ INT4, Calibration Samples=128, Calibration=Random2026.0474.7
LLAMA 1Size=13B2023.0773
LLAMA 2Size=13B2023.0772.8
MPTSize=30B2023.0771
LLAMA 1Size=7B2023.0770.1
LLAMA 2Size=7B2023.0769.2
Dip-SVDRatio=0.2, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=true, Zero-shot evaluation=true2026.0269.2
SVD-LLMRatio=0.2, Model=LLaMA-13B, Fine-tuning=true, Mixed-rank strategies=false, Zero-shot evaluation=true2026.0268.4
SAES-SVDRatio=0.2, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=false, Zero-shot evaluation=true2026.0268.4
MPTSize=7B2023.0768.3
AUSteer-HeadModel=Gemma2-27B-it2026.0267.25
FalconSize=7B2023.0766.3
AUSteer-FFNModel=Gemma2-27B-it2026.0266.3
SAES-SVDRatio=0.4, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=false, Zero-shot evaluation=true2026.0265.4
Dip-SVDRatio=0.4, Model=LLaMA-13B, Fine-tuning=false, Mixed-rank strategies=true, Zero-shot evaluation=true2026.0264.4
Pythia-12BParameters=12B, Zero-shot=true2023.0463.9
SVD-LLMRatio=0.4, Model=LLaMA-13B, Fine-tuning=true, Mixed-rank strategies=false, Zero-shot evaluation=true2026.0263.9
VanillaModel=Gemma2-27B-it2026.0263.61
AUSteer-FFNModel=LLaMA3.1-8B-Instruct2026.0261.56
Pythia-6.9BParameters=6.9B, Zero-shot=true2023.0460.9
AUSteer-HeadModel=LLaMA3.1-8B-Instruct2026.0260.38
AUSteer-HeadModel=LLaMA2-13B-Chat2026.0259.43
Pythia-2.8BParameters=2.8B, Zero-shot=true2023.0459.4
AUSteer-FFNModel=LLaMA2-13B-Chat2026.0258.88
VanillaModel=LLaMA3.1-8B-Instruct2026.0257.77
Pythia-1.4BParameters=1.4B, Zero-shot=true2023.0457.3
VanillaModel=LLaMA2-13B-Chat2026.0253.99
Pythia-410MParameters=410M, Zero-shot=true2023.0453.7
Pythia-1BParameters=1B, Zero-shot=true2023.0453.7
Pythia-160MParameters=160M, Zero-shot=true2023.0453.1
Pythia-70MParameters=70M, Zero-shot=true2023.0452.8