List your company
ResearchFormerly WizWand

Commonsense Reasoning on Winogrande (Average Score)

73.88Winogrande ScoreDPO
48.524855.107461.6968.2726Sep 27, 2025Nov 5, 2025Dec 14, 2025Jan 22, 2026Mar 2, 2026Apr 10, 2026May 19, 2026
Updated 5mo ago

Evaluation results

MethodLinks
DPO2025.0973.8870.68
SFT Model2025.0973.7270.28
SPPO2025.0973.4870.19
INPO2025.0973.4870.25
HT-MNPOReward Model=ArmoRM-Llama32025.0973.4870.83
HT-MNPOReward Model=Skywork-Reward-V22025.0973.471.8
HT-MNPOReward Model=Athene-RM-8B2025.0973.3270.99
TD-MNPO2025.0973.0971.08
SimPO2025.0972.9369.6
FP32Model=LLaMA3-8B, W-A-KV=–, W. Gran.=–2026.0572.85—
Quant.npuModel=LLaMA3-8B, W-A-KV=4-8-8, W. Gran.=per-channel2026.0571.9—
executorchModel=LLaMA3-8B, W-A-KV=4-16-8, W. Gran.=per-block2026.0571.82—
QuaRotModel=LLaMA3-8B, W-A-KV=4-8-8, W. Gran.=per-channel2026.0568.19—
executorchModel=LLaMA3-8B, W-A-KV=4-8-8, W. Gran.=per-channel2026.0567.56—
SpinQuantModel=LLaMA3-8B, W-A-KV=4-8-8, W. Gran.=per-channel2026.0564.48—
Echo-180MParams=180M, Tokens=10B, Zero-shot=true2026.0555.4—
Mamba-3-MIMO-180MParams=180M, Tokens=100B, Zero-shot=true2026.0552.9—
Mamba-2-180MParams=180M, Tokens=100B, Zero-shot=true2026.0552—
Mamba-3-SISO-180MParams=180M, Tokens=100B, Zero-shot=true2026.0552—
GDN-180MParams=180M, Tokens=100B, Zero-shot=true2026.0551.7—
Transformer-180MParams=180M, Tokens=100B, Zero-shot=true2026.0551.2—
Echo-50MParams=50M, Tokens=3B, Zero-shot=true2026.0549.5—